Re: Yahoo breaks every mailing list in the world including the IETF's
2014-05-15 16:46:29
Responding to John Levine, Brian Carpenter, Sabahattin Gucukoglu and S
Moonesamy:
The Problem
I have not read the DMARC protocol in detail, nor looked at any
software provided for implementing it.
What I have done is looked at the DNS record DMARC uses to "advertise"
a site's DMARC policy,
and the commentary surrounding that.
>From what I can tell, DMARC is not a very clever protocol.
What it appears to do, relative to a sending site X, is say
"Sending site X uses SPF or DKIM or both or neither, in order for you
the recipient to verify whether X
actually sent the email you're currently inspecting."
If that is ALL that DMARC does, I would suggest DMARC is a worthless
protocol and that it should be
suppressed and rescinded as an IETF-recognized protocol.
Why? If a receiving site cares to combat spam, it will be using SPF and
DKIM already; there is no
point for an extra protocol on top to say simply "yes, I the sender did
implement SPF and/or DKIM,
so /you should/ proceed to use those methods to validate email claiming
to come from my domain."
Namely, DMARC is simply redundant bureaucratic overhead; the ISP would
use these verification
methods without having to be reminded to do so. And if the receiving
ISP DOESN'T implement SPF
or DKIM, then DMARC doesn't refer to anything the recipient CAN do. So
DMARC is effectively useless
make-work.
I did note on the DMARC home page glowing self-reviews of how many
corporations were protecting
how much email using DMARC - but again, it WASN'T DMARC doing that - it
was SPF and DKIM.
Given that the primary adopters of DMARC seem to be large, for-profit
corporations, and given e.g.
Yahoo's policy of excusing dumping MILLIONS of valid emails with the
excuse of "DMARC policy
violations", DMARC appears to be a Corporate Trojan Horse and an excuse
to start tampering with
the public's email at will, with no accountability.
I run mailing lists for the Berkeley (California) Parent-Teacher's
Association (PTA.) So Yahoo is
essentially saying that no Yahoo users can get email sent to such lists
BY Yahoo users. This of course
makes no sense. Moreover, the issue has been outstanding for long
enough that Yahoo should by now
have REMEDIED the situation. But this seems to much to expect:
Corporate Accountability, in an age
where these corporations think they are "important" for having millions
of subscribers.
There were calls to complain to Yahoo (these are steadfastly ignored)
and to boycott Yahoo (these are
also ignored by a public on automatic pilot, bought out by the notion
of "free" accounts of all kinds.)
We are entering a stage in the Corporatization of America where it's
time to pay attention and fight back,
or these Corporations will WIN and effectively replace the IETF with
their own "common" (malfeasant)
practices. It has been suggested that Yahoo, Google, et. al. are busy
interfering in open-source-run
mailing list for the express purpose of making all independent mailing
list operators look bad.
The intended effect is "If Joe Smith, ISP, can't make his mailing lists
work then maybe we should switch
to using a Google/Yahoo list" - which is only to reward the very
outfits that are trying to commit this
piracy in the first place.
The problem is solvable - at a cost
I have rescued my own mailing lists from this corporate malfeasance -
at the cost of having to send
a unique email separately to each and every mailing list subscriber.
This burdens my server and it
burdens the Internet (but do the Corporations care? No.) it surely must
violate some "best practices"
RFCs published by the IETF.
I have provided a description of what I did to solve the issues to a
few fellow ISPs; I include these remarks below.
I would suggest people start complaining to the EFF (Electronic
Frontier Foundation.) This is something which is supposed to be
their "back yard", that they would address, but they ignored my initial
complaints to them. I think if they start hearing from the
public at large they will realize this is non-trivial and something
very worth their attention.
I think the
IETF needs to issue an emergency policy statement that condemns the
interference these corporations are causing with our Free
Speech.
(to repeat:) It is almost certain that what Yahoo, Google, AOL, et. al.
are doing violates a number of "best practices" RFCs that the IETF has
published.
It is multiplying Internet traffic loads for no valid purpose.
the fix-it advice:
===========================================================================================================
What I did to bypass the Google/Yahoo/AOL issues was simple enough.
1. install the latest mailman (2.1.18_1) - that is the version that
resulted from the bug I reported to them. So if the provider already
has 2.1.18 they still need to insure they have the LATEST version of
2.1.18.
2. As suggested to me by Mark Sapiro of lists.org, the two lines
suggested to be included in Mailman/mm_cfg.py:
VERP_DELIVERY_INTERVAL = 1
SMTP_MAX_RCPTS = 1
3. However, doing that did not have the desired/intended effect of
forcing one email per list recipient, so I did this additional hack to
Mailman/Handlers/SMTPDirect.py:
def process(mlist, msg, msgdata):
recips = msgdata.get('recips')
if not recips:
# Nobody to deliver to!
return
# Calculate the non-VERP envelope sender.
envsender = msgdata.get('envsender')
if envsender is None:
if mlist:
envsender = mlist.GetBouncesEmail()
else:
envsender = Utils.get_site_email(extra='bounces')
# Time to split up the recipient list. If we're personalizing or
VERPing
# then each chunk will have exactly one recipient. We'll then hand
craft
# an envelope sender and stitch a message together in memory for
each one
# separately. If we're not VERPing, then we'll chunkify based on
# SMTP_MAX_RCPTS. Note that most MTAs have a limit on the number of
# recipients they'll swallow in a single transaction.
#<>ecsd deliveryfunc = None
deliveryfunc = verpdeliver
On line 115, I manually forced the delivery function to be
"verpdeliver" instead of "None". The two affected lines in bold. This
had the desired
effect of forcing one email per list recipient (without having to make
sendmail behave that way for everyone on the entire server.)
4. In the list configs, as recommended by Mark Sapiro of lists.org
again, the choice under both "General" and "Privacy Options ->
Sender Filters"
is "Munge From" in the new fields referring to DMARC;
5. But again, since those settings did not seem to disguise a Yahoo
sender from being seen and rejected by Yahoo, I simply told the list YES
in the field "Hide the sender of a message, replacing it with the
list
address (Removes From, Sender and Reply-To fields)" under General
options.
#5 begs the question whether any of the DMARC (mailman) stuff was
necessary at
all, insofar as doing #5 would hide the "a yahoo user sent this email"
from Yahoo, apparently regardless the DMARC stuff new to mailman
2.1.18. It suggests that just doing #5 is enough to get past DMARC on
ANY version
of mailman, but I leave such testing to others.
===
WHAT GETS FIXED?
1. Forcing the list to send one email per user per list fixes:
* google's "guess which domains we host" shellgame policy that would
randomly block emails sent to @gmail and google-hosted domains
simultaneously.
* yahoo's "too many recipients so up yours" policy that effectively
killed emails sent to many yahoo users at once: and Yahoo will not
say
what
their limit is.
2. Doing the "anonymous list" change fixes:
* yahoo's and presumably AOL's DMARC policy
Of course, sending one email per list member means having to try to
parallelize sendmail's queue processing to overcome the inefficiency.
The complaint (that lists.org did not seem to care about as long as
they had a workaround no matter the inefficiency, and that the EFF
effectively
ignored with protests about how "busy" they were, as if this was not
important) is that: that these jerk corporations are thumbing their
noses at
IETF best practices by forcing a great deal more traffic onto the
Internet for no valid reason that they can demonstrate if such a
demonstration is
demanded of them. The sendmail default on "MAX RCPTS" is 100. I see no
reason to limit that number. Yahoo, some years ago, tried to impose a
limit of 6 on that number and finally relented, I'm guessing,
when too many people told them how stupid they were being. But they are
back to
that practice again, and this time they will not tell you what the
limit is (and I haven't bothered trying to establish it empirically.)
The Google "shellgame" is that Google feels free to reject inbound
email, when that email is addressed both to a @gmail user and to a user
inside
a private domain the recipient is having Google host, at the same time.
That is, suppose
foo.com in MX something.google.com
If I send an email
To: frodo(_at_)gmail(_dot_)com,samwise(_at_)foo(_dot_)com
I will get a Google bounce message like this (the two examples do not
use same domain names):
... while talking to aspmx.l.google.com.:
beth(_at_)berkeley(_dot_)k12(_dot_)ca(_dot_)us... Deferred: 451-4.3.0 Multiple destination domains per transaction is unsupported. Please try again. td10si7147932pac.181 - gsmtp
and neither recipient gets the email.
This of course is beyond any simple ability of a mailing list program
to fix -
UNLESS you limit the outbound email to one email per recipient;
although there is NO VALID REASON for Google to impose that restriction.
This general practice of "let's mess with independent mailing lists and
see how long it takes people to start using Google/Yahoo mailing lists
to get away from failures WE IMPOSED ON PEOPLE" has to stop: these
corporations have to be publicly put on the carpet for being at best
STUPID
in their "presumed" antispam practices - unless they are being
deliberately and calculatedly malfeasant. I suspect the truth is
in-between.
On Google's part they have to know they are wrecking mailing lists; I
think Yahoo's admin is just moron-managed.
I intend to post a notice to the various lists I manage asking the
Yahoo and Google users to complain EN MASSE,
and I intend also to suggest that people should look forward to
abandoning these "free mail services" if they refuse to behave
themselves.
However, American Human Nature is so craven and bought-out by now that
most people would scoff at the idea that (gasp) they might
consider paying a pittance per year to have their own email address
within their own domain, untampered with, unsnooped upon, and
unbranded by some third-party jerk.
I can play Google off against Yahoo, by telling the Yahoo users to
threaten to switch to Google if Yahoo won't behave; but it's no great
solution if they actually do so. Another recommendation to people is to
NOT HAVE Google host their email domains, but once people hear
"free" they ignore reason, logic and everything else. How sad it is how
cheaply people are bought out against their own interests nowadays.
===========================================================================================================
Solving the SPAM problem once and "for all time", the right way
The genuine solution to the problem of worldwide spam and 98% of all
email flows being "buy this garbage" solicitations and "please infect
yourself" virus-carrying messages is to QUIT USING MICROSOFT SOFTWARE
connected to the Internet. Every quarter without fail, cert.org
releases a list of "latest detected exploits" containing an average of
a half-dozen mentions of Microsoft (base system) exploits.
In contrast, Linux/Unix have had TWO issues affecting the base system
in the last FIFTEEN YEARS (both the telnet daemon, I think, and
telnet for login is deprecated anyway.) Get Microsoft software off the
Internet and all the "bot-nets" will be killed within two months,
worldwide problem SOLVED.
-----
-ecsd (Eric Dynamic)
Sysad for Transbay.net / Transpacific.net / UCTelecommunications.com
the polemic tone reflects my irritation, sorry. :)
|
<Prev in Thread] |
Current Thread |
[Next in Thread>
|
- Re: Yahoo breaks every mailing list in the world including the IETF's,
Eric Dynamic <=
|
|
|