On Oct 26, 2015, at 2:45 PM, John C Klensin <john-ietf(_at_)jck(_dot_)com>
wrote:
If we decide that the long-established semantics are the right
ones, then I think our email standards deserve to die, because
they don't currently work.
Ted, I think millions of users, passing around tens or hundreds
of millions of messages around a day, would probably disagree
with "don't currently work" or at least dismiss it as rather
extreme hyperbole.
I will admit to using an extreme form of the term "work," which is "work
without massive difficulties." Most of the massive difficulties happen behind
the scenes, so we fortunates do not have to deal with them, but the lengths
mail providers like Google have to go to to keep spam out of our inboxes cast
their shadow over the experience of every email user. Web registration
systems now tell users to whitelist mail from their domains, which would be
cool if it could be done automatically rather than through manual intervention,
and users are now accustomed to searching for important mail that hasn’t
arrived in their junk folders. I suspect the annual number of person-hours
spent on these two tasks would humble us.
My experience of "badly broken": pre-filter, for my email address
(mellon(_at_)fugue(_dot_)com <mailto:mellon(_at_)fugue(_dot_)com>) alone, spam
is currently arriving at a rate of about three messages per minute. Why?
Because the design assumptions for email did not account for the fact that in
the wild, email is an ecosystem, not a cooperative venture, and there is money
to be made with scattershot spam, and money to be made filtering it.
I’ve attempted to implement a whitelist on my server to make sure that mail
from people I already know will get through. What I’ve discovered is that
aside from the big providers, nobody sets their SPF up right, so I can’t rely
on it to validate whitelisted senders: I either have to just hope for the best,
and accept the occasional joe job attack, or else I whitelist IP addresses, or
come up with heuristics like "if the MX for a domain points to google, pretend
that the domain owner set up SPF according to Google’s docs."
The amount of brainpower that’s required to keep this rickety train on the
rails is astonishing. It is no longer the case that someone like you or I
with the resources of an individual can have a reasonably painless experience
of operating an SMTP server. To my mind, this means that SMTP does not
"work." There is no dependable method by which I can ask the question "did
this email message come from the source that it claims to have come from" and
get an answer.
That’s what I mean by "works." And that’s why I have every sympathy with mail
providers who are throwing up their hands in disgust and saying I’m going to be
draconian, even though the specs don’t technically allow it. If we want
something different to happen, we have to figure out a way to allow it to
happen, and not just say there’s no problem and Google ought to follow the
specs.
Perhaps I haven't been looking in the right places, but I
haven't heard Google claim that email is "badly broken", much
less "doesn't work". What I have heard is some claims about
blocking of some messages originating from bogus or unauthorized
senders. That is a sender authentication problem, not a "broken
email protocol" one.
It’s true that if we had a reliable way of validating senders, SMTP could
continue to operate. And it might even be that if we had this mechanism in
place, spammers would stop dumping crap on my mail server, so I wouldn’t have
to pay for gigabytes per day of useless traffic to my server. But in
practice, it would be much better to use a protocol that didn’t even trigger a
data transfer until the sender had been validated.
Equally important, if Google really
cares about either sender authentication or verification that a
sender who uses a particular backward-pointing address today is
the same entity who used it yesterday, we know of a large
variety of ways to approximate at least the latter. The
observation that Google isn't doing any of those things, even
the ones they could support with a very large fraction of their
users and without protocol changes, suggests that isn't the
issue.
Yes, we do know ways to do this. Most of them are too expensive to be
practical.
So, if you are going to claim that our existing standards don't
work, I think it would be good to have a clear explanation of
what you mean and what, precisely, doesn't work. Of course, I
can only hope that, contrary to your apparent claim, this
message will reach you in spite of non-working protocols and you
will be able to reply.
I have all the IETF mailing lists whitelisted, and IETF has a correct SPF setup
(and, actually, I get my IETF mail through nominum’s servers, and Nominum pays
Google a lot of money to filter spam, and they use their big data analytics to
do a good job of that, so in that sense it does indeed "work," but I shudder to
think what the carbon footprint of each valid delivered message is if you
amortize the cost over the total number of messages that had to be transmitted
and examined.