Michael Kaplan wrote:
I hope you would acknowledge that your sentiment is a
minority position not shared by a vast number of reputable
top email companies that frequently enable it in one fashion
or another. Officially the IETF does not consider the
auto-reply provisions of RFC 3864 abuse either.
Apparently we have a communication problem here: I support
RFC 3864, it's one of the best RFCs I know, and I seriously
hope that it progresses to a full Internet standard.
It does however (like all non-delivery reports) depend on a
single premise: Return-Paths must be no nonsense. For the
old (pre-1123) SMTP that was "guaranteed" by design - as far
as those guarantees go, in theory it was a sound design.
Certainly the "SPF FAIL or PASS" folks including me are a
minority. But without it stuff like RFC 3864 (incl. your
proposals) and NDRs can't work anymore, because the flood of
unsolicited auto-replies (incl. bounces) forces receivers to
block or drop them.
Anybody who used mail in one of the more obscure positions
like an unofficial UUCP link five hops from the Internet or
a Fido node behind various gateways to the rfc822-world,
would tell you that (store and forward) mail cannot work
without error reports (NDRs).
As soon as bounces are dropped wholesale the complete system
breaks together, mail vanishes in black holes. SMTP is too
"simple" to work without error reports. Today it's already
almost normal to confirm the arrival of important mail by
phone calls. This is worse than a decade ago, where I could
trust that the Fido-gateway or UUCP-link might be slow, but
generally reliable.
So in other words I strongly believe in "good bounces", and
an SPF PASS is not only a permission to send eror reports if
necessary, it's a plea. Like the opposite SPF FAIL is not
only a permission to reject forged mails. Anybody seriously
considering SPF FAIL as an indicator to drop mails missed the
point. Reject and drop are different.
I'd like for us to look beyond that for now - we know that
the major email companies will certainly look beyond it.
As far as I'm concerned you can get it right, do whatever you
like if you get an SPF PASS, and reject FAIL. For BATV I've
no clue how you identify it, or how you could know that it's
missing. But as you said those are minorities today, for the
vast majority sending challenges to unverified Return-Paths
won't fly.
A system that works only if the whole world uses BATV or SPF
is exactly as relevant as a system assuming that the whole
world implements STD 10 verbatim (without the RFC 1123 bug):
It's a wrong premise.
I think you underestimate the discriminating power of
Bayesian filters.
Possible. Feel free to fix the numbers, raw 5+11+84, filtered
(4+1)+11+(42+42). If your definition of "intermediate spam
likelihood" is 25:75 instead of 50:50 you get (4+1)+11+(63+21).
With that you'd arrive at 1+21 challenges (21 unsolicited).
42 of 43 was 97%, 21 of 22 is 95%. I've offered 40 of 43 (93%)
in the same article where I arrived at 42 of 43. And der Mouse
has proposed 80%, that's conservative, showing that the details
don't matter: 80% unsolicited challenges won't work. You'd be
blocked, and then your other challenges never make it.
How about if we bounce the 10% least spammy mail
With the same scenario in permille that's (10+40)+110+(756+84)
for 84 unsolicited challenges of 94, you're down to 89% spam.
If that's realistic for your case I'm fine with these numbers,
but 89% of all challenges unsolicited is still quite a lot UBE.
Your calculation also assumes that 100% of spam spoofs a real
email address.
Yes, if you use call-back-verification other addresses never
come near to your C/R system. And if you don't do it the smart
spammers have done it. Completely bogus addresses aren't very
interesting, you only need some disk space for your send queue
while you try (in vain) to send challenges to these addresses -
no problem.
A substantial amount (the vast majority??) of spam uses
randomly generated addresses.
That doesn't match my obsevations (catchall vanity domain):
The vast majority are parts of or complete Message-IDs, local
parts generated by Sober and other mail worms, and similar
cases, not completely random 1*63(ALPHA / DIGIT / "-" / more).
Any longer addresses I see are almost always real Message-IDs.
I suggest correlating incoming bounces with recent outgoing
mail and filtering bounces that don't correspond.
That's the BATV idea in a nutshell, doing it at the MUA has
its own challenges (not everybody can create Return-Paths on
the fly, other matching methods are tricky, and last but not
least ifit hits the mailbox before POP3 or IMAP the battle is
already almost lost, V90 users note it immediately, others
see it on their invoice if they pay by volume).
I used the term Joe Job because I suspect that the problem of
erroneous bounces is exaggerated.
I disagree. Anybody who doesn't drop bounces unread has a
serious problem. Less so for SPF FAIL folks, and almost ideal
for BATV users, but even then there are many "wannabe-bounces",
crap like Mailer-Daimon@ (sic). In 2004, when SpamCop did not
allow to report bogus bounces, I tried to filter such crap for
some months, it was a PITA. Since about two years my spammers
mainly got the idea of SPF FAIL, therefore it's under control
from my POV.
But I did note the two weeks in 2005 when they tested again to
forge my addresses. Even today I'd still note it if there are
1000 bogus bounces per day in addition to the "normal" spam -
but fortunately (or thanks to SPF FAIL) that's not the case.
I know that John Levine has been plagued by this issue, but I
would classify his problem as Joe Jobbing.
He told us that he uses BATV, and I'd be very surprised if the
professional spammers don't read this list to stay up to date
(among a few others). No "Joe Jobbing" in my book, an attempt
to (ab)use the reputation of abuse.net maybe.
Without SPF FAIL the receivers can't identify "missing BATV" or
"invalid BATV", they'd need CSV or an emulation of it to reject
bogus "EHLO abuse net" sessions. If that's what his box says,
...checking, no, not necessarily, it can be "EHLO iecc.com". I
forgot the details how to check CSV for this.
I suspect that much of the foot dragging on BATV is because
it addresses a problem that most people don't have.
You'd only see it with a domain, on other addresses (no vanity
domain) I've seen it only in conjunction with mail worms (the
worms forged my address, and I got the backscatter), not spam.
If you can site any literature on this issue then I would
certainly appreciate it.
You might find some links in the related Wikipedia articles:
<http://en.wikipedia.org/wiki/Category:E-mail_authentication>
Caveat: I had my fingers in some of those Wikipedia articles.
The most often quoted source I'm aware of is the Spamcop FAQ:
<http://www.spamcop.net/fom-serve/cache/329.html>
<http://www.clickz.com/showPage.html?page=3602516> quotes
Ironport, Messagelabs, and Postini, and if that's the source
where I saw 84% I wonder where I got the 11% - this article
says 9% misdirected bounces. I'm too lazy to dig for the
original Ironport report, if you find it please post a link.
"worldwide deployment" is completely unnecessary
Good. You did however hint that anybody who doesn't like the
unsolicited challenges can deploy BATV, and I doubted that.
I (and almost every major company) disagree with the concept
that you can't send an automated response to an email unless
that email was authenticated.
I guess we disagree about this, but as long as you reject all
SPF FAIL it won't affect me personally. I won't mind if you
challenge an SPF PASS from me, I'm free to ignore it or to
answer it - depends on what I want, I answer challenges from
say ICANN or IANA (again and again... so far ;-).
Frank
_______________________________________________
Asrg mailing list
Asrg(_at_)ietf(_dot_)org
https://www1.ietf.org/mailman/listinfo/asrg