I promised I would give Frank's detailed and thoughtful explanation careful
consideration, because someone who had taken so much trouble to explain
things deserves a proper and considered response. However, when I understood
the import of what he was communicating, I felt it raised issues which went
beyond the functionality fix I have been seeking. If SPF is really capable of
being used in the way he seems to suggest, the forwarding problem could be
seen as a network security problem, and SPF users could find themselves in
the same moral category as the forgers they seek to fight. Accordingly, I
have begun a new thread to consider the issue of potential network abuse
using this current feature of SPF.
I hope that those who seek to prevent users of "large domains" (see RFC 4408
9.3.3) being able to determine their own policies as a means of bringing down
the current SMTP standard are in a minority, and the others will see how
important it is to the reputation of SPF not to let them continue to abuse it
in this way.
On Tuesday 14 November 2006 19:59, Frank Ellermann wrote:
K.J. Petrie wrote:
SPF is a voluntary standard, but in this area it is formulated in a way
which would only make sense for a compulsory standard.
Not exactly, I've no serious problem if you use a "traditional forwarder"
before a FAIL rejecting next hop. Of course my mail to you would FAIL.
Hopefully it's rejected. The forwarder accepted it before, therefore it
is then obliged to send a non-delivery report to me - ideally with the
complete error reply it got from the next hop.
That non-delivery report would typically indicate your address at the
next hop, and I can resend my mail directly to your real address. That
is an almost perfect emulation of SMTP error 551 "user not local" from
my POV, and exactly what I want, otherwise it would be stupid to publish
a FAIL policy.
SPF is a voluntary standard. It therefore needs to be designed to work
when it is adopted in a piecemeal manner at various times by various
people
The one pseudo-551 I got in now over 30 months worked from my POV. And
others here reported similar cases, it's very rare but works as expected.
Or actually I got three, but the other two were manually forged by well-
meaning but misguided folks, unrelated to any forwarding side-effect.
The problem here is that SPF is trying to place responsibility on
someone who is under no obligation to accept its authority. That's
just not a way to design a voluntary standard.
It's an emulation of the original RFC 821 intention for this scenario:
1 - Forwarders can behave like a mailing list (rewrite the MAIL FROM to
one of their addresses while they rewrite the RCPT TO) => no problem.
2 - They can refuse to forward mails to third parties if that causes
too much trouble for them (rejected by next hop etc.), that's the
concept of "551 user not local", indicating the new address to use.
3 - They can accept the mail for forwarding and _full_ responsibility.
This IS mandatory, it's not a part of SPF, it's STD 10 (RFC 821),
In that case they might use "251 user not local" to encourage the
sender to update the address. And in the original SMTP they must
modify the MAIL FROM adding their own name to the reverse path.
If anything goes wrong later RFC 821 NDRs would use this reverse
path, hitting the 251 forwarder. Only this 251-forwarder knows
why they accepted that mail for forwarding. It's alone their
responsibility, nobody forced them to 251-accept instead of 551-
reject.
So far SMTP could work as designed. But then they decided to deprecate
source routing, because it was cumbersome, and in the forward direction
not worth the trouble, the MX is a far better concept than the obsolete
routing stuff (SMTP routes, or the similar bang paths in UUCP).
BUT while they deprecated forward paths they also deprecated reverse
paths. That was a huge mistake, because now "forwarders" could accept
all mails without taking the responsibility for later NDRs, the NDRs
were directly sent to the MX of the MAIL FROM. Not back on the reverse
path. In esssence that meant forwarders could forward any MAIL FROM as
it pleased them without fearing the consequences (of NDRs queuing up in
their outbound because the reverse path was wrong).
There was no more incentive to carefully distinguish between 551 and 251.
And it apparently continued to work, after all forwarding scenarios were
still rare, and it makes no sense to send mail via A to B, if you could
also send it directly to B, one point of failure (forwarder A) less.
But the situation deteriorated, folks began to use any MAIL FROM they
liked best, send via S, their favourite smart host, and receive error
messages at P, their foavourite POP-provider, no matter if S == P or
not, just say MAIL FROM P actually submitting mail at S != P.
And it got worse, the spammers detected that they can use just any
MAIL FROM, it only needs to exist for "call back" tests. And we got a
huge flood of forged MAIL FROMs, only because of a serious design flaw
in RFC 1123 - 2821 (not 821), the most expensive error in a protocol
design I've ever heard of.
Thank you for this very useful historical overview. At least we can all see
where people are coming from, and that helps understanding enormously.
However, I'm not sure I can agree with the position, because...
SPF merely fixes it back to an emulation of the original 821-SMTP, as
you said on a voluntary basis. Nobody is forced to implement it, or
to publish FAIL.
But those who publish FAIL know why they do that. And they want that
forwaders again take the responsibilty for their actions, or take the
simple 551-emulation exit, nothing is wrong if the next hop rejects
an unmodified MAIL FROM, quite the contrary, that's very good. The
senders publishing FAIL want it that way.
You as receiver might have other priorities, but that's an "R" issue,
not an "S" issue.
Or could it be an "NA" issue? For if I understand you correctly, you are
saying people publish FAIL not to reject mail originating from
spam/virus-type forgers, but to cause service denial to those using
forwarders which comply to the current version of SMTP, rather than a
previous, now deprecated version, which they think was better. Now we may not
agree with the current standard, but trying to force change by denying
service to innocent third parties who are using standards-compliant systems
is sabotage against the network. Moreover, I'm not convinced that this has
been explained to everyone who publishes FAIL. I suspect some are being drawn
unwittingly into this attack in the belief publishing FAIL will protect them
against on-line identity theft, unaware they are being used as pawns in a
calculated form of Internet terrorism. If not following the deprecated parts
of RFC 821 can be called forgery, don't you think engineering a protocol to
break RFC 2821-compliant mail systems could be described as network abuse?
This raises my proposal from a functionality fix to a security consideration
and makes it more urgent, if SPF is being used as an attack vector against
standards-compliant systems.
I am also unconvinced by the thesis that changing nothing is forgery.
It is wrt RC 821, relays were supposed to modify the MAIL FROM, either
by adding their name to the reverse path, or as it still is in the case
of mailing lists and gateways, modifying it completely. "Unmodified
MAIL FROM" doesn't work, as demonstrated by now 10% of all mail traffic
only consisting of "misdirected bounces" (to forged MAIL FROMs) hitting
innocent bystanders.
Please explain what the problems are with my modified proposal.
It doesn't work as expected as soon as you do something very harmless
like start an auto-responder (see RFC 3834). It's fine that you accept
responsibility for your "SPF-disabling-RPF", but as soon as you have
disabled SPF you get those forged MAIL FROMs on the routes where you
disabled it, and your RFC 3834 auto-replies hit innocent bystanders.
That presumes the good practice implied (but alas not spelt out) in RFC 3834
is not followed, namely that autoresponders should be set up not to reply to
unverified addresses. I would rather presume the (very short) routes on which
SPF would be "disabled" would also be expected to ensure services like
autoresponders were disabled or configured to ignore unverified MAIL FROMs.
Why are you calling it RPF, incidentally? I discontinued the RPF proposal many
posts before I wrote what you were responding to here.
K.J. Petrie.
-------
Sender Policy Framework: http://www.openspf.org/
Archives at http://archives.listbox.com/spf-discuss/current/
To unsubscribe, change your address, or temporarily deactivate your
subscription,
please go to http://v2.listbox.com/member/?list_id=735