RE: Explain please (Was: SPF Stats)

David,

I appreciate your specific answer.

At 04:10 PM 7/5/2005, you wrote:
> On Tue, 2005-07-05 at 14:11 -0600, Commerco WebMaster wrote:
> > SPF caused failures *should* occur when someone is trying to send
> > email from a given SMTP server, claiming to be MAIL FROM: a domain
> > name that has not been authorized for said SMTP server via an SPF DNS
> > TXT entry in the domain's zone file.  After all, that's what SPF does.
>
> Your later comments seem to indicate that you've entirely missed the
> problem here.
>
> Think what happens when you send a mail to someone, for example, at a
> virtually hosted domain, and your mail is forwarded to a real mailbox at
> an ISP somewhere.
Okay.  Let's try working through the process to see if the issue can 
be resolved.
The receiving MTA at the ISP managing the virtually hosted domain 
gets a message from a sender MTA where the original MAIL FROM: domain 
holder has an SPF record protecting their domain.  Presumably the 
receiving MTA also checks for same to discover that the original 
outbound server from that MAIL FROM: domain is valid.  So far so good.
That ISP MTA forwards its message onto another MTA, placing its own 
stamp on the message header and connects to another or the final 
target MTA, the intended recipient's MTA.  Now then, would not it be 
prudent for the final recipient's MTA to look at where the message 
came from?  In other words, wouldn't the recipient's MTA know the set 
of MTAs that it would allow to send forwarded messages from their own 
ISP by simply checking that ISPs SPF record at connection time?  Say 
they forward through more than one forwarder.  Is it really that much 
of a burden on the final recipient MTA to check the incoming 
connection to make sure it is really an authorized forwarder to the 
final recipient's MTA for even several forwarding MTAs?
Let's presume it is an acceptable burden, after all, the real burden 
to determine acceptable forwarders is being place on that final MTA 
handling mail for the virtually hosted domain in your example and not 
the ISPs forwarding MTA, so as the end point, that burden should not 
be unreasonable.
> The mailhost which hosts that virtual domain is _normally_ going to send
> your mail on to its final destination intact, and that includes the
> reverse-path.
Yes.

> Then the final recipient, if they are strictly checking SPF, is going to
> reject that genuinely forwarded mail because the mailhost which does the
> forwarding _isn't_ explicitly listed as one of the valid hosts for the
> original sender's domains.
And perhaps that should be true in all cases, but those where the 
forwarders are known and trusted by the MTA receiving the forwarded 
message.  After all, the receiver should logically be able to know 
their allowed and trusted forwarders.  A sender cannot know the 
recipients forwarders as it just sends to the MX it finds in DNS 
(which for your example should be the ISP's MTA), but the final 
recipient's MTA certainly should.
In your scenario, perhaps the receiving MTA getting the forwarded 
message should really look at an "-all" as an "~all", but only for 
this case of incoming messages from a trusted forwarder.  In other 
words, the recipient logic looks like: "Although this message fails a 
format SPF '-all' check per the domain owner's published records that 
I would normally do were the sending IP not a trusted forwarder, I 
can regard this as possibly (indeed probably) from the originator as 
claimed, _only_ because I trust the forwarder who checked and passed 
the SPF record that brought the message to me".  Now, if the 
forwarder breaks trust, that becomes a judgement call for the final 
recipient in your scenario as to whether or not to continue with that 
ISP and certainly is not an issue for the publisher.
This process would seemingly address your concern of forwarding 
inappropriately dropping messages for strict SPF checking.
> The effect is that genuine mail is rejected. That's not a _good_ thing;
> it's an unfortunate side-effect of the way SPF works. This is what SRS
> was invented for -- SRS is the baroque method of rewriting sender
> addresses to make the forwarded mail pass SPF checks again. This needs
> to be implemented _everywhere_ that does forwarding, in order that valid
> mail won't be lost.
Only if one is going to trust their own forwarders, else, one can't 
really implement SPF in its strict evaluation case.  That reality 
does not mandate that SPF be implemented _everywhere_ at all.  The 
final target of the forwarding chain (for that matter any other 
forwarding MTAs in the chain) need not even implement SPF checking, 
if the first MTA in the forwarding chain actually receiving the 
message from the original sender has done the SPF check and all 
members of the chain are trusted.
> Now, some would have you believe that this breakage _is_ a good thing.
> They claim that they way the world has always worked is 'forgery' and
> 'abuse' and try to make out that it should never have been permitted.
> But truly, the only reason for changing it is to make the original
> invalid assumptions of SPF come true.
I'm still not clear that your point is correct, if forwarding MTAs 
act responsibly in checking to see if the members of their forwarding 
chains are reliable.  If you cannot verify that, perhaps you need to 
revisit the MTAs you use to receive forwarded messages (e.g., get an 
ISP you can trust to handle your email forwarding needs).
> Now, that's perfectly reasonable if it's actually _necessary_. I agree
> 100% with what you say here:
> > Change is a reality in every part of life, including the Internet.
> > The issue and goal in change is how to minimize the impact and
> > inconvenience of any change proposed so as to cause least
> > inconvenience for the largest numbers.
>
> But the point is that such massive changes _aren't_ necessary. There are
> plenty of alternative solutions which _don't_ require such massive
> changes to the way the world works. So I disagree with your following
> sentence, where you suggest that SPFv1 'achieves that goal handily'.
>
> SPFv1 requires massive changes to infrastructure which aren't actually
> necessary in order to achieve the goals which SPF sets out to achieve.
> Therefore it _doesn't_ achieve your stated goal, handily or otherwise.
Why?  I still don't see this for the vast majority of implementing sites.

> Now, people are working on fixing the forwarding problem, by means of
> SRS and other techniques. But _until_ that is done, SPF isn't viable.
> You are putting the cart before the horse if you are pushing for IETF
> blessing on SPFv1 before that problem is fixed.
Perhaps I am wrong, but I always had the impression that SRS dealt 
with email at the granularity of the senders rather than the domain 
level.  For current purposes, it is probably most productive just to 
get any domain level issues resolved first in SPF version 1 with an 
eye on future SPF extensions and not try to solve everything at 
once.  The first problem SPF needs to solve is the issue of domain 
name identity theft in email, which it does in most current environments.
> You then go on to discuss the popularity of SPF. VHS is more popular
> than Betamax -- what relevance has that? I certainly don't deny that SPF
> has good marketing hype, and the fact that it's so easy to gloss over
> the massive and fundamental flaw in it -- even to the extent that you
> don't seem to have noticed it at all after following the discussion --
> is very helpful in its marketing.
As to your VHS vs. Betamax parallel, it seems that one can actually 
get a VHS video at the rental store (i.e., because of the numbers 
involved one can find more MTAs which actually support SPF than any 
other solution attempting to address domain name identity theft in email).
I think that in successfully addressing your concern above, perhaps 
the concern is not such a massive and fundamental flaw after all.
> You also seem to suggest that because this is an SPF list, the primary
> goal should be to encourage the adoption of SPF even when it's not the
> best answer to the original problem we were trying to solve. I don't buy
> that -- I don't think anyone here would claim that they would favour one
> solution over another for anything but technical grounds.
If the purpose of the input is to improve SPF at a technical level, 
then yes.  It is unclear from your original message that this was your intent.
> There are people who are brought here, as I was many moons ago, because
> of the hype which is spread about SPF and want to learn more about it. I
> don't think it does them a service if it's entirely a love-fest without
> any critical discussion and comparison with alternative methods.
If you have been on this list for any time, you know that the SPF 
list is hardly a "love-fest".  Many of the most ardent supporters of 
SPF on this list have and will go after each other with a vengeance 
if they disagree on some specific fine point of the standard.  For 
the not fully indoctrinated to this list, reading some of the 
messages here might give a misimpression that some of the most 
sincere supporters of SPF are detractors, but that is certainly not the case.
> I'm not particularly tied to BATV, although it was the first alternative
> method I implemented and so far the _only_ alternative method because I
> haven't actually had any need to do anything else. You'll note that the
> BATV draft doesn't even bear my name. _Any_ of the alternative methods
> (other than SenderID which manages amazingly to be even more broken than
> SPF) would be a viable alternative. If my occasional voice of dissent
> causes even a few people to actually stop and think for themselves and
> to do something other than SPF, then I believe I've done them a service.
I am glad you have found a solution that works for you.  Just so, I 
have one that works for me, however in both of our cases, this is 
anecdotal.  So the issue becomes which case works for the broadest 
sector measured by the adoption of the publishers and MTAs.  I too am 
fairly sure the SPF draft does not mention me either, nevertheless, I 
support SPF and its goals.
Being a voice of dissent should never be a problem, as long as with 
the concerns expressed also come with solutions (or minimally 
suggestions to reach a solution).  Throwing one's arms up and saying, 
"the other solution is better" does little to contribute to 
correcting the issue expressed in the original concern.  Why is the 
other solution better and more importantly, can the other solution 
successfully be adopted into the subject of the list (SPF) to make it 
stronger?  Answering such questions brings both a concern and a 
solution to this list, which seems a more productive way to proceed.
> --
> dwmw2
Best,

Alan Maitland
WebMaster(_at_)Commerco(_dot_)Net
The Commerce Company - Making Commerce Simple(sm)
http://WWW.Commerco.Com/