RE: Explain please (Was: SPF Stats)

David,

You make some interesting statements in your message below, so now I 
would ask that perhaps you could please explain them better so others 
like me on this list can understand the basis and rationale behind 
your statements.  I'll try to help you by offering my own responses 
to your points and perhaps you can explain where my logic might drift.
At 01:08 AM 7/5/2005, David Woodhouse wrote:
> I don't make a distinction between 'not delivered' and 'lost' because in
> the general case a large ISP can't make that kind of distinction. Users
> are often not capable even of reading bounce messages and understanding
> basic concepts like "user unknown", let alone interpreting an SPF-caused
> failure. Have you never made someone read a bounce message to you over
> the telephone and then just paraphrased it back to them?
SPF caused failures *should* occur when someone is trying to send 
email from a given SMTP server, claiming to be MAIL FROM: a domain 
name that has not been authorized for said SMTP server via an SPF DNS 
TXT entry in the domain's zone file.  After all, that's what SPF does.
Of course, as you point out, some users may well be fairly 
unsophisticated and may not understand nuances of email, but then 
those users are generally part of the set 
"unsophisticated(_dot_)user(_at_)largeISP(_dot_)tld" or, in other words, those users 
generally are part of the ISPs email domain, so naturally, the ISPs 
SPF record will cover those cases for their own domain name.
One might argue that if a user is sophisticated enough to have their 
own domain, "somebody" must be maintaining their DNS record.  That 
"somebody" should clearly create the appropriate SPF DNS TXT record 
for that domain or it might be argued that they are not serving their 
paying customer's interests very well in not offering their customers 
domain protection from identity theft in email via SPF, the most 
widely adopted standard for same on the Internet.
So please help me to understand your original point here?

> The effect when the ISP starts publishing SPF records with '-all' is
> that users suddenly start finding that some of their outgoing mail isn't
> getting through, and blame the ISP.
Why?  The ISP is going to publish their outbound SMTP servers in 
their SPF record.  Logically, if an ISP was about to go "-all", they 
would give their users ample heads up time.  If the user has their 
own domain and wants to send via their ISP, they would reasonably 
include the ISP in their SPF record, so where is the problem?
> The effect when the ISP starts _honouring_ SPF 'fail' results by
> rejecting mail is that users suddenly start finding that some of their
> _incoming_ mail isn't arriving, and blame the ISP.
Why?  I can't even figure out what possible scenario this could cause 
inbound messages to be failing unless the domain holder of the 
incoming sender has a badly structured SPF record and that is not the 
certainly not fault of the ISP.
I remember a time when the Internet was less about finger pointing 
and more about helping to resolve issues to facilitate communications 
between networks and to build a better infrastructure.  Just because 
a user complains about something does not make it so, moreover a 
quality ISP is going to help their user solve problems by employing 
simple educational processes (e.g., point to the page on the ISPs 
site the concerned user should have the sender who experienced 
problems go to find out how to fix their problem).  AOL (a very large 
ISP) seems to have a very nice set of pages constructed to help SPF 
publishers to do that very thing.
One final thought here.  If a domain owner chooses not to publish an 
SPF record for their own MTAs, the result should be no different than 
the way email works today.  That is, their domain becomes a marvelous 
target for those who would want to commit identity theft by stealing 
their name to be used in MAIL FROM: for bogus email.  Something no 
responsible domain owner should ever want.
> 'Large fraction'? There are currently about 60 sets of hosts listed
> there -- is that really a 'large fraction' of all the hosts out there
> which are involved in virtual domain hosting? How many companies are
> there which will register your domain for you and then forward its mail
> to addresses you specify? I wouldn't call 60 a 'large fraction' of that
> number.
Why would an ISP that is doing mail forwarding not want to provide 
good customer service by setting up or at least assisting their 
customer in understanding how to set up an SPF TXT record for the 
domain they are handling that includes their own outbound SMTP 
servers?  Who cares about such things as a trusted forwarder when the 
ISP and the domain holder are actually publishing their own SPF 
records?  As I understood trusted forwarder, it was intended to be an 
early stop gap measure until larger ISPs published their own SPF 
records.  As that happens, logically trusted forwarder tends to 
become less useful under the current structure, something I think the 
operator of that service to promote early SPF adoption even notes on 
their site.
> That's true for yourself, but larger companies who are paid to provide
> email service would find it a much harder decision. It you're paid to
> provide reliable email service, it's much harder to just say "Feel free
> to reject failures from my domain" in the knowledge that this will
> include valid mail sent by paying customers.
Why should it?  If the mail is coming via the ISPs own SMTP servers, 
they would logically have covered those in an SPF record.  If the 
mail is claimed MAIL FROM: a given domain sent by the customer, then 
the customer would logically include their own ISP in their SPF 
record for the domain.
Now, if the customer is trying to send mail from other servers and 
represent themselves as being from their ISPs servers, that is a bad 
thing for both the ISP (reputation damage potential from a spammer) 
and the customer (who is misrepresenting the truth of from where they 
are sending) because this is the very problem SPF addresses.
Will some things in email probably have to change because the 
apparent original presumption over two decades ago was that email 
users would not abuse email was erroneous?  Of course.  Will some 
have to change the ways they do things to accommodate such changes? 
Yes.  Should and will those broadly adopted changes impact the vast 
population of email users to any significant degree? No.  Is asking 
people to change to make a better, more reliable and more email 
environment possible with little or no change to basic existing 
infrastructure unreasonable?  Of course not.
The dogs may bark, but the caravan moves on.  That is true if we are 
speaking of SPF as regards domain name identity theft or any other 
future solutions to combat email spam.  Change is a reality in every 
part of life, including the Internet.  The issue and goal in change 
is how to minimize the impact and inconvenience of any change 
proposed so as to cause least inconvenience for the largest 
numbers.  For what SPF version 1 is supposed to do to protect domain 
reputation, it achieves that goal handily.
> (Scott Kitterman wrote):
> >  I think the benifits are worth the minor inconvenience
> > associated with these edge cases.
>
> Perhaps -- but again, others don't see those same benefits. My users
> certainly wouldn't -- BATV already gets rid of just about _all_ the fake
> bounces, and also allows recipients who use SMTP callouts to reject joe
> jobs _without_ much risk of losing valid mail. What further benefit
> would SPF '-all' provide on top of that?
David, this is an SPF list.  I think that this list has been 
historically really very tolerant of statements like yours above, 
however, discussions of why other protocols are "much better because" 
serve little purpose in forwarding the objectives of SPF, something 
that would be of importance on an SPF list.  That is unless, of 
course, the "much better because" feature can be successfully 
incorporated into SPF without undermining or negatively impacting the 
reason SPF exists, how it works or its goals to minimize adoption impact.
Before electing to wandering down the path of "x is much better 
because", please keep in mind which domain name identity theft 
prevention solution mechanism has the lions share of adopters (clue: 
SPF) and exactly why that might be (Clue: SPF offers an easy and 
reliable mechanism for domain holders to publish SPF DNS TXT records 
to document approved outbound MTAs for their domains without having 
to become a rocket scientist or needing buy anything new to do 
so.  Further, SPF offers a relatively easy standard for MTA suppliers 
to adopt and thus allows them to implement a broadly accepted domain 
name theft prevention system feature in their software.  That is 
probably why a growing number of MTA major and minor suppliers have 
done so already today).
I chose to become an SPF publisher and later upgrade to SMTP servers 
supporting the SPF version 1 standard because of the scope of 
adoption (lots of other domains, including very large ISPs, publish), 
general maturity of the standard (it is a solid standard designed 
specifically to prevent domain name identity theft), general 
applicability of the standard (it covers most email situations in the 
vast majority of networks and all of them I am concerned with from my 
own perspective and experience).  Add to that the fact it is a public 
standard unencumbered by potential legal issues, and finally and most 
importantly, SPF is a direct fit in addressing and fixing a serious 
business problem (domain identity theft in UCE) and it is easy to see 
why I made this choice.
If your intent is to evangelize BATV, perhaps you should do so with 
MTA creators who might wish to embrace the BATV process, but it is 
counterproductive for you to seemingly do so on an SPF discussion 
list.  In apparently attempting to discredit a widely deployed 
solution that serves the needs of those who have already embraced it, 
you arguably do little more than discredit yourself and the very 
solution you seemingly hope to promote here.
I think that one of the goals of this SPF discussion list seeks to 
figure out ways to get SPF even more widely adopted in environments 
where the standard works (for nearly all adopters).  It is 
appropriate to stay focused on what classic SPF 1 does here right now.
> (Scott Kitterman wrote):
> > Note that I'm saying it works for me, not that AOL or Verizon or 
> anyone else
> > should immediately switch to -all.
>
> Right. Such people really would need to wait for SRS to become
> ubiquitous before they could sanely consider such a move. Again, I say
> 'sanely' because they can _always_ do something stupid like banning all
> non-US hosts, etc.
What reason do you have for this claim?  Your wording keeps referring 
to "sane" this and that, but in fact many quite sane individuals have 
published and rely upon SPF today with great success.  It is prudent 
and good judgement for larger entities to show care in making sure 
that changes they make in their own networks and services cause 
minimal impact for their customers.  That is an entirely sane thing 
to do and doubtless why larger ISP take their time in moving to 
publish as "-all".
I look forward to the day when AOL or another similarly large adopter 
decides to attempt a series of real world "-all" experiments to see 
exactly what happens beyond what has already been explored and 
planned for during their "~all" phase experimentation, and then 
moving forward to the point where they can publish a "-all" SPF 
record, getting past this straw man argument of "but <insert large 
ISP name here who is testing SPF as part of an implementation plan in 
their network> doesn't publish SPF as '-all'".
That day will come and probably sooner than later, because SPF is 
another "feature" that an ISP can rightly claim helps and protects 
their users.  Offering features other ISPs do not offer provides ISPs 
who do offer such features a competitive advantage and in the very 
competitive world we live in, that reality is not lost on them.  The 
good ones research and take their time in implementing features that 
serves to benefit their customers.  Nevertheless, they do and will 
continue to implement such tools as SPF to the benefit of their own 
email users.
> As far as I can tell, SRS isn't going to become part of default
> configuration of common mailers any time soon. It's still not even
> supported at all by most of them in the default build -- if you want it
> you have to rebuild the MTA or install extra software.
So it seems that, for most environments, SRS does not have to become 
a part of the MTA build for SPF to work as advertised.  Okay.
> When RFC2821 is updated and mandates SRS-like behaviour, perhaps that
> will change. That's the main problem for SPF deployment en masse.
I think that one of the goals of SPF was to have the standard be SMTP 
related RFC independent.  There are still a fair number of SMTP 
RFC1869 servers out there and I think that an RFC821 server could 
even be revised to support SPF.  I'm not sure RFC2821 has all that 
much to do with SPF deployment.  Certainly not from a publisher's 
standpoint.  All publishers presumably want is to avoid having their 
domain name reputations tarnished by those who steal domain name 
identities of unsuspecting domain holders for use in their UCE and to 
avoid suffering the collateral network abuse brought on by invalid 
bounce messages which such actions by these thieves cause.  SPF fits 
that need very well for most environments.
Perhaps the most reasonable action to increase SPF deployment is to 
continue to promote awareness among domain holders (who *become* the 
SPF publisher community and the key to SPF proliferation), as well as 
education on what SPF is not and what SPF does to prevent domain name 
identity theft in email.
I think the answer to SPF adoption is far more about getting the word 
out, having domain holders publish proper SPF records and continuing 
to gain even more MTA makers who build servers that properly 
interpret and support those domain holders' published SPF records.
> --
> dwmw2
Best,

Alan Maitland
WebMaster(_at_)Commerco(_dot_)Net
The Commerce Company - Making Commerce Simple(sm)
http://WWW.Commerco.Com/