Re: State of the SPF website (was: SPF+SRS vs. BATV)

I haven't replied to most of the threads in this conversation, but John's 
message shows an honest attempt to get to the heart of the problem.  Thank 
you for taking the time to write it.
--John Hinton <webmaster(_at_)ew3d(_dot_)com> wrote:

> Ahhh... but in the 'real' world of 'delivering email to clients', we have
> to sometimes deal with these busted systems. I two weeks ago, had one
> client lose $600.00 because their client didn't have reverse DNS.

That's a good example.  Hopefully now that AOL is starting to reject 
no-reverse or wrong-reverse that trend will shift.  But it's a painful 
process.  I guess the best any of us can do is patiently explain (as you 
have done several times before) that the sender needs to fix something on 
their side.  As more of us patiently explain, and stick to our guns, it 
becomes that much easier for people coming along after.

> I too had a -all record set up on one domain for 6 or 8 months. I did get
> several rejects. I made contact with some of these sysadmins to explain
> to them their errors. In about 50% of the cases, they didn't even know
> what it was, apparently it was just a 'on/off switch' to them, which
> looked like a good spam filter. I think there are going to be a LOT of
> these broken mailservers in the next year or so.
>
> I think my only choice is a ?all at present, followed by a ~all sometime
> in the future (as acceptance and testing is completed) and then -all once
> the world catches up. I see a proper implementation as a very fluid
> process.

It's a reasonable opinion, and I can see how you've come to it.  You can 
also try "soft fail" which I think would give you the effect you want.
In a perfect world, the domain owners who publish SPF are making the 
statement that "These are the agents sending on my behalf - anyone else 
handling mail from my domain is not authorized by me."  It's sort of 
implied that if a forwarding relationship exists, that the forwarder is an 
agent for the receiver, not an agent for the sender.  So, I believe that 
it's reasonable to publish "-all" and expect the receiver to take 
appropriate care.
But, as you pointed out, there will be receivers who just turn it on and 
just assume it works.  In that case, if you are trying hard to not have any 
false positives at all, publishing ?all or ~all is a good compromise for 
now.  There will still be others of us (myself included) who are content to 
publish -all and can deal with bounces as they come, and try to educate the 
receivers on whitelisting.  Nothing wrong with either approach, in my view.

> I am 'extremely' frustrated with SPF! I have been on this list since back
> in July or August. I have not read perhaps 90% of the postings as they
> were sometimes over my head and sometimes just not what I needed to know.
> I'm sure if I had read everything, I'd have a better understanding of
> SPF, but gee, asking all the sysadmins out there to decipher 'where we
> are' is in my opinion a bit much to ask. I want to put into place SPF
> records for the 600 or so domains on our systems... I 'want' SPF to 'make
> it'!

I understand your frustration.  Significant problems DO exist, and some of 
them have no ready solution.
I would classify SPF as "ready to be tested on medium to large scales" but 
it's certainly not ready for people to turn it on without paying attention. 
You're at least doing your part by publishing an SPF record, even with a 
?all at the end, it is a help to the overall effort.  I appreciate the time 
taken by yourself and others to test it out and report your experiences.

> My frustration comes in two parts.
>
> First, simply deciding on 'where' to begin. I think I will need to plan
> '?all' first and remain fluid as the standard evolves into a working
> system. My idea is to place under one of my domains a '?all' record and
> use that as an include or redirect for the rest of the applicable
> situations on the other domains. I'm trying to avoid micromanagement of
> those other 600 and perhaps a need to edit all of them again, when I can
> edit one during this 'fluid' time. I later plan to move forward with
> customized records for each domain. I can't even seem to get a clear
> 'Yes' or 'No' on that idea. And there certainly isn't enough information
> on the website to help me in my decision.

That seems reasonable.  If you have time to spend on analysis, I could 
recommend something like I did for altavista.com, which triggers an extra 
DNS query that I can log and analyze.  Even if you don't want to go to that 
level, publishing ?all is a good first step.

> Second, there is a huge lack of information. I feel pretty bad about even
> bringing this up as I know the people on this list have worked very hard
> putting in many hours. I did make a post maybe 6 months ago about the
> website or lack of one. I understand and see that it is at least 'alive'
> again. But, even to me, one who is not a complete neophyte with regards
> to SPF, the website does little good. One is just as likely to find old
> information as new information, some of the old perhaps being wrong? The
> bottom line is the website is of no relible use to someone new trying to
> set up their SPF record(s).

You are right about that.  SPF needs work in a few different capacities. 
The web site is just as important as actual testing, data gathering, 
experimentation, discussion, etc.

> Someone mentioned the promotion of SPF and it was mentioned that there
> was no money for promotion. The website could be and likely should be the
> main area for promotion, so the above statement changes from 'money' to
> 'time'.... which is not all that different, but still is different. As
> one who is trying to look at SPF from an outsider's point of view, I
> would say 'forget this!' Looking at this from a bit of an insider's point
> of view, my frustration level is high. I want SPF to make it... but I
> can't even come to a clear decision for my own system! It was suggested
> that I hire a consultant.. well gee.. I have never hired a consultant and
> if all the sysadmins wind up being asked to do that, SPF will never make
> it.
>
> I would like to with the utmost respect for the great people on this
> list, issue a challenge. I would like to see these great minds put
> together an effort to create the website the world needs. I would think
> that setting aside the work on SPF2 for a period of 2 weeks to 1 month
> and that time being put into efforts toward a useful website.
>
> The foundation to the 'house of SPF' is built, but almost no work has
> been done on the 'house'.

I quoted the above bit without clipping, because I think you're totally 
right.  The web site is what keeps attracting the right people to the 
cause.  I think it should correctly reflect the state of things, and should 
not try to convince people that SPF is ready for prime time.
Perhaps if we had a wiki, or a plone.org type of site, maybe people would 
more readily contribute their content and get it pushed live?  It seems to 
me that sitting down and doing the writing work is the most important thing 
right now.  Hopefully we can come up with an open structure that allows a 
LOT of people to contribute.
> Yes, if you feel I'm totally out of line here, go ahead and tell me to
> ride back out on that high horse I rode in on...  Otherwise, I will
> volunteer time and efforts towards what would be most helpful on the
> site, although I'm not so sure I know enough about the workings of SPF to
> actually do much in the way of creation.

It is appreciated.  I'm sure anything you can contribute would be welcome, 
even if it's just a record of your experience.  FAQs need questions as well 
as answers :)
I don't feel you are out of line.  It would be out of line if you were to 
reply dozens and dozens of times to say that the problem can never be 
solved, but since you are volunteering to be part of the solution, you have 
my respect.

> One idea I have, like was
> mentioned above, is FAQs, but we need a system very much like FAQs only
> perhaps called 'Scenarios', an FAQ based layout, with a menu of things
> like 'I have all my mail forwarded through my ISP', 'I send all my mail
> to my mailserver' and for each provide an explanation and an example
> record. Ultimately trying to cover all the 'Scenarios'.
>
> Clarifying the Wizard would be a big help, many of us on this list have
> complained about it, but I haven't noticed any changes to it.
>
> I'm very much wanting SPF to become a 'defacto accepted internet
> standard' (i.e. actually used by largeISPs). But I am extremely
> frustrated and this has built over the months and gotten a lot worse in
> the last two weeks. I know this is my problem,  not yours, but I feel
> this is how a lot of the rest of the world must feel.
>
> So how about it? How about some lowly web work? I know this is a bit like
> asking a Senator to type their own letter.. :)
All good ideas.  I have lost track... is there someone out there organizing 
the web work?  If folks like John and I start writing material, could 
someone organize it and stitch it in?  What's the current status of the web 
site and how can we help?
--
Greg Connor <gconnor(_at_)nekodojo(_dot_)org>