Re: New ideas for RFC2822 headers checking with SPF

William- thanks for taking the time to reply.

I am going to disagree with a few things you said, but let me preface it by 
saying that I think we are more in agreement than disagreement.  Scope 
choice is important.

Taking the last point first, because it influences everything else a lot:

Greg Connor wrote:
> > The main idea here is that I really do think that MOST domains will have
> > the same policy for multiple headers (that is, a list of mailservers
> > that  they trust to some degree or another).
--"william(at)elan.net" <william(_at_)elan(_dot_)net> wrote:
> I don't think this is so, the email infrastructure has too many
> complexities that do not make it possible to verify last-hop based on any
> single header.

This is one thing that was talked about extensively at Marid and in the 
early proposals of Unified SPF.
What you said is probably correct: It may be impossible to verify anything 
based on a single header.  BUT, that is not in conflict with my point, 
which was that we should take headers one at a time, NOT in combination.
The way I understand Unified, from when Meng published the slide show and 
when I spoke to him at great length about it right before that, was that 
there are a series of tests, and each test is a separate and distinct 
SPF-style lookup.  Every test within Unified is implemented by a function 
f(header,IP) = (pass,fail,unknown).
I disagree with you about whether one header can pass or fail by linking to 
another header.  I believe that is what was wrong with PRA.  Linking two or 
more headers together is not a bad idea, I just don't believe it adds 
enough value to justify the complexity that comes along with it.


> Microsoft got one thing right - there are number of companies and people
> who want to stop phishing. They said that is their main "motivation" but
> the way they go about it seems to indicate that is not and that was just
> good excuse for trying to push proprietary solution to become a standard
> even though they know its technically troublesome and politically
> directly  again open-source software.

Agreed.


> > There was another idea that would allow SPF2 to cover other headers, and
> > I  think does so a bit more simply than eh=, though it is probably not
> > compatible with SPF1 because it introduces a new macro.  The new macro
> > is  %{e} and expands to a keyword telling which header or identity is
> > currently  being checked.
> >
> > For most domains, I believe the policy for From: or HELO or Sender: will
> > be  very similar to their policy for MAIL FROM, so probably most users
> > will not  have to change anything.  If for some reason the HELO policy
> > is more  restrictive, or the From: policy is less restrictive, or
> > whatever, the  domain owner can break out the %{e} macro and redirect to
> > the appropriate  record.
> Macro requires new SPF protocol version to support it and also means you
> have to check header exactly as is instead of combination.

Right, that was the essence of the two questions I posed following this... 
I am making the point that 1. we should not screw with v=spf1, and 2. we 
should not create more layers of complexity by linking multiple headers 
together.  (See below for more detailed discussion about these two points.

> As has been
> discussed before making assertion for "From:" that it comes from certain
> domain or making it by itself equivalent to MAIL FROM is easily possible
> the number of errors would be too large.

I didn't quite understand that sentence but I think it will be addressed 
below anyway :)

> > Let me ask two questions to see if I'm on the same page as you are :)
> >
> > 1. Do we want to talk about v=spf1 or only about SPF2?
> >
> > Your eh= proposed addition would be compatible with spf1 records, and is
> > an  opt-in affair, but it is still a code change (since no existing SPF
> > parsers  understand the modifier).  I am sensing an active and vocal
> > segment of the  list who don't want to mess with v=spf1 at all.
> I think the way things are going, v=spf1 would stay for quite a bit longer
> then SPF2 and in fact SPF2 may become an expansion of original concept
> with new features that are all backwards compatible. That is exactly how
> I'd  like it to be.

Can you be more specific?  Are you suggesting changes to v=spf1, or are you 
suggesting that we make scope a feature of v=spf2 and leave v=spf1 alone? 
(I am proposing to leave spf1 alone and work on scope as part of spf2.)
It's OK to disagree with me :)  If so, let's be explicit about it so we 
won't run into context-related misunderstandings down the line...

> > My position was (and still is) that we should check ANY or ALL headers
> > and  data items, but not in relation to one another.  That is, we should
> > evaluate MAIL FROM by fetching the SPF info from the domain in MAIL
> > FROM,  and we should check the HELO against the domain in HELO, and
> > From: against  the domain given in From:, etc.
> That position seems somewhat incompatible with UnifiedSPF concept which
> is  that one identity can influence checks done on other identities and
> how its combination that matters in decision if the mail is to be
> accepted or not.
>
> I personally do not believe that its possible to enfore header checking
> based on single header, but the syntax provided will allow to make such
> as assertion i.e. like that MAIL FROM should be equivalent to only one
> header (From:) if publisher believes that to be for all his email.


I am going to disagree with you there.  The Unified SPF concept was "You 
should check a bunch of headers, each against their claimed domain".  It 
was pretty clearly NOT "You should check headers by comparing them to each 
other" and it certainly was not "You should check header A to see if the 
rules for that header contain requirements for other headers".
In other words, Unified SPF gives you a to-do list of headers to be 
checked.  Each item on the list is considered using the same recipe:
 - what header are you checking?
 - what domain appears in that header?
 - what is the SPF record for that domain?
 - does the sender IP pass the test given header,scope,ip?

I see a lot of value in adding scope to spf2.  I see relatively little 
incremental value, and a lot more confusion, in allowing spf records to 
link multiple headers together.

(Again it's OK to disagree with me but it will make it hard to have 
sensible discussions on the finer points later on...)





> > This was sort of the idea behind "unified spf" which I still think is a
> > good idea.  Unified did not give any header more importance, or power to
> > control other headers; it was just a way to check headers one at a time
> > using the same process over and over.
> That is idea of scoping support for SPF (i.e. SPF records used for more
> then just mail-from), but Unified SPF is in fact about making decision
> based on combination of identities, making one identity dependent on
> another especially if its close to current email practices is not bad.

Not to say that linking multiple headers is "bad" -- I just think it adds 
more confusion than value.
For example, if  mail from and from are different domains, and their spf 
records are conflicting, which one takes precedence?


> >              domain. IN TXT  "v=spf2 +mx +a +ptr  redirect=%{e}.spf.%{d}"
> > mailfrom.spf.domain. IN TXT  "v=spf2 +include:trustyisp.com
> > ?include:funkyisp.com ~all"
> >     helo.spf.domain. IN TXT  "v=spf2 -all"
> >     from.spf.domain. IN TXT  "v=spf2 +include:majordomo.spfaware.com
> >     ?all" *.spf.domain. IN TXT  "v=spf2 ?all"
> I agree with Meng that we want to avoid having to make publisher have too
> many records. The more complex the system is the less likely the
> publishers to adopt it (in addition both publishers and recepients
> would suffer from multiple dns lookups, number of lookups should be
> kept to minimum for greater majority of cases). Redirects should be
> avoided except with special circumstances.
>
> That is one of the reasons I never really liked scoping macro as the
> only way to deal with records of multple scopes, although I'd like to
> have it as an option for those who really can use it (but that does
> require SPF2 which would allow for new syntax like that).

I agree with you that the mechanism should not be complex.

I really think that 99% of domain names will be able to serve all their 
needs with one simple record that doesn't refer to scope at all.  This is a 
pretty important point, so let's try to see if we can agree on this.
Remember, we are talking about a single *domain*, and not a single mail 
message that might carry multiple domains.  One domain, one SPF record. 
Usually, that SPF record will be a simple statement of "What mailers are 
allowed to use my domain name in their mail?"
- If you use one domain for From and MAIL FROM, look up the SPF record 
twice and use the same criteria for both tests.
- If you use three different domains for HELO, From, and MAIL FROM (for 
example, mail1.domain, user(_at_)domain, return(_at_)bounce(_dot_)domain) then we lookup 
and parse each of the three, but I assert that none of them really need 
scope tagging.
If you actually have different usage policies for different scopes, there 
are other options probably more palatable than scope tagging in the record.
1. Write an SPF record that includes all possible uses of that domain, in 
any scope.  For example, if a domain may be used as HELO by certain 
servers, and as MAIL FROM by others, the SPF record for that domain should 
include all the relevant servers.  The edge case where a specific server is 
authorized to use MAIL FROM but is likely to forge HELO is extremely 
unlikely.  Point 1. is an extension of the idea that "you either trust that 
server, or you don't"
2. If you need to have different policies for different headers, you are 
still free to use different domains.  (If you already use different 
domains, no problem.)

I am having trouble even thinking of any cases where you would need 
different records for different scopes, which probably explains why I feel 
strongly that MOST domains will only need one record and can ignore scope. 
Can you think of some situations where a single SPF record doesn't describe 
all possible uses of that domain well enough?

As for whether to use a macro or a positional modifier, I don't think this 
is quite as important, and I would be happy allowing either or both.
I am just saying, let's spend time and effort making common uses easy, and 
other creative uses possible, but let's not spend too much effort making 
uncommon uses easy, as long as they are possible.
--
Greg Connor <gconnor(_at_)nekodojo(_dot_)org>