Re: Re: op=dk

Frank Ellermann wrote:
> Radu Hociung wrote:
>
>  [DK modifier]
>
> > Actually it would mean "My DK information is at ...",
> > or "My PSG information is at ...".
>
> Okay, then that's no simple opt-in option, maybe it's a proper
> modifier like redirect= or exp=, e.g. dk=.  And what's the
> exact meaning after I found whatever-it-is at the given domain
> in conjunction with SPF ?
>
> How does this dk= modifier "modify" SPF results, if at all ?
None of the o* modifiers have any effect at all on SPF results. The SPF 
client should simply relay their contents to the MTA. If the MTA knows 
what to do with them, fine, otherwise, just move on.
> > The call to the spf_check function would return the SPF
> > result as per SPF rules, and also any "other" information it
> > may find:
> Among these results could be PASS, SOFTFAIL, and FAIL, what's
> the role of DK in these cases ?  It's clearer for UNKNOWN, then
> the dk= simply means "maybe use DK or forget it".
DK is not really a good example, because, like you mentioned, DK uses a 
selector + _domainkey extension to find the location of the key. 
Besides, that combination is not known until the message has been 
received, as the "selector" is given in the DomainKey-Signature: header.
But for this example, let's say that the domain owner publishes an 
odk=yes extension. As I have shown above, it makes no sense the SPF 
record to specify the location of the DK record, but perhaps only it's 
existence.
In this case, say that the recipient MTA gets an SPF result of FAIL on 
their check of the MAIL-FROM. The SPF checker passes the 
spf_also_found["dk"]="yes" to the MTA.
In order to decide whether to reject after the SPF fail, or to give it 
one more chance using DK, it may use any other information it has 
available. Perhaps the RCPT user is known to make heavy use of 
forwarders, so a lot of his mail gets FAIL SPF results, even though it 
is legitimate.
At this point the MTA may decide to receive the message anyway, instead 
of rejecting it right away due to the SPF fail. Then it would decide 
whether  the message is authentic if it finds a DK signature and if the 
signature checks out.
For a site that doesn't have any such information, the odk=yes would be 
completely meaningless, so they would have to act only on whatever 
whitelists, blacklists they use and also on the SPF result (FAIL in this 
case).
How the various spam fighting methods are combined is the MTA vendor's 
problem. SPF cannot say anything on the subject. SPF really only 
provides one input into the spam software's decision making process.
Where the usefulness of the extension is most clear is when SPF3 gets 
published and starts being adopted. Let's assume that SPF3 operates on 
the message headers. Thus, it is evaluated after SPF1, but its results 
make sense even if SPF1 is a FAIL. Let's also assume that SPF3 is 
designed to be a 1-query protocol, and it contains a signing key or 
something like that, so there is no redirect=mechanism:
As the domain owner, one has to plan how to publish their records as to 
impose the lightest DNS load on themselves.
This means that they have to share the 400-bytes available in the TXT 
space between the two SPF records. Let's assume that the two records are 
300 bytes each.
Given the design of the mechanisms, the only possible plan is:

1. Publish SPF3 in full in the TXT record
2. Publish 100 bytes of the SPF1 record and extend it with redirect=

This would work. But at the time SPF3 is invented, SPF1 is very popular, 
and the installed base of SPF checkers is very large.
This means that just by publishing SPF3, the domain owner will double 
the number of queries received from the installed base of SPF checkers.
On the other hand, if the SPF1+3 checkers know about the ospf3= 
extension in the SPF record, the domain owner could take the following 
decision:
1. Publish the 300 byte SPF1 record at the top domain, and add ospf3=_s3
2. Publish the 300 byte SPF3 record at _s3.domain.com

Now, by publishing a new SPF record, the number of queries seen from the 
large installed base of SPF1 checkers is the same as before, except that 
each response is 10 bytes longer.
The number of queries to _s3.domain.com would slowly ramp up as the 
adoption of SPF3 grows.
The SPF3 checkers already know to look for spf_also_publishes["spf3"] if 
we specify the o*= extension in the SPF1 check.
I am assuming that an MTA vendor would use an SPF3 implementation that 
are also supports SPF1, so the same library is called both for the SPF1 
check as for the SPF3 check. Thus, it will be aware of the o*= extension.
The SPF3 specification would have to provide guidance to the 
implementors to look for an ospf3= keyword if an v=spf1 record is found.
If the SPF3 does not specify such guidance, the ospf3= makes almost no 
sense.
The cost of not guiding implementors to look for ospf3= is that adoption 
of SPF3 will be slowed by the fact that a domain that wishes to publish 
SPF3 would double their DNS costs for very little or no benefit, until 
the proliferation of SPF3 checkers becomes comparable to the 
proliferation of SPF1 checkers.
The alternative would be for SPF3 records to be specified with a 
residence at _spf3.domain.com. Unfortunately this means that MTAs would 
have to "hunt", first at domain.com for an SPF1 record, and then at 
_s3.domain.com for an SPF3 record.
This "hunting" will not go well with the regulatory bodies that have to 
promote SPF3 to the standard track.
So, without an extension like o= in SPF1, the barrier to entry of future 
 DNS-based methods is much higher than with the o= extension.
This is because SPF was the first to hog the top-level TXT space. If it 
was another protocol, it would be sensible for that protocol to make 
these kinds of provisions for the future methods.
This o*= extension would have been much harder to implement if Sender ID 
had been successful and if it used the top level TXT record, because the 
o* function would have to be built into Sender ID as well as SPF.
I think it is a priviledge for SPF to be allowed to use the limited real 
estate in the top-level record, and with priviledge, responsible use and 
planning is needed.
> > For instance, if an SPF record specifies
> > spf_also_found["reputation"]=blah.com, the receiving MTA
> > should probably ignore it, as it's the recipient's choice of
> > which reputation services to check, not the sender's.
>
> Phil and Meng intended to publish an I-D in this direction,
> draft-hallambaker-accreditation-00.txt, but that never worked,
> it didn't pass the I-D nitpicking.  For an unofficial copy see
> <http://purl.net/xyzzy/home/test/> + draft-hallambaker etc.
Interesting. I see also why it did not work.

I also pointed out that having an SPF record that specifies 
oreputation=blah is kind of useless.

> > I would prefer that all o* modifier namespace be set aside
>
> op= is older, pick another character.  In Phil's scheme it's
> accredit= plus accredit.substuff= IIRC.  You could in theory
> reserve some= plus all some.thing= for arbitrary values of
> "some" (like "p") and arbitrary sub-modifiers "thing" (as in
> "p.whatever=").
I'm not terribly partial to "o", but if "op" has been in use already, it 
would be the responsibility of newly specified methods to not recommend 
the use of "p" as their recognized extension.
> It's not exactly necessary to invent a general rule for this
> concept now, you could simply say that you need "p" (or "o"),
> and all possible modifier names p.whatever= (or o.whatever=).
>
> For obvious reasons you'd better stay away from exp, redirect,
> accredit, op, match_subdomains, default, etc., and you'd also
> avoid the names of mechanisms like "a".  I've forgotten some
> of the proposed names for positional modifiers, one was "not=".
I do not agree with this argument, as the syntax makes it clear what is 
a mechanism and what is a method.
There is no way to mistake a mechanism for a modifier that I can 
imagine. Thus the two namespaces are distinct, and cannot have 
collisions, as far as I can tell.
I would not use the "." as it provides no value other than a little bit 
of readability. But I'd be willing to hear more arguments in favour of 
the incremental readability.
One reason to make the extension as efficient as possible is because 
several extension may be necessary to be published, and the dot will 
take space each time a new extension is needed.
Since the shortest possible extension would be 5 characters, (e.g. 
"<SP>or=1"), the 100 byte space in the example above can accomodate a 
maximum of 100/5=20 simple extensions. If a dot were added, the number 
of possible extensions would be 16. That's a 20% reduction.
Another thing that would have to be specified is that the o*= extension 
content should be as spartan as possible, as opposed to verbose, in 
consideration for future needs.
eg.
oabc=_a

  should be preferred to:

oabc=_myabcrecordat.mylongnameddomain.com


> > From a system perspective, all these details matter
> Fine, but a say op=pgp is still useless, and a modifier like
> pgp=domain.of.keyserver is also a PITA, if it's irrelevant for
> the purposes of SPF.  We also don't need a homepage= or ssn=,
> we have the hard limit of UDP for q=spf and q=txt DNS replies.
Exactly. The SPF publisher would have to plan what they publish.

But something like ohomepage=... only makes sense if there will be in 
the future some protocol that requires the use of the TXT at top level, 
and they specify their desired SPF1 extension to be "homepage"
Perhaps common sense in the "homepage" camp, as well as in the adopters 
will prevail.
But then, SPF only needs to make this extension available officially, 
not regulate its use.
> > some of the extensions don't make sense, and the record
> > publisher should know not to waste TXT record bytes by
> > including them.
> Yes, and an op=dk or a dk=domain or a p.dk=whatever should be
> analyzed very carefully.  The obvious problem as stated in the
> op-paper, the absence of a modifier (excl. exp= and redirect=)
> means nothing.  The presence of a modifier can be also ignored,
> old implementations don't know it, and new implementations are
> still free to ignore it.
Good point.

The absence of the "other" is meanigless, but then the SPF3 
implementation should expect to find the info it's looking for at the 
top-level TXT, and nowhere else (ie, the SPF1 and SPF3 records sharing 
the same TXT space of 400 bytes).
This also means that in the spirit of enabling as high MTA performance 
as possible, an SPF1 implementation should pass any other TXT records 
found at the top level to the MTA. The MTA would then pass the extra TXT 
records received from SPF1 to SPF3, so that SPF3 would not need to do 
the resolver call to find it.
Transfering that information in memory is faster than if SPF3 needs to 
do a DNS query, even if the query goes to a cache on the local network, 
which would still take milliseconds or tens of milliseconds. The 
in-memory passing of the extra records takes nanoseconds. (ie, a million 
times faster)
This behaviour does not need to be specified in the SPF specification, 
but rather it's something that MTA implementers that want high 
performance might want to consider.
> So it should be very useful in the remaining cases, and so far
> I don't see what a p.dk=whatever does for a receiver.  If the
> receiver wishes to support DK it's still forced to test DK in
> the way offered by DK, because the absence of a p.dk= has no
> meaning.
The extension is not very useful for DK, since DK does not expect to 
share the TXT space at the top-level.
But it may be marginally useful, as I've shown, in deciding whether to 
reject based on SPF FAIL alone, or to allow a second chance using DK.
> A useful modifier could be a hypothetical op=nodk, if it stands
> for "don't waste your time with DK queries, I don't use it".
Yes, but the publisher would have to care about the local performance of 
the receiving MTA, and be willing to add 8 bytes worth of DNS bandwidth 
to make the recipient's mail faster.
It would be nice, but I don't see much incentive for this.

In the case of ospf3=no a possible incentive would be if the TTL of the 
SPF record is very low, even 0, which would cause the subsequent SPF3 
query to not use the recipients DNS cache, and instead generate traffic 
to the publisher.
Anyway, even if there is not too much need for the extension right now, 
in the future it may just provide a way out of an over-crowded TXT record.
Good engineering includes some foresight, I think, and this extension is 
just that.
Regards,
Radu