Re: Drive Towards Consensus [was Re: On Extensibility in MARID Re cords]

A lot of good discussion is already going on, and I don't want to take too 
much time by repeating what others have said.  Therefore, I will try to be 
brief  (Hmm, I guess "try" is the operative word here :)

First off, extensibility is not bad.

I don't think anyone is seriously arguing "extensibility is bad".  Rather, 
I think everyone has their own ideas about how important "extensibility" is 
compared to other attributes like simplicity, human readability, 
byte-economy, etc.  Since this is a pretty wide spectrum, and "correct" 
answers lie all along the scale for various stakeholders, I will not try to 
pull anyone in any direction.  The best I can do is encourage everyone to 
take a few moments to rank qualities in their minds that make up a "good" 
proposal.

Second, let me stress that SPF syntax actually IS extensible.

SPF provides four vehicles for extension.  Let me review them quickly -- I 
know people are familiar with the draft and SPF site but just in case it 
wasn't stated clearly, here they are.

1. Modifiers

Modifiers are attribute=value pairs that the receiver may ignore if he 
doesn't understand them.  For example
example1.com. IN TXT { "v=spf1 a mx ptr "
        "rate=100 +include:example1.com.cust-spf.esp.com "
        "rate=133 +include:example1.com.spf.personalmail.com ~all" }

In this example (based on Margaret's first scenario) the "extra" 
information is understood by whoever wants to understand it, and ignored by 
anyone who doesn't.
(One sort of limiting factor about modifiers is that they are considered to 
be order-independent and the current spec encourages them to be placed at 
the end to make this clear.  If I were to propose any modification to the 
current spec, it would be to allow modifiers to be placed inline allowing 
for the possibility that they might apply/associate the terms appearing 
after them, if appropriate.  This minor change in the spec should not 
affect any current parser or published records and would just be a 
clarification...)

2. exists: mechanism

exists: makes it possible to do something DMP-like (factored) plus a lot 
more.  Most sites will not need it, but those that do will be quite 
thankful it is there.
I am describing exists: as a "vehicle for extensibility" because it makes a 
lot of things possible that would not otherwise be achieveable, including 
many we have not yet thought of.  Let me use an example based on Margaret's 
third situation.
example3.com. IN TXT "v=spf1 redirect=example3.com.cust.maridrus.com"
example3.com.cust.maridrus.com. IN TXT { "v=spf1 a mx ptr "
        "-exists:CL.%{i}.FR.%{s}.exceptions.maridrus.com. "
        " ?all abuse=abuse(_at_)maridrus(_dot_)com" }
exceptions.maridrus.com IN NS ns-log1.maridrus.com.
exceptions.maridrus.com IN NS ns-log2.maridrus.com.

In this example, example3.com delegates their MARID policy management to a 
third party "maridrus.com".  They in turn use an exists: check which is 
(usually*) triggered only if the a, mx, and ptr tests are not met.  By this 
mechanism, the receiver is required to issue a query ending in 
exceptions.maridrus.com and containing the IP of the sending client, and 
the return path of the questionable message.  The queries are then logged 
and unforseen sources of example3.com mail become exposed.  (This is also a 
GREAT way to monitor how well your MARID record is doing at catching all 
situations, and also a great way to find forwarders that have not been 
white-listed by the receiver.)
(* I say "usually" here because all currently existing SPF implementations 
have the processing happening in strict order, but if a streamlined 
parallel-type processor is created later, it's not much of a problem.  As 
long as the receiver re-assembles the steps in the correct order and 
arrives at the same result, it is permitted to make "extra" queries -- it 
is the responsibility of the domain using exists: to screen out 
false-positives.  It's also possible for a malicious sender somewhere to 
just make extra queries to throw off the logging - anyone doing logging 
would need to be alert for this too.)
My second reason for describing "exists:" as a vehicle for extensibility is 
the little-known secret of exists: the domain owner may decide to customize 
the DNS server itself.  For example, let's say that the maridrus.com 
service provider wants to do something with the "exceptions.maridrus.com" 
zone that a static zone file can't do.  They want to answer YES to all 
exceptions *unless* a threshold is reached, and then reply NO to a certain 
IP, or sender address, or both, if that IP or address has been used too 
many times outside the policy.  They can teach their DNS server to answer 
NXDOMAIN for the first 100 uses of any IP or email address, and then answer 
127.0.0.2 after the limit is reached.  This requires a non-standard DNS 
server, but the sender may bear 100% of the costs of this type of 
extensibility and still be within the standard.  (I know the idea of 
returning inconsistent results will make Jonathan twitch, but the fact is 
that DNS servers that serve dynamic/varying information are already out 
there, e.g. 3DNS/akadns :)

3. Unknown/non-standard mechanisms.

The behavior when we come to a non-standard mechanism is well-defined: 
currently the spec in use today says that processing should stop 
immediately and an unknown result should be returned (as if ?all had 
appeared where the non-standard mechanism is).  This is not a "forbidden" 
or even a "bad" practice - it simply gives a reasonable escape clause if 
you decide to use a mechanism that is not in the pre-existing core set of 
features.
The SPF spec actually uses domainkeys as its example in the draft.
 v=spf1 a mx ptr domainkeys:_dk.%{d} -all

Basically, this means, if you understand what domainkeys: means, proceed 
ahead with either accepting the mail with DK, or rejecting unsigned mail 
with confidence.  But if you don't understand what domainkeys: means, the 
record behaves like this for you:
 v=spf1 a mx ptr ?all

That means there IS a way to add semantic (feature-wise) extensibility, but 
only one way.  The "default to unknown" rule is a reasonable default that 
probably applies well to 90%+ of the cases where you might ever want a new 
non-core feature.
I know that I have proposed alternatives to this default in the past, but 
frankly I have changed my stance on this a bit, after listening to all the 
feedback on what people really want from extensibility.  Semantic or 
feature-wise extensibility scares people, a bit.  It's not 100% clear that 
we need it built in the spec, so I am starting to think that "default to 
unknown" is an extremely reasonable rule.  Basically, this is a sane 
default that *DOES* allow for semantic extensibility -- if the sender and 
receiver both choose to use a new keyword, they can -- but provides a 
safety valve that everyone understands.  I doubt that people really want 
more feature-wise extensibility than this, but if they do, they can 
continue on to the next section.

4. A new version tag

This is basically the equivalent of publishing a new/enhanced spec, for 
example:
nekodojo.org. IN TXT "v=spf1 a mx ptr ?all"
nekodojo.org. IN TXT "v=marid2 a mx ptr on_error=? { magic:happens } -all"

But keep in mind that *anyone* can publish new TXT records with a different 
prefix; they are not necessarily constrained to obey only the v= values 
that we dole out.  For example:
nekodojo.org. IN TXT "v=spf1 a mx ptr ?all"
nekodojo.org. IN TXT "v=spf1+d +domainkeys:valid 
?exists:%{ir}.non-compliant-lists.yahoo.com -domainkeys:unsigned -all"
If length of either becomes a problem, we can do what we always do: use 
redirection for one version or the other (or both).
I know the idea of coming out with a new spec bothers people, but I think 
PHB's presentation at ETC last week underscores the need for this type of 
upward mobility: the perfect is the enemy of the good.  We can produce good 
protocols and update them after deployment, much faster than we can roll 
out perfect protocols.  (Demonstrated using example of SSL exploits and 
updates/fixes).


Finally, let me close with asking "What kind of extensibility is *enough*?"

Mark addresses this more eloquently than I probably can, so let me quote 
him a bit here.
--Mark Lentczner <markl(_at_)glyphic(_dot_)com> wrote:
> I understand the desire to design a more comprehensive mail policy
> format.  And I understand the desire that such a format be extensible for
> future approaches that are now only contemplated.  However, the proposed
> XML syntax doesn't achieve this either:  While it is true that XML
> provides a much richer syntactic framework in which to extend an original
> document format, it says nothing about the semantic meaning of such
> extensions.
>
> I maintain that, in the problem space these projects were designed for,
> namely authentication of the origin of mail based on the domain name, the
> SPF classic syntax does just fine.  There is a little room for extension
> (defined both syntactically and semantically), just enough to handle a
> few future ideas within this problem space.  I think Dave Crocker's
> posting from earlier today
> (http://www.imc.org/ietf-mxcomp/mail-archive/msg02139.html) speaks for
> this approach eloquently.  When a larger framework for mail policy
> emerges, SPF can easily add a modifier for pointing to it.  Or perhaps
> SPF will have run its course, and the newer framework will simple take
> over.  Either way is fine.
>
> Solving the original small, but important slice that all of the original
> projects targeted should be our first priority.  If we can do that, then
> we show the world, and ourselves, that we are capable of tackling more.

I interpret this to mean:

1. Some extensibility, especially relating to our target problem, is good.

2. Our spec maybe (probably) will be part of a larger "email policy" 
framework in the future.
3. Creating the larger email policy is not exactly our job.  Creating a 
vehicle flexible enough to carry such a framework is not exactly our job. 
Perhaps we COULD do it, but I am concerned that trying to do so might get 
in the way of other objectives.
4. We all understand that the LMAP problem is but one step on a long road. 
If our spec becomes outdated, superseded, deprecated, dropped or whatever, 
it's probably because something better has come along.  That doesn't mean 
we failed.  If our spec is superseded in a year or two, and during that 
time it makes other things possible (like domain-based reputation systems) 
then we will have succeeded.
Thanks,
gregc



P.S.  You know, who says we have to stop working together when our first 
spec is proposed standard?  I think it's entirely possible that we can keep 
going with other proposals, best practices statements, etc.  If not us, 
who?  :)
--
Greg Connor <gconnor(_at_)nekodojo(_dot_)org>