Extensibility, and error handling. Could SPF lose to XML over it?

I am surprised that nobody has really responded to this thread. If usingXML is such a hot-button issue, perhaps people would be interested inupdating the SPF spec to allow for better extensibility and error-handling?

I think XML allows you to extend the language, but still has the samelimitations when it comes to adding new features.

So... if people don't like the idea of being forced to use XML, what wouldwe like to see added to the existing SPF spec to allow for extensibility?

Part of the proposed merger would be that receivers honor both XML and SPF.MS expects that everyone who wants new features later will be compelled toswitch to XML. If SPF adds some extensibility language NOW, we could avoidforcing people to switch to XML later.

(Personally I don't have a strong feeling about plain SPF vs SPF-over-XML,but if it turns out that 2-3 years from now there are 100,000 plain SPFrecords and less than 1000 XML records, maybe XML is the one that will getleft behind. Who knows? Letting the market decide is a good thing, Ithink. Except, plain SPF is deficient in this area, and I will bet the XMLformat won't be :)

--"Stuart D. Gathman" <stuart(_at_)bmsi(_dot_)com> wrote:

While I oppose XML because of the patent problem, here is a specific
example of how XML extensibility would be good.

Proposals to extend SPFv1 to handle optional mechanisms and 'include'
or 'redirect' mechanisms which might fail all have one feature in
 common: they specify some sort of test or error trapping, and need
to delineate to scope of said test or trap syntactically.


--Greg Connor <gconnor(_at_)nekodojo(_dot_)org> wrote:

Very good point.  I touched on this briefly in my post on 5/22, Ideas for
semantic (feature) extensibility.  (Hopefully it actually went out to the
list... I didn't get any replies :)

It turns out that it is possible to add a little bit to the language, and
get some reasonable fallback behavior using modifiers.  For example, you
can tell SPF before you get to the include that you want to catch certain
conditions (like error, unknown, or even pass, fail) and take certain
actions instead of the default action (like break, continue, error,
unknown, pass, fail).  It is possible to do this without nesting (you
could actually use include or redirect as a kind of nesting if you want.)

This kind of logic could be used for multiple things:
  graceful failure or error handling
  skipping over unusable mechanisms. so that the sender can specify the
fallback   even including someone else's record and re-casting the
results, for example "Go look at comcast.net, but what they call "Pass" I
call "Unknown", and what they call "Fail" I call "Come back and look at
the rest of my record"

The key to doing this without nesting is defining the behavior well ahead
of time, and possibly relying on included or redirected records being
able to specify one fallback position, parse some terms, and then specify
a different fallback for the next few terms, and so on.


Granted, XML would be one way to make nesting work, and it would be
logical and easy to understand, but that doesn't give you feature
extensibility; you still need to do the hard work of defining the
fall-back plan really intelligently, or letting the user specify the
fallback plan.  I am not saying either SPF or XML is inherently better,
just that we need to do the hard work of deciding what to do when unknown
mechanisms come along, and decide it well ahead of time.



--Greg Connor <gconnor(_at_)nekodojo(_dot_)org> wrote:

I will take a stab at identifying some requirements and methods for
semantic extensibility.  Please comment... I would like to collect
comments from SPF users and then include MS folks in a second round of
comments if we all feel like we're going in the right direction.


Here are the goals I want to get to:

1. Let's assume that new keywords and new features will crop up.
2. Assume that any client might see some keyword or language element for
which it doesn't have the feature, and in most cases doesn't understand
the new feature. 3. Allow the publisher of the record to define the
"fallback position" (i.e. the logic for dealing with an unknown feature.)
There are a wide range of different situations... we can't really solve
the problem by mandating one and only one fallback position. 4. The
publisher needs to know predictably and reliably what will happen on the
receiving end, both with the new feature present and without.

With these in mind, rolling out a new feature should be possible, but we
would need to add a little bit to the language.


(I am sidestepping XML vs SPF TXT for now.  Here is how. :)  Based on
some conversations at the meeting, we think that XML allows for "syntax
extensibility", meaning you can add new keywords to the language pretty
easily.  It's also fairly easy to add to SPF TXT using the same
concepts... in either case, the hard part is that you must agree on what
to do when you come to an unknown mechanism.  The examples I'm using here
look like SPF, but please don't take that to be an endorsement of one or
the other.  For the record I support the proposed compromise of "SPF and
XML running concurrently for now" and it seems likely that we will need
to solve both problems anyway.)


The essence of the question is, "Here is this unknown/unimplemented
feature, and I can't use it like you want me to, so I need to skip over
it intelligently while still honoring your intentions."

Here is an example of one possible fallback situation

  mydomain.com. v=spf1 +ptr +mx +domainkeys -all

"If the message is not from my servers (by ptr or mx) it may also be
allowed if it has the new feature "domainkeys".  Otherwise, no exceptions
are allowed."

Here are a couple of ways to state what this publisher probably wants:


0. The current SPF strategy is:
    v=spf1 +ptr +mx ?all
    v=spf2 +ptr +mx +domainkeys -all

This allows you to do exactly what you want, but the version numbers of
the record mean that you have to repeat yourself.  Also, some folks have
given feedback that the version number would limit SPF to only those
features recognized by a centralized version numbering authority.  We can
sort of get around this by pointing out that version numbers could be
like 1a or 1+ or similar.  But, if there is more than one optional
feature released later, you may get an even worse situation: 1, 1d, 1j,
1dj.  Also, the backward compatible records will be slow to go away...
and since they are all TXT as far as DNS is concerned, you have to return
all of them, hence there is less space for each iteration.

So, what follows are three additional methods/mechanisms that would add
by-feature extensibility with a predictable fallback.


1. Structured logic:
    +ptr  +mx   IF domainkeys { +domainkeys -all } ELSE { ?all }

The structured logic provides a very specific "fork" in the road... if
the feature is available, take one track, if not, take another track.
Either track can have multiple follow-on steps.   This is a somewhat
heavy solution syntax-wise, I think, because we are adding delimiters and
nesting, as well as conditional tests.  (I am pretty sure we don't want
1. -- it's just here for comparison with what other higher-level
languages might do.)


2. Choose one and only one failure action.
    +ptr  +mx  unsupported=?  +domainkeys -all

The modifier "unsupported" tells what action to be taken when an unknown
feature is encountered.  That action will probably be one of:
unsupported=?  Stop processing and return "unknown" ("neutral")
unsupported=-  Stop processing and return "Fail"
    unsupported=+  Stop processing and return "Pass" (pretty unlikely ;)
    unsupported=abort  Stop processing and return error result  (current
behavior)     unsupported=continue  Ignore the unknown keyword and resume
with the next step This provides a predictable action if the feature is
not supported, though it will only do one action, not a new sequence.
However, the record could include another unsupported flag later in the
record, so that one feature fails using "unknown" and another feature
fails using "Fail".


3. Trap style
    +ptr  +mx  trap(unsupported=break){ +domainkeys -all }  ?all

There is no "test" as there would be with if/then/else, but there is
nesting, which allows you to do pretty much the same things.  The trap
could be any one of the actions from #2, in addition to "break" which
means "skip to the end of the trap section, then continue".


Brief discussion comparing 1, 2, 3...

My thought is that 1 is the hardest to do and 2 is the easiest (and
closest to the current SPF spec).  3. introduces nesting, but not
conditional tests, so it should be relatively easy to implement, but
because it is "structured" it will give people the "illusion of
complexity".

I feel that 3. is the direction that gives us the greatest number of
options later down the line.  With a Trap{} mechanism you would be 100%
sure of what will happen if your features are missing, and it also gives
some more clever options that linear processing doesn't give you.  BUT,
there are probably very few cases that would need 3. that are not solved
adequately by 2.

If we were talking only about SPF alone, I would probably lean toward 2.
to keep the nesting out of it and keep everything linear.  However, if we
are thinking about XML, it already has this type of nesting built in.
Consider the following xml-like snippet.

  <ptr/>
  <mx/>
  <trap unsupported=break>
    <domainkeys/>
    <all action=FAIL />
  </trap>
  <all action=UNKNOWN />

So, I guess given the courtship period we are entering with MS CID, I
would want to take a second look at option 3. (Trap mechanism) and not
dismiss it out of hand.  However, I would love to hear feedback from
others on this.


(Here is where I digress a bit from "adding features" - this example is
about more things a Trap mechanism would allow us to do, other than
feature extensibility)

The "trap style" could also be extended to protect you from other errors,
or even to include/reference someone else's record in a way that you
control.  For example, here is another scenario altogether that might
benefit from a trap mechanism.

  mydomain.com:  +a +ptr include:comcast.net include:earthlink.net -all
  comcast.net: +a +ptr -all
  earthlink.net: +a +ptr -all

In this case, you are saying that your mail may come from your servers,
Comcast, or Earthlink (perhaps you have users on those two networks).  If
I understand include: correctly, if the result for comcast.net is PASS,
then the result is also PASS for mydomain.com.

Here is what happens with include:comcast.net..
  comcast.net results:    mydomain.com results:
  PASS                    PASS
  NEUTRAL, FAIL           CONTINUE
  ERROR                   ERROR
  No SPF record exists    FAIL

I believe the redirect=comcast.net would return either Pass, Fail, or
Unknown, depending on whatever the straight comcast.net test would have
been, but since their record ends in -all (or any "all") then my
processing would never continue on to Earthlink.  (Comcast could assist
me by publishing a redirect.comcast.net that has +a +ptr +mx but no -all,
I think if the redirect doesn't contain an "all" it is possible to come
back to the main line.  But, still no way to change a + for them into a ?
for me)

Now, let's say what I *really* want is to return "unknown" (neutral) for
Comcast, since I have some users, but there are also a lot of forgeries
coming from there.  If Comcast says it is a PASS, maybe I really want to
interpret that as an unknown.  In other words, I am going to (include,
redirect, whatever) to comcast, and I'm fishing for a + or ? which I will
treat as a ?, and if I get a - or error I will move to the next fork.

So, what I would like in this case is something like this...

  mydomain.com: +a +ptr
      trap(+=?,?=?,-=break,error=break){ redirect=comcast.net }
      trap(+=?,?=?,-=break,error=break){ redirect=earthlink.net }
      -all

Anyway, that's a quick example of how a trap might help in other ways not
having to do with adding features...  Kind of a "nice to have" but not a
hard/fast requirement, I think...


--
Greg Connor <gconnor(_at_)nekodojo(_dot_)org>