[Top] [All Lists]

Re: SPF I-D for review: draft-schlitt-spf-classic-01.txt

2005-05-22 14:21:50

On Fri May 20 2005 16:15, wayne wrote:

There is a new I-D for the SPF email anti-forgery system available for
review.  This draft tries to document the current practices of the
[remainder of marketing blurb elided]


The ID-tracker says that the intended status -- which is nowhere mentioned
in the draft itself -- is Experimental.  Experimental RFCs typically
define a specification which is part of a research or development effort,
not "document [...] current practices" -- that would be an Informational
RFC (or for non-protocol issues, a BCP RFC).  In either case, there may
be problems with the draft's use of BCP 14 imperatives and other keywords.
Experimental in particular would seem inappropriate for another reason
noted in the IESG comments; viz. that the draft attempts to document a
version which has apparently been superseded.  Moreover, the message to
which this is in response claims a different intended status (see below).
Discussions about this, and previous, drafts have been taking place on
the spf-discuss(_at_)v2(_dot_)listbox(_dot_)com mailing list.  To subscribe 
or read
the archives, see

While I have been reading this mailing list for many months and will
note any comments posted here, sending email to the spf-discuss list
or to me personally is probably better than flooding this list.

The draft notes "The SPF project DOES NOT expect the IESG to review
   this version.  Instead, a two week pseudo last call will be put out
   to collect any final changes.  After that, a new I-D will be
   submitted with the expectation of an IESG review.".  I have copied
the IESG mailing list so that the IESG can be kept informed of the
discussion -- if the intent is as stated not to subject the draft to
further public review in view of the IESG, then it would at least seem
appropriate to have suggested that copies of comments should be sent
to the IESG list.  If the intent is in fact to change the intended
status from Experimental (I-D tracker) to Standards Track (as stated
by the author in the predecessor of this message; see below), then
the IESG may wish to initiate a New Last Call of at least 4 weeks
duration in accordance with the procedures for an individual
submission for Standards Track status as outlined in RFC 2026 (BCP 9).
I realize that the whole subject of SPF (and similar systems) has a
certain amount of controversy to it, but for the purposes of this
draft, I am very reluctant to try debate these issues.  The goal is to
document a de-facto standard.

Use of the word "standard" is going to cause problems for either
Informational or Experimental status.  I note that the document uses
the term standard in several places, including:

o the IPR boilerplate, which is probably inappropriate for a draft or
  RFC intended for Experimental or Informational status, or one that
  has been moved to Historic status.  I realize that the boilerplate
  is not the author's handiwork, and that authors currently have zero
  latitude to correct such errors in boilerplate according to recent
  IETF Secretariat policy.  I mention it to bring the issue to the
  attention of the IESG, which may choose to consider the problem.

o in the BCP 90-derived registration template.  "Standard" status would
  be inappropriate for an Experimental RFC, or for one which provides
  Informational documentation of an obsoleted or otherwise undesirable
  mechanism.  In the case of an Experimental RFC, the registration
  should probably be in the Provisional, rather than Permanent registry,
  and the status per BCP 90 section 4.2.1 should reflect the actual
  intended status corresponding to the draft's publication, viz.
  "experimental" or "informational", both of which are explicitly
  provided for in BCP 90 section 4.2.1.  That section also provides for
  "obsoleted" and "deprecated", which may be appropriate in accordance
  with issues registered in IESG comments and as discussed herein.

Debates about better techniques, why 
SPF is evil, etc. are probably best discussed on things like the IRTF
ASRG list, SPAM-L, the NANAE newsgroup, or on spf-discuss on a
separate thread/subject.

On the contrary, I believe the IESG should have the benefit of
community input on such matters, as that may help to gauge community
consensus on relevant issues.  In particular, I concur with at least
parts of the noted issues regarding "abuses" of and/or "concerns with"
DNS as it pertains to the draft.  Public discussion in plain view of
the IESG may also raise issues of concern which had not been previously
brought to the IESG's attention.

SPF does not directly change RFC2821 or add any extentions to SMTP.

Indeed, one of the IESG comments suggests that a more appropriate
mechanism might have been to use such an extension.

It does, however, allow for domain owners to say which IP addresses
are allowed to use their domains in either the MAIL FROM or HELO/EHLO

The MAIL FROM argument specifies a return path for delivery-related
notifications.  As such, it does not carry the semantics of the sender
(as in "Sender P[...]").  [it may coincidentally happen that the sender
wishes delivery-related notifications to be sent to a personal mailbox,
however it may also happen that such notifications are intentionally and
legitimately directed to a shared mailbox of some sort or delegated to
another person (e.g. a mail system administrator) for handling] Moreover,
except where some redirection of delivery takes place (e.g. at a mailing
list exploder), the MAIL FROM return path remains unchanged (in the
current state where source routing has been deprecated for a couple of
decades) as the message is sequentially forwarded through multiple MTAs,
each of which will necessarily have a distinct IP address.  That makes
the MAIL FROM argument unsuitable for the mechanism described above, both
because of the semantics which differ from what is claimed and the fact
that the argument remains unchanged during the store-and-forward process
which entails (possibly) multiple "hops", each with distinct and largely
unpredictable IP addresses of the sending MTAs.  Moreover, the MAIL FROM
return path is explicitly permitted to be a null path, in which case
there is no domain.  Indeed, a null path is required in some cases to
prevent damaging mail loops.

At a bare minimum, if the intent of the draft is in fact to document the
described mechanism, it should clearly delineate and emphasize those
rather serious defects in said mechanism, defects which are only
partially and begrudgingly acknowledged in the draft, and which are
undermined by language such as "It is RECOMMENDED that domains publish
SPF records".  In particular, note that no authentication "failure" can
meaningfully be attributed to any tests based on MAIL FROM argument
domain vs. IP address, for three separate reasons explained above, viz:
1. MAIL FROM path may legitimately be (required to be) null.
2. MAIL FROM path does not correspond to a well-defined set of IP
   addresses as a message traverses the mail transport store-and-forward
   process, as IP addresses vary at each "hop", whereas the MAIL FROM
   path is usually invariant.  I.e. the MAIL FROM path which is
   associated with a message does not correspond to a set of IP
   addresses of predetermined (set) size or content.
3. The MAIL FROM path is intended to specify the mailbox for *receipt* of
   delivery notifications, which may be (and often is) legitimately
   unrelated to the notion of a *Sender's* IP address (e.g. mobile users
   of the mail system, who may wish to receive delivery-related
   notification messages at one (of possibly several unrelated)
   mailbox(es) but who may send mail from essentially any IP address).

[I would add that lookup of MX records corresponding to the MAIL FROM
path, regardless of whether that would be in conjunction with an SMTP
extension, would not resolve the inappropriateness; MX records and the
return path are both intended for *delivery* (specifically of
notifications in the case of the return path) rather than the *sending*
of mail.]

The HELO or EHLO argument suffers from similar problems:
o it may specify an address literal, in which case it can trivially be
  made to correspond to a suitable record which lists that literal IP
  address.  The message to which this is in response describes the
  draft's specified mechanism as an "anti-forgery system".  The simple
  fact that an address literal may be specified in EHLO/HELO (RFC 2821)
  belies that claim; anybody can trivially specify any address literal,
  and the address need not be under the control of the true message
o widely-deployed MUAs use a HELO/EHLO argument taken from one of
  several sources, with corresponding problems for the assumptions
  behind the described mechanism:
  1. the local host name, which is often not fully-qualified and/or is
     misconfigured.  In the former case (unqualified), the MUA may be
     unable to determine an appropriate domain qualifier, and/or the
     domain may be misconfigured.
  2. the domain from a message header originator address field (From,
     Sender, or Reply-To) may be used.  Each one carries different
     semantics, may legitimately be different from one another in a
     particular message, and may vary among messages composed by the
     same author(s) and which are entered into the transport system by
     the same entity.  The semantics of the From field are those of the
     message author(s), which may be different from the entity
     initiating transport (the sender).  The semantics of the Reply-To
     field are of a suggestion for responses and the syntax provides for
     an address-list rather than a single mailbox.  The domain of any
     mailbox within the specified address-list may legitimately be
     unrelated to those of the author(s) or the sender (e.g. for
     purposes of delegation of followup issues).  The Sender field, if
     present, may initially correspond to the person initiating
     transport, but in practice the field is sometimes altered (and
     sometimes not) during mailing list explosion and similar
     operations -- the lack of uniformity of handling makes a HELO/EHLO
     argument derived from the Sender field of dubious value at best.
  3. some fixed string which cannot easily be changed by the user; it may
     bear no resemblance to a domain, and it if does happen to do so, it
     might not correspond to any *relevant* domain name, if such a thing
  4. some string which can be configured by a user.  A typical user will
     not even look at such configuration; many who do will have no clue
     what to use.
  Many of these issues are not under control of the users, administrators,
  software developers, and/or network operators.  The reality is that
  determining an appropriate domain name or address literal for use in an
  SMTP session is not as simple as a cursory read of the SMTP
  specification might lead a naive reader to presume (it is also true
  that commercial operation of the Internet with widespread deployment
  of mobile connectivity is a different situation from the research
  network connected via fixed hard-wired nodes that led to the
  development of the SMTP specification).
o Mobile users in particular may send mail from many IP addresses, from
  facilities including convention kiosks, transportation hub (airport,
  railway, etc.) courtesy lounges, cafes, restaurants, coffee shops,
  public libraries, public access points, hotel rooms, facilities
  provided by an unaffiliated or only loosely-affiliated entity (e.g. a
  salesman visiting a client), and so on.  Several points:
  1. It would be impractical, even where possible, to reconfigure an MUA
     or host/domain name for the temporary and often spontaneous use
     typical in such cases.  Even if a suitable host/domain name were
     known to the user, configuration mechanisms understood, etc.
  2. IP addresses may be assigned by DHCP or similar mechanisms, and may
     be unavailable to the MUA or user (e.g. hidden behind NAT, where the
     DHCP-assigned IP address may be unroutable).
  3. It is easy for scheme proponents to wave their hands and say things
     like "use your ISP's MTA as a relay", "set up a VPN", etc., but such
     comments simply ignore the realities of port 25 blocking and similar
     issues as well as further conflating the *sending* of email with an
     ISP who provides MX records and other resources for the *receipt* of

SMTP servers (SPF clients) that choose to listen to domain 

   4. Any "domain owners" whose users (ISPs' customers, organizations'
      staff, etc.) might travel, ever, cannot meaningfully specify IP
      addresses associated with sending of mail, as that might happen
      from unpredictable IP addresses as noted above w.r.t. mobile users.
      Likewise for organizations which provide courtesy Internet access
      for users, customers, etc. (hotels, transportation providers,
      convention sites, public access points, restaurants & cafes, etc.).
      Note for example that an organization with 50,000 personnel,
      49,999 of whom stay put cannot publish a fixed set of IP addresses,
      as that would screw things up for the one person who travels.
      Likewise for an ISP who wishes to do business with customers who
   5. Some domain names exist for the sole purpose of specifically
      having a registered domain name which can be placed in SMTP
      HELO/EHLO (because, as noted above, an address literal which
      corresponds to a routable IP address and/or a suitable domain name
      corresponding to a routable IP address cannot always be determined,
      and SMTP compliance requires either a literal or a fully-qualified
      domain name.  Such domain names may have no associated MX records,
      and any IP addresses associated with those domain names in the DNS
      may be artifacts of domain name registration procedures (and/or
      registrar marketing methods).  Indeed, the hosts at those IP
      addresses may be incapable of sending mail, and might not be under
      the control of the registered domain name holder.

can use this information to accept or reject email or use as 
input into a spam scoring system.

At a minimum, the draft should also clearly document and emphasize those
problems associated with the described mechanism of associating IP
addresses with an EHLO/HELO argument.  As with the MAIL FROM issues, the
draft currently only begrudgingly acknowledges a small part of the
problem, and undermines even that with BCP 14 language that runs counter
to common sense in light of the problems with the mechanisms. 

As noted in some detail above, the described mechanism is incompatible
with SMTP as standardized, widely deployed, and used for mission-critical
purposes by an extremely large and diverse community of users on the
Internet which has evolved from the research network on which SMTP was
first developed and implemented.  [As an aside, it may very well be the
case that no similar mechanism may be suitable for use with SMTP, as SMTP
as deployed lacks some of the semantics incorrectly attributed to parts
of the protocol by this and other mechanisms, and has as a fundamental
premise that it explicitly provides for communication with no prior

It is widely known that SPF is much more widely used by spammers to lend
the false air of legitimacy which is claimed for SPF to their missives
than it is used by legitimate email users.  SPF as described and
specified is in fact favorable to spammers for that reason, while
simultaneously causing trouble for legitimate email users as detailed
above.  Any mechanism which shares those characteristics (favorable to
spammers, harmful to legitimate users) is doubly harmful to the email

As such, there is a lot of stuff in 
this I-D that could use a lot of review from SMTP experts.

You say you are reluctant to debate the shortcomings of SPF (many of
which are shared by similar schemes), yet you ask for review.  About the
only way that makes sense is to use the review to document the flaws in

Which brings this discourse full circle -- a scheme with so many defects
is probably not something the IETF should encourage for experimental use.
It may, however, be suitable to document the failed experiment as an
Informational document pointing out the issues.  As such, a custom IESG
Note stating that the specification is not a candidate for any level of
Internet Standard, pointing out that the mechanism described interacts
badly with established Internet Standards (DNS and SMTP), and disclaiming
fitness or noting unsuitability of the described mechanism may be
appropriate.  The only factor making Historic status unsuitable is the
fact that RFCs get to Historic status from the Standards Track (RFC 2026),
whereas the mechanism described in the draft has not been suitable for
Standards Track status...

My intention is that after a two week review, I will make a "final"
revised draft and ask the IESG to consider it for a Proposed Standard

That makes no sense at all.  The I-D tracker says Experimental, and you
have characterized the intent as documentation.  Proposed Standard would
be appropriate for Standards Track entry for a specification that has no
known technical omissions, is believed to be well-understood, has resolved
known design choices, is considered valuable, and is suitable for
deployment with a goal of development and eventual full Standard status.
I believe that SPF as described above and in the draft fails on all of
those points.

  A strong conviction that something must be done is the parent of many
  bad measures. -- Daniel Webster