-----Original Message-----
From: SPF_0x1b [mailto:SPF(_at_)0x1b(_dot_)com]
Sent: Tuesday, May 25, 2004 2:19 PM
To: spf-discuss(_at_)v2(_dot_)listbox(_dot_)com
Subject: RE: [spf-discuss] The New SPF: overall outline - CAUTION GNU
REQU IRED
Syntactic extensibility is the typical Microsoft path to
Embrace Extend and
Extinguish[1]. This must be done under the GNU license as the
only form of
defense against Microsoft's well documented pattern of
behavior. The issues
of software 'patents' lie heavily over these efforts. Prior
art please?
assume non-disclosure.
[1] http://en.wikipedia.org/wiki/Embrace,_extend_and_extinguish
Other than that, thank you for coming to a common solution.
It is very much
appreciated and XML isn't all that bad ;), neither is
Microsoft, but you
can't ignore a track record that well worn.
Thank you Everyone!
-----Original Message-----
From: Meng Weng Wong [mailto:mengwong(_at_)dumbo(_dot_)pobox(_dot_)com]
Sent: Thursday, May 20, 2004 12:05 PM
To: spf-discuss(_at_)v2(_dot_)listbox(_dot_)com
Subject: [spf-discuss] The New SPF: overall outline
This is a rough outline of the convergence between SPF and Caller-ID
that Wayne, Greg, Mark, and Meng (from SPF), and Harry, Bob, and Jim
(from Microsoft) worked out during this week of MAAWG and MARID
meetings.
It is still in flux so don't take it as a fully-baked
proposal; but it
does lay out some new directions that people in the MARID interim
meeting seem reasonably happy with in principle. More than
anything I
think they're relieved that we finally have some kind of convergence
so the uncertainty goes away. It's kind of like how the
stock market
moves in November: uncertainty is always a depressing factor.
Doing both 2822 and 2821 is a big step forward too, and
makes a lot of
the ultimate "buyers" of this proposal happy.
When you read this, pretend you've forgetten everything you
know about
SPF and Caller-ID, and apply beginner's mind. People who
are totally
new to all this might actually be able to understand it more easily
than people who already have some attachment to SPF.
The New SPF
Thu May 20 14:51:44 EDT 2004
CURRENT ISSUES
* Forwarding is perceived as the biggest problem with SPF.
SRS has been proposed as the workaround.
* SRS sucks. Nobody's happy with it, and it's a pain in the butt.
If we could find a better workaround that would be great.
* The Microsoft guys (and lots of other guys) really want to do 2822
verification because that's what the user sees and that's a good
contribution to fighting phishing. Nobody opposes this, except
maybe Yahoo who think we can just go straight to crypto signing
without stopping at the Designated Sender step along the way.
* The SPF guys really want to do 2821 verification because you don't
spend the bandwidth of DATA. They also want to be able
to get fewer
joe-job bounces, and if everybody out there implements
SPF checking
then that will come true.
* Seth Goodman and others in the SES camp have pointed out that
instead of requiring everyone in the world to implement SPF
checking, you can stop joe-jobs more unilaterally simply by
implementing SES, Signed Envelope Sender. Dave Crocker has
independently invented this also.
* Critics of Signed Envelope Sender have said that it's
unrealistic to
expect big ISPs and enterprises to actually do the
signing thing for
all outgoing mail.
* On the other hand, Yahoo seems to think that putting a
signature in
the headers of all outgoing mail isn't so bad.
* Also, mailing lists do VERP today, and SRS and SES are
pretty close.
* Finally, Microsoft has plans to do something very similar to SES,
except it's a signed header that they expect to get back in bounce
messages. So it's the same idea basically just in a
different place
with different operational tradeoffs.
* So the large-scale deployability of SES is an open question we
probably shouldn't try to address right now.
THE BIG NEW IDEA: let's define a new ESMTP field, "RFROM".
* To recap the Caller-ID concept, you pull out a Purported
Responsible
Domain (PRD) from the headers using some kind of selection
algorithm, and perform Designated Sender checks against that. You
may have to show users the whole trust path if there's a big
forwarding relationship.
* We expect senders to put that PRD address into the RFROM
extension.
Suppose original(_at_)sender(_dot_)com sends mail to
alias(_at_)pobox(_dot_)com which
forwards to <final(_at_)recipient(_dot_)com>. When the final recipient
gets
the message, they see
MAIL FROM:<original(_at_)sender(_dot_)com>
RFROM:<alias(_at_)pobox(_dot_)com>
Resent-Sender: <alias(_at_)pobox(_dot_)com>
From: <original(_at_)sender(_dot_)com>
To: <final(_at_)recipient(_dot_)com>
Observe that the Resent-Sender, which would be the output
of the PRD
algorithm, matches the RFROM.
So now the SPF check runs against ... what, the RFROM?
Yes. In the Old SPF, checks ran against the MAIL FROM.
This caused
problems for forwarders.
In the New SPF, if a MAIL RFROM is present, checks now run against
it.
But what if a MAIL RFROM is not present?
If a MAIL RFROM is not present, checks run against the
PRD from the
headers ...
... Until the Flag Day, when you can go back to checking MAIL FROM
if the RFROM is not present.
More about Flag Days at the end.
Wait, do senders have to run the Caller-ID algorithm to extract a
PRD?
Probably not. For Caller-ID, senders are prepending stuff to the
headers. They just need to take what they prepended and dump it
into RFROM. They don't have to *read* the headers, just know what
they're prepending.
So there are a few corollaries:
* If the RFROM turns out to be different from the PRD in
2822, reject
the message with confidence. So even after a receiver does an SPF
test against the RFROM, they still need to run the Caller-ID
algorithm against the 2822, just to confirm that the PRD from the
2822 actually is the RFROM from the 2821.
* If the MARID lookup on the RFROM address is anything but
a PASS, you
can safely reject it. The justification is: if you're making up
RFROM, you know what you're doing. This prevents spammers from
forging innocent third party domains into RFROM, which is
a huge win
over the old SPF, where spammers could forge innocent domains who
don't publish SPF into MAIL FROM.
Earlier you talked about what forwarders should do. Do you have
prescriptions for all the different kinds of senders?
Yes.
SPF asks certain constituencies to change (eg forwarders
have to do
SRS).
Caller-ID asks certain constituencies to change (often to
prepend a
new header.)
The short answer is, the New SPF asks the above
constituencies to do
both, except instead of SRS, it's RFROM.
The long answer is, we'll have full prescriptions by Monday.
SEMANTIC CONVERGENCE
* The SPF model is procedural: start here, try that, if
that matches,
short circuit.
* The Caller-ID model is set-theoretic: define the good, define the
bad, let's locate the client in that space.
* Bob Atkinson decided that the procedural and set-theoretic models
are isomorphic. The SPF ordering is fine and survives translation
into a set model.
* Microsoft approve of all of the SPF mechanisms, including ptr and
exists.
* SPF has to add a caveat telling people that mechanisms should not
expect guaranteed side-effects. In other words, if you
want to use
"exists" for logging purposes, you shouldn't; if folks do
this, they
should know that their mileage may vary.
* After discussion of macros, Bob concluded that macro
expressions are
sufficiently cacheable and are therefore acceptable.
* To make the above statement true, SPF has to give up the %h macro,
which it's willing to do since nobody's using it anyway.
* Semantic convergence has been achieved between SPF and Caller-ID.
SYNTACTIC CONVERGENCE
* To XML Believers, there is no extensible syntax but XML.
Surprisingly, though Meng is not a member of this
religion, when he
wanted to object to this, he suddenly felt like a teenage punk
telling his elders that they don't know what it's like,
man. Doing
XML just feels like listening to the advice of the elders.
* But it doesn't have to be all-or-nothing.
* The SPF syntax already goes a long way. One day we may
find a need
to express concepts that are not expressible in simple
SPF, and that
require XML. For example, perhaps some senders need to express
accreditation assertions in a very particular way. When that day
comes, they want to be able to switch to XML with minimal pain so
they can take advantage of those new features.
* Until that day comes, though, the simple SPF syntax will do just
fine for a huge majority of senders who don't need the extra
features.
* Observe that HTTP/1.0 and HTTP/1.1 are logically different, yet
servers can understand both, and old browsers are still welcome to
use HTTP/1.0.
* Similarly, the New SPF does not obsolete either the Old SPF or
Caller-ID. Imagine a "one interpreter, two parsers" world: we
expect client implementations to be able to parse both the Old SPF
records, and the equivalent-but-more-extensible XML representation
of same.
* Meng expects that the vast majority of folks will publish using
v=spf1 notation.
* Microsoft expects that when certain folks start needing
new features
that are not expressible in v=spf1, they can publish their records
in XML and all the clients out there will be able to read those
records.
* We'll need to be very firm that client implementations be able to
read both forms.
* This is probably the most contentious issue.
* The desire for XML arises from foresight: we want to save
the world
today, and we think we can do that with the existing SPF
TXT syntax;
but we also want to make it easier to save the world in
the future,
with less work than we needed this time round. And that is very
challenging because we don't know today what we'll be saving the
world from tomorrow. So it seems wise to let XML take care of
extensibility, at least at the syntactic level. (We'll need to
define semantic extensibility no matter what we go with, XML or
something else.) Also, Meng had the nagging feeling that if we
didn't allow an upgrade path to XML, one day someone would say to
him "told you so", and he hates that.
* The desire to support the "legacy" SPF syntax arises from the
principle: keep things simple for the simple cases; and to avoid
burning the early adopters. I would be quite happy to
keep telling
people to publish in SPF syntax if they don't need any of the
extensibility points and new features you find only in XML.
* The desire to support both syntaxes arises from "be
liberal in what
you receive".
THE NEED FOR A FLAG DAY
* The promise made to the SPF camp was: you will one day be able to
reject forged MAIL FROM based on SPF lookups. I intend
to keep that
promise.
* Under the Old SPF, until the forwarders were all SRS compliant, we
used a hacky workaround with trusted-forwarder.org.
* Under the New SPF, until the forwarders all use RFROM, we
are in the
same boat.
* Either way we need the industry to announce a flag day: we need to
strike a balance between giving forwarders lots of
notice, and being
able to block stuff before DATA.
* Changing the burden to RFROM instead of SRS makes things a
lot easier
for forwarders. That reduces their adoption threshhold.
Everybody
wins.
* We can't decide today when the flag day will be, but one
day soon we
probably will be able to.
* Before the Flag Day:
- Campaign for MTAs to start supporting RFROM.
- In the absence of RFROM, receivers SHOULD NOT reject
spoofs until
after seeing data, and SHOULD use responsible addr from 2822
header.
- (Receivers who know exactly who's forwarding to them
may choose to
reject anyway. Local policy, etc.)
- Fight Joe Jobs with SES.
* After the Flag Day:
- MAIL FROM RFROM now expected from forwarders.
- MAIL FROM spoofs without RFROM MAY be rejected.
- Flag day requires industry coordination to ensure MAIL FROM /
RFROM support is out there.
- RFROM harder to spoof because only a PASS result is acceptable.
BUT WHAT ABOUT PREVENTING JOE-JOBS?
* Under the Old SPF, we wanted all the SMTP receivers in
the world to
become SPF aware, and to reject mail that failed SPF tests.
* Hosts that do not upgrade will continue to send bad bounces. This
is seen as a moral failing on their part.
* Under the New SPF, senders pragmatically and unilaterally
implement
SES. They get the benefit of joe-job protection without
waiting for
the whole world to change.
* Now, doing SES may be impractical for big ISPs. But SES and
worldwide adoption of SPF can meet somewhere in the middle.
-------
Sender Policy Framework: http://spf.pobox.com/
Archives at http://archives.listbox.com/spf-discuss/current/
Latest draft at http://spf.pobox.com/spf-draft-200405.txt
Wiki:
http://spfwiki.infinitepenguins.net/pmwiki.php/SenderPermittedFrom/
To unsubscribe, change your address, or temporarily
deactivate your subscription,
please go to
http://v2.listbox.com/member/?listname=spf-discuss(_at_)v2(_dot_)listbox(_dot_)com
-------
Sender Policy Framework: http://spf.pobox.com/
Archives at http://archives.listbox.com/spf-discuss/current/
Latest draft at http://spf.pobox.com/spf-draft-200405.txt
Wiki: