spf-discuss
[Top] [All Lists]

RE: Domain Keys vs Sender Permitted From vs SenderPolicy Framework

2004-06-03 09:51:57
From: Mark Shewmaker
Sent: Thursday, June 03, 2004 5:28 AM


Thanks for your thoughts.  The original post that william(at)elan.net
[william(_at_)elan(_dot_)net] was responding to was
<MHEGIFHMACFNNIMMBACAGENDHPAA(_dot_)sethg(_at_)GoodmanAssociates(_dot_)com> 
from Wed, 2 Jun
2004 11:46:53 -0500.  The arguments are a little better detailed there.


On Wed, 2004-06-02 at 22:50, Seth Goodman wrote:

You are correct that RFC2821 defines MAIL FROM: as the address
to receive
DSN's.  While there is currently no requirement for it to match
any other
address in the headers, what I was asking was why _couldn't_ we
require it
to match the From: or Sender: header?  In legitimate email, it
very often
does match one of these headers.  I was trying to think of a
legitimate case
where it would need to be a different address, and I can't
think of any.  If
we could require this, an SPF pass on the first hop would mean
a lot more
than it currently does.

The new spf using RFROM and the PRD concept makes the problem of the
MAIL FROM return-path not matching the From or Sender headers
irrelevant, because if the return-path doesn't match the PRD, the sender
will use an RFROM that does match, or risk the message being
new-spf-rejected.

While requiring RFROM to match PRA is a very good thing, the user never sees
any of these fields.  It is perfectly possible for you to construct an email
that passes SPF, PRA matches RFROM, and yet is a complete forgery because
From: can be totally unrelated (at least that is my limited understanding of
how PRA extraction works).  Since the user actually sees From: but none of
the other stuff we check, this is a bit of a problem.  Some people will just
counter, "well, that is 2822 stuff so that is out of scope for SPF".  That
is currently accurate, but what I am asking people to consider is the
question of whether or not we can easily solve this problem without
unnecessarily encumbering SPF.  I suspect this is possible.

One large step to get there would be to require the MAIL FROM: address to
appear in either From: or Sender.  Using your postal mail analogy further
down in your post, it really isn't legitimate for the sender address on the
envelope and the sender address on the enclosed letter to disagree.  Current
practice has allowed that, and this has been used more for forgeries than
for legitimate purposes.  That makes me question the wisdom of this feature
of the protocol.

I'm asking people to take a long, quiet look at MAIL FROM: with regard to
From: and Sender: and think about why they _need_ to be different.
Remember, Reply-To: can always be a different address.  This _is_ something
the user sees when they hit the reply button in their MUA.  MAIL FROM:, on
the other hand, is really two things:  it is the address for DSN's and it is
the original responsible sender.  The merged SPF-CID proposal recognizes
this by not requiring RFROM on the first hop:  if there is no RFROM, do the
SPF check on MAIL FROM: instead.  If MAIL FROM: is indeed the original
responsible sender, it really makes sense for it to appear in From: or
Sender:, which the user sees.  In fact, if this were a requirement, there
would be a lot of positive benefits and few, if any, negative ones.



But there's no reason such a mismatch has to be considered bad, and it
can be useful..

For physical mail, the inside of a letter contains an inside return
address--the recipient can use this address for further correspondence
with the sender.

Email also has a separate Reply-To: header for this purpose.  I wasn't
advocating putting any restrictions on that.



The outside of a letter has a postal return address--the post office
will use this address for postal returns.

The addresses don't have to match, even though they're (IMHO mistakenly
and implicitly) assumed to match in most any "how to write a business
letter" type of document you might run across.

I don't think that is a mistake at all and it is certainly implicit in
normal postal mail practice.  Most people would be somewhat upset if they
received a postal letter from their local phone company, probably a bill or
a notification of some change in phone services, and found inside a
fundraising appeal from a political party.  This would not be accepted and
people don't do it.  We can certainly think like mathematicians and
construct some oddball corner case where someone finds this requirement
burdensome.  In fact, you can't make everyone happy, no matter what you do.
The present system allows the MAIL FROM: address, which the merged SPF-CID
recognize as the original responsible sender, to appear _nowhere_ inside the
envelope in a form visible to the user.  The damage being done by forgeries
using this mechanism far outweighs, in my mind at least, the slight
inconvenience for a couple of corner cases.



For a large bulk mailer, it makes sense to have different return
addresses for these sorts of things--postal returns can go to a place
that can handle volumes of return-to-sender mail and fix up addressing
problems, while correspondence returns can go elsewhere.

As you point out later, DSN's to an account can be handled differently at
the originating end.  Particularly for a large bulk mailer, they have the
resources to create such a setup.


(And in the case of billing letters, you're likely have even more
addresses--possibly you'll have internally noted "remit payments to" and
"send correspondence to" addresses, but at the very least you'll have
one internal address which may or may not match the external return
address.)

The "remit payments to" address is the analogy of a web link to make an
electronic payment in an email.  Both postal mail and email users fully
understand and accept that.  The separate address for correspondence
concerning bills can be different from the letterhead return address, and
this is the exact analogy of the Reply-To: address in email.  In addition,
an email may contain a mailto: link for correspondence.  With postal mail,
neither of these are typical methods to perpetrate a scam or get you to open
junk mail you would normally discard, so recipients are not concerned when
there is are explicitly different letterhead, "remit to" and correspondence
addresses.


(Oh, and supposing the outside postal return-address differs from the
inside return-address printed in the letterhead, the letterhead may be
pointing to a generic office or group, while the letter is written
so-called "from" a certain person, but typed up and sent (ie, "Sender")
by a secretary.  So that's two different return addresses, and a Sender,
and a From, and none of them match any other.)

I'm with you on this except for the outside return address differing from
the letterhead.  For that to be the case, they had to go out and make
printed letterhead stationary but didn't bother to print up matching
envelopes.  I think we should be careful to differentiate what is remotely
possible from what is normal, reasonable and expected.  There is a very
large gain to be had if we _could_ require that the MAIL FROM: address
appear in From: or Sender:, where the user will see it.  The important
question to ask is in what real scenarios are individuals or groups harmed
by having such a requirement?  I still can't think of anything of
consequence.


For email, the same thing applies.  I may want automated mail system
returns to go to a different address than replies, and if I'm using any
verp-like system, I *can* have a completely different system process
bounces than the system that processes correspondence.

That is the real point.  The new SPF suggests that people do SES to avoid
bogus bounce spam, which is one of the end results of a joe-job.  There is
nothing stopping you, as a mass mailer, from directing valid bounce messages
resulting from one specific account to any other account in your system.


(As a side note, I never thought the fact that preventing this sort of
mismatched set of return addresses would really in fact be a potential
problem until just a few days ago when reading through a debian mailing
list thread.   In that thread, someone objected to SPF on the grounds
that he wanted to send mail from his ISP and with his ISP-provided email
address in the From: body header line, but with bounces and reply-to:'s
going to his personal server elsewhere.  While at the time I wasn't sure
why he'd want to do this, I couldn't convince myself that there was any
real reason for the new SPF to *have to* prevent this, given that RFOM
and the PRD can show this person's account as the responsible sender,
and then you can simply trust him or not as you please.)

The reason you'd want to prevent this is to avoid forgery.  We could wait
for DomainKeys, or something else far more expensive than SPF, to solve this
problem.  However, if we think about it a bit, we may have the means to do
it right now without adding very much to what we already do.



But to a reason you might want this sort of difference with email:

Let's say you're sending out some sort of notice by email to all your
one million customers, perhaps a monthly invoice.  (You're a huge
business.)  Some of those customers will respond to the email itself,
but other responses you get will be mail system bounces.  At that scale,
it is probably useful to have a separate system handle VERP bounces and
cause postal notices to be sent after permanent failure errors for those
specific messages, (though it's overkill for us mere mortals.)

Exactly, and it's chump change for those large outfits to implement
something like that.  By using the Reply-To: field appropriately, you can
already direct user responses to a different address.  It's only the
bounces, and large organizations have the resources to deal with that.



Of course with email, it's still possible to process bounces and
responses on the same machine, but I can imagine times where it's nice
to be able to separate it.

Here's the crux of the matter.  In so many places in the RFC's, the authors
have considered the potential for really oddball cases and refrained from
putting any restrictions in place so as to not irritate that tiny group that
may exist in theory only.  By so doing, they have ignored the interest of
the vast majority of users who are then harmed by malicious individuals
taking advantage of the built-in "looseness" of the protocol.  I think this
may very well be one of those cases.  Again, let's try to think of some
cases where someone really _needs_ to have the MAIL FROM: _not_ appear in
From: or Sender:.  We should then balance that against the fact that the
average email user, even if SPF is widely deployed, will have no way of
knowing whether From: is a complete forgery.


So, I don't see anything really fundamentally wrong with an invoice from
your ISP with a "MAIL FROM:
<SES-string-invoice-bounces(_at_)bounces(_dot_)accounting(_dot_)example(_dot_)com>"
 in the
envelope, but with "From: invoice(_at_)accounting(_dot_)example(_dot_)com" in 
the body
header, along with a "Reply-To:
invoice-questions(_at_)accounting(_dot_)example(_dot_)com", and "Sender:
A-L-sender(_at_)accounting(_dot_)example(_dot_)com".

Again, four ~"From"s, not matching, and nothing's really wrong with
this.  (I admit this is a more contrived example--it's harder to make an
email scenario with *all four* being different, especially the From and
Sender being different in this example.)

On a highly theoretical, philosophical level, I do agree with you.  However,
on a practical level, having From: or Sender: include the original
responsible sender (MAIL FROM:) is of such a large benefit to so many people
that we really need to consider who would actually be inconvenienced and to
what extent if we actually made this a requirement.


Such a combination was somewhat discomforting with the old spf, since it
readily exposed the fact that the old SPF verified a return-path that
isn't seen or really understood by most users, but the same combination
seems just fine to me with the new spf.

Not really.  The PRA extraction, as I understand it, does not necessarily
use From: or Sender:, so even with the merged SPF-CID, those fields are
subject to forgery.  I think the PRA extraction is a traversal of the
Received: headers, but maybe someone with more specific knowledge can
correct me.  I guess the key question that the CID gurus out there could
hopefully address is, "Does PRA extraction on the first hop always yield an
address that is in From: or Sender: ?".


Off-topic side note:
---------------------

I just realized/remembered something--not sure which!

If there's ever an spf modifier for SES, (say for an "exists"-type,
non-CBV SES check), then if you have a MAIL FROM line as follows:

MAIL FROM:<SESed-address-person(_at_)ONE(_dot_)example(_dot_)com> \
     RFROM=<person(_at_)TWO(_dot_)example(_dot_)com>

And you do new-spf checks and any SES-described-in-spf-modifiers test,
you'll do the spf check after querying the "TWO.example.com" domain, but
you'll do the SES check after querying the "ONE.example.com" domain!

The new-spf focus on RFROM/DAVE doesn't change anything for SES tests
for what domain would have to be checked to get SES modifier info from.

My understanding of the merged proposal is that the first hop would not use
RFROM, thus forcing the SPF check to be done on MAIL FROM:.  When there is
an RFROM parameter, we use that and ignore MAIL FROM:.  That is a good
reason for making it a requirement that the originating hop MUST NOT use
RFROM, thus forcing MAIL FROM: to be validated by SPF at least once during
message transit.

That is actually part of my motivation for considering a requirement that
the MAIL FROM: address appear in either From: or Sender:.  If MAIL FROM: is
validated by SPF at the first hop, and we then require this address to
appear in From: or Sender:, we present to the user an address that has been
validated.  This is extremely valuable.  So what if it was outside of the
original scope SPF?



That's not really related to the current thread, but...I just suddenly
re(?)-realized it.

Mailing lists are handled differently as explained in the RFC.  They
preserve the original From: address, put their own address as
Sender: and
set MAIL FROM: to be their bounce address.  Forwarders, on the
other hand,
are not supposed to change MAIL FROM:, From: or Sender:.  They typically
just change To:, prepend another set of Received: headers and resend the
message.

And they should add Resent-* headers.  :-)

Good point.

--

Seth Goodman


<Prev in Thread] Current Thread [Next in Thread>