spf-discuss
[Top] [All Lists]

RE: Bootstrapping trust model (was Re: a "never relays" parameter)

2004-06-14 09:05:59
From: Greg Connor
Sent: Sunday, June 13, 2004 11:15 PM


--Seth Goodman <sethg(_at_)GoodmanAssociates(_dot_)com> wrote:
Actually I think I see where Seth is going here.  The ses:
implies that we
are checking the MAIL FROM: of the original hop, not the rewritten MAIL
FROM.

Yes, that's exactly what I'm saying.


Cool, it sounds like we are thinking along the same lines.


I think this means checking the SES at every step, even if MAIL FROM is
rewritten to another forwarding domain.  (or, stating that in
terms of the
Layers post I just wrote, if you are able to validate Layer 2,
that trumps
Layer 1, and the mail is considered Verified Direct.)

Exactly!  SPF is a hop-by-hop validation (layer 1).  If you can validate
the original source of a forwarded (end-to-end validation, layer 2), the
hop-by-hop validation (layer 1) is redundant.  If forwarders do
SPF checks
on incoming, we can trust that anything they hand us has been
validated at
the source, so long as we validate their identity with SPF.  In other
words, if a forwarder is on a local list of trusted forwarders (they
perform SPF test on incoming), validating the forwarder's
identity (layer
1) indirectly accomplishes originator validation (layer 2).  This is a
very useful shortcut.


Right.  That's what I was thinking when I posted the Layer concept but I
hadn't got around to writing the follow-up, which says basically that.

This kind of "bootstrapping" is important.  If you are getting mail
forwarded to you, you are going to miss pretty much all the benefits of a
straight IP-based check, because the thing you are checking
(either PRA or
SRS) will always be your forwarder.

Precisely.  The holy grail for 2821 checks is validating the originator
return-path.  In the case of a forward, SPF can, at best, do this indirectly
only if the forwarder is explicitly trusted.


This is even more reason why the forwarder should appear on a personal
whitelist.  Once you check the box that says "Trust this sender as a
forwarder" then you can believe whatever previous thing they validated.
They are working on your behalf, after all.

This is important enough to the protocol that it should probably appear both
in the spec and the reference implementations.


One step further... let's say the message has clearly been forwarded, but
the forwarder doesn't check SPF.  You could probably apply
SPF-Like checks
to the data that came before it, by pulling data out of the now-trusted
Received: lines.

That's certainly possible.  But let's first ask if we can accomplish the
same thing before DATA without creating a whole mess of extra "sneak peak at
the headers" ESMTP parameters.  It turns out that we can validate a _signed_
originator return-path before DATA, and with far more confidence than an SPF
check based on a possibly forged set of headers.  We do the validation
either through the much-maligned CBV or a lightweight DNS implementation of
the same using the exists modifier with appropriate macros in the SPF
record.  This seems so superior to groveling through the (non-standard)
headers _after_ DATA, which are possibly forged, and then doing an SPF
check.  I think it this makes a good argument for _requiring_ signed
originator return-paths specifically to permit their validation _before_
DATA without waiting for new ESMTP parameters.  I'm not opposed to all
DATA-phase checks, but anything we can check before DATA, especially without
creating new ESMTP parameters ought to be a no-brainer.  In addition to this
before-DATA validation being lighter weight, it is far more immune to
forgery.  Let's do it!



However, a forger can pose as a forwarder very easily.  If a
forwarder is
_not_ on the local list of trusted forwarders, or if there is _no_ local
list of trusted forwarders, we can't tell if he is:

i)   a legitimate forwarder who does SPF on incoming,
ii)  a legitimate forwarder who does not do SPF on incoming or
iii) a forger.

For cases ii) and iii), we need to validate the originator address if we
want to avoid forgery.  However, unless the forwarder is on the
local list
of trusted forwarders, we can't distinguish i) from ii) and iii), so we
have to originator checks whenever the forwarder is not on the
local list
of trusted forwarders, _if_ we really want to avoid forgeries.


I agree, and I think that people will probably be aware of their
forwarding
addresses enough to list them, so usually it won't be a problem.

One of my axioms is: You are either an agent for the sender, or an agent
for the receiver.  If you aren't either, you have no business
handling the
mail.

Agreed.  I think the concept of the trusted forwarder is an important
concept in SPF.  Without it, you have to validate every originating
return-path in a forward, since none of the forwarders are explicitly
trusted.


If a sender uses submitter, not a big deal, as long as it matches PRA and
matches their IP.  But since they are not MY forwarder, I won't
take their
word on who it's *really* from.  I will probably accept the message, but
will mark it as "really from the PRA/submitter" and I would mark
the From:
and Sender: as "Unverified"

And as you say, if the From: or Sender: checks out with the IP,
never mind
the Submitter, though it's likely to be the same.  If the sender
uses some
kind of forwarding arrangement, they should list that in their
SPF record.
"Sender-side forwarding" is just a subset of "sending" to me :)

As a thought experiment, let's put aside PRA for a moment.  Just as the holy
grail in 2821 validation is the originator return-path, the holy grail in
2822 validation is From: and Sender:.  The Received: headers are of some
interest, but the user never sees them, so From: and Sender: have uniquely
important functions in email forgery.  I posit the following:

"In a correctly constructed email, the originating MAIL FROM: can _always_
be identical to either Sender:, if it exists, or From:, if Sender: does not
exist."

If this _were_ the case, validating the originator return-path, either
directly or through a trusted forwarder, would give us an incredibly
lightweight method of validating either Sender: or From:.  The validation of
the originator return-path is done before DATA.  Once DATA starts, the
_only_ thing we have to check is that either Sender: of From: matches the
validated originator return-path.  No groveling through the headers, no
depending on a PRA algorithm that works "95% of the time" (what about the
other 5%?), no tangling with anyone else's intellectual property, no
DomainKeys or other strong crypto validation at he recipient, no XML records
to fetch and parse.  We can validate one of the two the key 2822 headers for
free after a simple before-DATA test.  It doesn't get any better than that.

There are a few corner cases where MAIL FROM: is presently different from
Sender: and From:, but I will argue that there is no reason for this to
continue other than "we've always done it that way".  Here are a number of
cases that illustrate why I believe it is both possible and desirable to
_require_ the originator's MAIL FROM: to be identical to either Sender: or
From:.

1) Single sender - the senders address appears in From: and there is no
Sender:.  An MTA that is well-configured can require that the From: address
either be the users local account, or if it is a foreign domain, the local
MTA MUST be designated in the foreign domain SPF record.  In either case, it
is perfectly legitimate for the MTA to use the From: address in MAIL FROM:
as it will pass SPF at the next hop.  The headers will show the actual
message path so there will be no obfuscation.  In this case, validating the
originating return-path validates From:, which is the big brass ring.  The
MUA can show From: as validated.  This case covers the majority of non-bulk
email.

2) Multiple senders - this is rarely used, but it is in the RFC's.  If there
are more than one From: addresses, there must be a Sender: header indicating
who actually sent the message.  The Sender: address takes precedence.  In
this case, I suggest that a well-configured MTA can require that the Sender:
address either be the users local account, or if it is a foreign domain, the
local MTA MUST be designated in the foreign domain SPF record.  In either
case, it is perfectly legitimate for the MTA to use the Sender: address in
MAIL FROM: as it will pass SPF at the next hop.  The headers will show the
actual message path so there will be no obfuscation.  In this case,
validating the originating return-path validates Sender: but not From:.  It
is not possible to validate From: with this methodology since it contains no
signature.  That's fine, the MUA can show Sender: as validated and From: as
not validated.

3) One user sends mail on behalf of another user - this is even more rare,
but is also in the RFC's.  In this case, there is one From: address and one
Sender: address.  The Sender: address takes precedence and the behavior
should be as in 2) above.

4) Mailing lists - a mailing list preserves incoming From: header (hopefully
after doing an SPF check), places its own address in the Sender: header and
places their own address in MAIL FROM:.  Unfortunately, the address the
mailing list puts in Sender: is often not the same as the address they put
in MAIL FROM:.  The reason for this is to direct bounces to a special
address.  As a bulk mailer, they could easily redirect bounces addressed to
any particular account to any other particular account.  This relatively
small change on their part would make it possible to validate the From: or
Sender: header using 2821 information, which is a huge benefit to the larger
community.

5) Other cranky people who insist on setting a different address for
bounces - while some people have done this for years, as the RFC's did not
strictly forbid it, this is a very weak argument for allowing the sender
address visible to the user to be different from the address used for
bounces.  In the end, a much larger benefit is achieved by enforcing this
limitation, even if a few people complain without giving any good reasons
aside from minor inconvenience.  I maintain that they can either direct
bounces to their normal address and automatically separate them with
existing content filters, or set up their MTA's to separate their DSN's upon
receipt and send them wherever they wish.

While having a different address for bounces than normal return addresses
has been a mild luxury under the current email system, it is _not_ a
necessity for anyone.  Furthermore, requiring the two addresses to be
identical allows simple validation of the key 2822 headers at virtually no
cost.  If there are any significant cases that this breaks, not as in "I
might have to change something", but rather "I must have this functionality
for the following reason and I can't use any of the workarounds", please
speak up.




Mailing lists are an interesting problem, since they are usually
agents for
the sender AND agents for the receiver.  Mostly the mail you get from the
mailing list will just be identified as "From X List"... but if you take
the time to add the list robot to your trusted forwarders list, you could
take the chain of trust back one level there as well (either by groveling
*their* received line, or taking whatever result they already
validated in
Received-SPF).

See above.




but SES will
always match successfully, since it is not based on IP of the
current MTA.
Usually you would not want to reject based on the SPF of the
From: but in
this case the domain owner is trying to tell you "Go ahead and
reject based
on this"

The only problem is that unless we _always_ query the SPF record for the
originating domain, we have no way of knowing that the domain owner said
that.  We could require that, but it is an extra DNS query at every hop
(not really that bad).  If we did that query _before_ the query for the
current sender, and the result was pass, we could skip the query for the
current sender.  Another alternative is to say that a forwarder on the
local list of trusted forwarders has already validated the originator
address, so we don't have to.  The only fly in the ointment here is that
we have to be sure that the forwarder also uses this same methodology
when they encounter a forward on incoming.  Unless our forwarder also
checks originating addresses on incoming forwards where the incoming
forwarder is not whitelisted, the shortcut scheme falls apart.  We could
easily test our forwarders with some contrived forgeries, including
forged forwards, to make sure that they reject them before whitelisting
them.

Considering how easy it is to get the above forwarder audit wrong, it
might be sensible to _require_ first trying to do originator validation.
What do people think?


I would probably want to do one of:
 * Always check both layer 1 and layer 2 identity
 * Check the forwarder (trust and IP) and then trust what they say

I probably would not stop with only the layer 2 From:/Sender:
check, but I
haven't had time to think about that much, perhaps I will change my mind.

Yes, this is debatable.  Here's one argument.  If we are really interested
in validating the originator return-path, which is what we originally set
out to do, then having done that, why go further?  OTOH, checking the
current sender first adds very little overhead, so it may be a non-issue.
As long as the originator return-path is validated, either directly or
through a trusted forwarder, I don't care about the rest.


If there are more than 1 forwarders involved, all of them should
be on your
trusted list (for example, alumni and pobox, or mailing list and
forwarder).  Something acting like a forwarder (ie. PRA is
Resent-From and
!= From: or Sender:) but isn't on your trusted list can be flagged as a
suspicious thing but I probably wouldn't reject it - it could just be
because the From: is sf.net and PRA is sourceforge.net.  Just mark it as
"really from" the entity giving the Pass result, by your check or by a
trusted third party.

In other words, there are probably some cases where there are more than 1
forwarder, but if they are all trusted and all verified, no problem...
bootstrapping is still possible.

That the key.  It's a lot easier to validate one originator return-path,
which is all we really care about anyway, than to grovel through the headers
and proceed to do SPF validations on every hop.  Since we weren't there to
see that the IP addresses were what the headers claim they were, this method
is of dubious value and a clever forgery will pass.

--

Seth Goodman


<Prev in Thread] Current Thread [Next in Thread>