spf-discuss
[Top] [All Lists]

Re: SPF/SRS problem domain (was: SRS integration with qmail)

2004-02-19 04:11:35
On Tue, Feb 17, 2004 at 08:27:08PM +0000, Mark wrote:
I just finished implementing SRS, system-wide. Now all outgoing mail, for
all domains I host, has an SRS signed envelope-from. And the Milter takes
cares of weeding out invalid bounce messages (along the lines of what we've
been discussing from the last few days). So, far, everything goes
wonderfully well.

And you get the immediate protection from joe-job bounces and spam sent from
<>, right?

I wonder if it's worth trying to put together an explicit problem statement
for SPF/SRS, that is, what is the minimum set of things wrong with SMTP that
we must fix? (So we can focus on the smallest, simplest solution to them).

From my point of view, and I do receive lots of spam and spam bounces, I
would say:

(1) Prevent the reception of bounce messages except in response to mails
that we actually sent. That includes:
  (1a) Reject bounces from joe-jobs
  (1b) Reject spam with empty envelope sender

On the flip side, valid bounces are a key part of the integrity and
trustworthiness of E-mail and it's very important that we don't lose them.

(2) Validate the MUA 'From:' header, or at least tag possible forgeries as
such. (Not just because of spam, but also because of more sinister uses)

Actually, that's it. I welcome additions though. I would add that any fixes,
if they involve a lot of work or changes to established practices, must be
strong "once-and-for-all" solutions which abusers cannot easily work around.

Perhaps controversially, I will say that IMO the following is a problem
which *does NOT* need to be solved:

(X) Validate the envelope-sender for successfully-delivered messages (i.e.
with a valid recipient). By definition they are not bounced, so do not fall
under (1a); and the Return-Path: header is rarely seen by end users, so they
don't fall under (2). Of course, if someone sends you a mail with a forged
Return-Path it may be a weak indication of spam, in the same way as a
subject line of 'V1AGRA', so might be of interest to those engaging in the
spam arms race. But as filtering goes, this is really poor; a spammer can
legitimately own a domain and send spam using it. So you then get into the
area of 'trusted' domains, how you deal with newly-registered domains,
domains which have been registered by spammers and quietly "laid down" for a
year before use, and so on; or else spam will be sent with an empty envelope
sender (case 1b). If you ask me, IP RBLs are instantly more effective
against spammers. A spammer may have thousands of domains but only one IP
netblock, plus the known pool of open relays on the Internet.

"But what about all those viruses I receive?" I hear you shout. True, I get
lots of those too. But if you imagine an SPF world where envelope-senders
are validated, viruses will be written to send with the users' real E-mail
address as the return path (or someone else at the same ISP). Filtering on
invalid envelope-sender will not prevent viruses being sent, so long-term
will not reduce the number of them you get. So you have to go back to
content-filtering to identify "virus" versus "non-virus" mails.

However, (X) is the problem which SPF aims to solve. As a side-effect, it
intends to solve (1a) - but this will only happen if/when the vast majority
of ISPs implement SPF. Enabling SPF for your home machine, today, will have
only a small benefit to you, and a small benefit to the rest of the
Internet. It doesn't address (1b) at all; I don't believe that validating
the HELO name on bounces will have any anti-spam effect, and will have a
small negative impact on the reliability of receiving bounces. But SPF might
form a framework for solving problem (2), header forgery, which doesn't
really seem to have been talked about much here.

Problem 1: Dealing with bounces
-------------------------------

From my POV, the smallest/simplest way to solve (1a) and (1b), once and for
all, is to include a signed cookie which is returned in the bounce, and
discard silently all bounces which don't include it. The top contenders for
this are:

(1) By using VERP to rewrite the envelope-sender.
    Problems: 1a. some MTAs enforce the 64-character local-part min limit.
              1b. may screw up some filtering by mailing lists etc
              1c. some really broken MTAs don't even bounce to the envelope
                  sender but to the From: address
(2) By including a header with a signed cookie.
    Problem:  Some MTAs do not return the headers when bouncing a message.

Having thought about this, I think that including a header is probably the
cleanest, simplest way forward, especially because we don't have the
64-character limit. It is not a hard requirement that MTAs include headers
in bounces, but it is strongly implied by the RFCs (for example, the DSN RFC
says you can set options to return "headers only" or "full message"; it does
not allow you to set an option to "return nothing!") And if you get a bounce
back which doesn't even have the headers, it's pretty useless for debugging
anyway.

The only thing to beware of is autoresponders (mailing list subscription
confirmation, "I am on vacation until Monday", etc). They may well be sent
with an empty envelope sender; equally, they are usually sent to the From:
or Reply-To: address and therefore VERP would not be applicable. Many of
these applications *do* quote back the headers, particularly mailing list
subscriptions. Those which don't, like vacation autoresponders - well I
could live without them :-) Seriously, if we can settle on a *standard*
header for bounce cookies, such applications will in time be modified to
include that line in the response.

For creating the cookies, we have a choice of:
- a simple secret key / hash solution, as used by SRS
- public-key cryptography, which is more complex but allows other handling
MTAs to validate the cookie should they so wish

If this were the only problem to solve then the secret key solution would be
the simplest; it would require users to send their outgoing mail via the
ISP's outgoing mail server (as is usually the case anyway) to get the
signature header added, and the ISP's incoming-MX server could then validate
it. Both would contain the same secret key.

A public-key solution could be considered if it can kill some other birds
with the same stone.

The great thing is, if you implement this on your system you get full
protection straight away, independent of what anyone else on the Internet
does or does not deploy. I will almost certainly implement it myself. If it
can become a BCP with a standardised header then we can make things like
autoresponders a bit more reliable, but otherwise it will just work.

Problem 2: Header forgeries
---------------------------

This is in many ways the most pernicious of SMTP's problems and cries out
for a real solution. Headers (or the From: header at least) need to be
'certified' somehow, so that a domain owner can say that this E-mail did
originate from one of their users, preferably one individual user. And of
course this certification status needs to be presented to the receiving user
in a useful way.

--- 8< --- Aside: presentation issues ---
Presentation could be:
(1) If the MUA enforces the validation, then it can present it how it likes.
However in the current world MUAs are frequently disconnected from the
Internet, so probably unable to perform validation (unless we have a signing
chain which resolves to some fixed 'root' certificates, probably too complex)
(2) The final receiving MTA at the ISP can add certification headers, which
the MUA can be modified to use; problem is that most existing MUAs will not
show this info unless you explicitly ask for full header display
(3) The final receiving MTA can modify the From: headers, e.g.

From: "Dubya" <president(_at_)whitehouse(_dot_)gov>
becomes
From: "Dubya [president(_at_)whitehouse(_dot_)gov PROBABLY FORGED]" 
<president(_at_)whitehouse(_dot_)gov>

Any existing tag saying that the address was or was not forged would have to
be removed of course.

(4) The final receiving MTA can simply bounce messages with likely forged
headers, by a 550 error after the DATA portion. If the envelope sender was
also forged then we have a joe-job, but we already talked about how to
protect against bounces to forged addresses.
--- 8< ----------------------------------

Anyway, how do we allow a domain owner to certify the From: headers of
messages?

SPF certainly looks like a solution here. The solution says, in the simplest
case, "for a message claims to be From: *(_at_)mydomain, if it came from IP
address X,Y,Z then it is certified, otherwise it is forged". By using macro
expansion you can implement separate policies for individual users at your
domain, so roaming user 'bob' who uses a different ISP at home can have that
ISP's IP addresses added to his list of certified IPs. All good and well so
far, although this is a different application of SPF than validation of
*envelope* sender.

SPF, though, has real difficulties with forwarding; you don't want your mail
tagged as a forgery, or worse bounced, just because the recipient is
forwarding it to another mailbox. If we were validating the envelope sender,
you can rewrite it with SRS in a hacky but mostly workable way when
forwarding. However here I'm talking about the headers, and you definitely
do NOT want your From: header rewritten with SRS when forwarding mail!!

In any case, SRS adds a hash generated from a secret key, which can only be
validated by the person who knows that key, i.e. the originating ISP. That's
fine for checking a bounce which comes back the same way it was sent, but
doesn't help the recipient of a message which may have been forwarded
through several hops to know if the address was valid or not. The recipient
could make a 'callback' (a live TCP connection to the originating ISP) to
have the hash validated, but that's very inefficient. At that stage, unless
anyone can think of any other solutions, we're probably better off going
down the public-key route.

That would mean, in the simplest case an ISP publishing their public key in
the DNS, and all mail relayed via their smarthost with From: 
*(_at_)example(_dot_)com
having a signature added using the corresponding private key. The trust
level would be declared as "RHS is OK, LHS unknown". If the end-user relays
using SMTP AUTH then the smarthost can check the LHS too, and sign it
differently saying "LHS and RHS are both OK"

If the ISP wants the end-user <foo(_at_)example(_dot_)com> to be able to sign 
their own
mails then they could publish the end-user's key under a subdomain (e.g.
foo.example.com) or else the ISP could sign a certificate for them ("this
key belongs to foo(_at_)example(_dot_)com") which they could include in 
messages they
send. Both solutions would allow user foo to send outgoing mail via a
different ISP, as long as their MUA could handle the signing action.

Mails can be forwarded through many hops; the final receiving MTA, or even
the MUA, can validate the signature against the key in the DNS.

Of course, once you've gone this far you could consider signing the whole
message, not just the From: header (apart from the fact that this breaks
mailing lists which munge headers, add footers, etc). But there seems to
have been minimal take-up of public-key crypto so far. I suspect this is
because (a) out-of-the-box MUAs don't implement it, (b) users don't
understand it, and (c) people are not publishing E-mail keys for 
foo(_at_)bar(_dot_)com
in the DNS under bar.com, but rather ad-hoc mechanisms like PGP keyservers.
If all the signing and checking could be handled on the ISP side rather than
the MUA side, so it is completely transparent to the user, and the DNS used
for key distribution, it might be workable.

Is this a sledgehammer to crack a nut though? Just how important is From:
header forgery? If the content of the mail says "you have just won $35m in
our lottery, please E-mail us back to find out how to claim" then having a
valid From: header is no less likely to make this a fraudulent mail!

Answers on a postcard please :-)

Regards,

Brian.