From: Lars B. Dybdahl
Sent: Monday, April 19, 2004 1:53 PM
Mandag den 19. april 2004 19:28 skrev Seth Goodman:
- Being able to e-mail with friends. SPF and Autowhitelisting in
spamassassin makes this easy.
This is all post-SMTP.
I will only be deploying SPF filtering post-SMTP for a start, that's
correct.
Since most domains haven't adopted SPF, yet, there's not much savings
in deploying SPF at the SMTP transaction level, yet
Good point.
- but as SPF
catches on, SPF at the SMTP port will become more important. But
here, you still have to analyze the different kinds of e-mails:
- E-mail addresses can be whitelisted using SPF right now.
- As SPF usage increases, a larger part of the current spam/virus
e-mail types can be rejected using SPF.
- Some spam/virus can be rejected based on blacklists/SPF.
You can never kill all spam at the SMTP level, because people define
"spam" differently. You can, however, reduce the number of e-mails
going into the system, and thereby reduce the load on spamassassin or
other tools (like the bayesian filters in Outlook, POPfile, Mozilla
etc.).
This was exactly my point. SPF and SES target only envelope-from forgeries,
but it's at least a start and it puts the mindset in place that senders have
to be accountable for what leaves their MTA's.
Doing SPF checks after accepting the whole message is better than
not doing them at all, but it throws away two very important
benefits: reducing the load on the MTA and reducing the number of
forgeries passed on to the user.
It does not reduce the number of forgeries if you set up your
mailsystem right. Doing an SPF check before or after receiving the
message body can give exactly the same result.
Another good point, I missed that. If your MTA does the SPF test after
accepting the message and the result is fail, it can't send a DSN but it
needn't deliver it to the user. I'm not sure that is strictly RFC
compliant, but it's the best you can do.
From the MTA's standpoint, once you accept a message, you have to
deliver it or send a DSN.
The complete mailsystem doesn't have to do that. Many e-mail systems
are silently removing worms and spam today, if not most e-mail
systems.
Many systems do this because they are up against the wall, but that doesn't
mean it is a good practice or that we want to keep doing this in the future.
I think the intent of RFC821 and 2821 are quite reasonable: if you accept a
message, you accept responsibility for either delivering it or sending a DSN
back to the originator. Without a method of detecting whether the
return-path is forged, sending DSN's can be an abusive action, even though
required by the RFC's. Hence, determining the authenticity of the
return-path, and refusing to take responsibility for messages with forged
return-paths, is an essential first step in getting back to a more sane
email environment.
By the way - who says that spamassassin and the MTA cannot cooperate,
making spamassassin provide an autoblacklist/autowhitelist to the
MTA? Remember, that SPF is still in its infancy, and that many
mechanisms will be built on top of SPF.
No reason why they can't. Local policy is just that.
It was meant to bring some level of
accountability to email senders and thus make it harder for
spammers to make a living.
Interesting - I didn't know that anyone cared about spammer's living
standards. Personally, I just care about bandwidth usage, server load
and amount of spam e-mails floating around in my systems :-)
LOL :) I do recall that SPF, by protecting individual domains against
forgery, was intended to "herd" to spammers into an ever-shrinking part of
the net that would be more easily identified and thus blocked. In the end,
the best way to fight spam is to make it harder and more expensive for them
to do business. If we can decrease their standard of living in the mean
time, I can't say that bothers me!
appreciate the fact that you don't think SES should be discussed on
this list, but it is really a discussion of flaws in SPF+SRS and a
search for possible solutions.
I don't see the flaw? I normally define flaws/errors as something,
that does not behave according to the specification. Please give
details about, what in SPF+SRS that doesn't behave according to how
it was intended to behave, or please define "flaw".
OK, here are the two flaws that I feel are the most important. Sorry this
is long-winded, but I'm trying to be precise.
1) The SPF architecture checks the return-path at each SMTP-server with
regard to the SMTP-client's permission to use the domain in the return-path
as expressed by the domain owner in the SPF record. Because of that design
choice, the return-path must be rewritten at each forwarding hop. This is
complicated and is the source of a lot of the resistance to SPF adoption.
To some people, SRS is a deal-breaker. That would be a pity. I would
prefer to see SPF modified so that SRS was not necessary.
2) The goal of SPF is that any MTA receiving a message can, through
automated means, detect whether the return-path is forged according to the
policy of the domain owner and potentially refuse to accept the message.
While each MTA in the message transfer chain can assess the credentials of
the SMTP-client sending them the message with regard to the domain name in
the return-path, the end user has a somewhat different perspective. They
don't particularly care about the credentials of anyone in the message path
besides the originating gateway MTA. Those credentials were tested by SPF
at the first hop. By passing the message on, and perhaps by inserting an
SPF header, the first hop site asserted that the return-path as it received
it was valid according to SPF.
If there is a second forwarder, their passing the message on asserts that
the return-path was valid in terms of SPF when they received the message,
but also implicitly that it was valid at all preceding hops. When the
message reaches the final destination, an SPF pass result gives you, at
best, a "composite" assertion as to whether the return-path at the first hop
was valid. Since SPF policy will vary at each hop, as the final recipient,
you have no idea how much faith to put in the final SPF pass result. If
anyone along the chain, particularly the first hop, had a policy of not
rejecting due to SPF fail results, the return-path could well be a forgery
and the final recipient won't know it. I consider that a serious flaw,
since the primary goal of SPF is to detect envelope-from forgeries.
Well, a lot of people do see SRS as extreme and it is probably the
single biggest impediment to widespread SPF adoption.
Not many people are forwarding e-mails. Most ISPs deliver a mailbox,
which is emptied using POP3. Most e-mail clients can empty multiple
mailboxes today, and frankly, I don't know anybody besides myself,
who has a forward on an e-mail that they can't handle this way.
Don't say that in front of the large companies (like pobox.com) who are
primarily email forwarders! Whether either of us uses this feature today is
not particularly relevant because it is ubiquitous. That is why SRS _is_
such an issue. If forwarding were but a minor part of email traffic, I
suspect no one would be terribly bothered.
Then there are e-mail lists, but:
- Most people aren't on an e-mail list... yes, that's true!
- For a start, most SPF checkers will probably be the one in
spamassassin, which will not junk e-mails with incorrect SPF records,
but only uses it as another check for spam points
- Most e-mail lists are based on providers, who have the knowledge to
implement SRS, or are run by nerds, who also have the knowledge to
put in SRS when asked to do so by the list members
Email lists don't have to use SRS since they are "redistributing" the
messages, in RFC parlance, rather than forwarding. They must rewrite the
return-path according to RFC2821 and the resulting message will pass SPF
checks with no additional effort.
Did you know, that Active Spam Killer kills spam just by whitelisting
sender e-mail addresses? It assumes, that the spammers cannot figure
out, which people I have whitelisted. E-mails from other people using
the same domain name are handled using cookies inside the e-mails.
And it works, which might seem very impressive to some people. This
method can also be used to whitelist mailing lists for a start.
Whitelists can work, depending on your environment. I should point out that
a whitelist is valuable to the extent that you can verify that the mail
actually came from the whitelisted address and is not a forgery.
Frankly, I don't need SRS for my own personal mail. I can benefit very
much from SPF without anyone implementing SRS, and later, when SPF
adoption has grown, and it makes sense to use pre-SMTP SPF filtering
to save bandwidth and load, SRS will be widespread, too. SRS is not a
roadblock; consider it to be the small plastic cap you put on the
screws in your furniture to make things look nicer (in case you ever
bought something from IKEA).
Well, I doubt that even most early adopters and SPF evangelists would
trivialize the SRS issue like that.
In fact, the SPF model depends on
_every_ MTA in the message path to do an SPF check on the incoming
message.
I'm not sure what SPF model you mean, but most forwarding system that
I've seen, are like this:
- One or more e-mail servers are MX for a domain
- They all forward to an Exchange server or something like that inside
in a company
- The exchange server is firewalled so that it can only receive
e-mails from those outside e-mail servers
There is no reason to make the Exchange server check anything, since
it only receives e-mail that was already checked by the servers
outside on the internet.
That's not forwarding. What you describe is normal MTA and MSA operation.
Forwarding is when the message is delivered to a foreign MTA, the
destination address is changed and the message passed on. To make things
more complicated, there can be more than one forwarder before the message
reaches the destination gateway MTA.
An SPF fail result says that the
message source does not meet the domain owners published policy for
designated senders for their domain.
Which means that:
- The owner has an incorrect SPF record or
- The owner forgot about his SPF record or
- The from-address is forged
And it's not always a good solution to wait until the owner got a
bounce back and then resends the e-mail. SPF is nice, but if it
breaks the flow of important e-mails in a critical situation, the
boss gets very angry. Therefore, a fail should not mean total
rejected at this early stage of SPF adoption - otherwise a lot of
bosses can get very angry. That's one of the reasons why I see
spamassassin to be one of the most important tools for SPF adoption -
if an e-mail fails, there is more than one reason, and the boss won't
demand removal of that SPF filter thingy. Later, when SPF adoption is
much better, more people will start to block on SPF fails.
If you don't block on SPF fail results, you don't get much benefit from SPF.
Though you can use it as an additional clue for post-SMTP content filters
like SpamAssassin, how many messages does this additional clue push over the
spam threshold?
You probably won't see Meng say something like I said here, because he
must tell people how the ideal SPF world looks like, so that they
understand the principles, but reality is, that SPF won't conquer the
world by june 2004.
He's doing what he needs to do to get SPF adopted as quickly as possible.
However, I agree that this timetable is optimistic. That's OK, he's not
going to get it adopted by being a pessimist.
That's certainly what this is all about. Part of the notion of
"email working well" is receiving fewer forgeries in your inbox.
Huh? Personally I don't care if an e-mail contains a forgery in sender
e-mail address, as long as I want to read that e-mail. SPF doesn't
change that, since a forgery is as simple as borrowing another
person's computer. I think we agree, that you mean "fewer unwanted
e-mails", but be careful about your choice of words if you want to
apply deduction to your statements :-)
I suppose we disagree on this one. I have no interest in reading a message
that I know has a forged return-path. In an SPF-compliant world, depending
on how your SPF record is set up, you can't necessarily borrow someone
else's computer to send your messages with your return-path using their MTA.
You may have to connect to your MTA with SMTP AUTH, use a VPN or if all else
fails, use a web interface to send the mail. It's a similar problem to
digital certificates on your computer. I doubt that many people would think
it's OK to use someone else's computer to send email and then sign the mail
with one of their certificates. That is clearly forgery, and in SPF terms,
sending mail with your domain as return-path through someone else's MTA is
forgery as well.
--
Seth Goodman