On Sat, 14 Aug 2004, Philip Miller wrote:
Dean Anderson wrote:
On Tue, 10 Aug 2004, Rand Wacker wrote:
On Tue, 10 Aug 2004, Dean Anderson wrote:
It has been reported that AOL is already using SPF to give reduced
filtering to SPF-using domains. Is this a good idea?
Incorrect, see http://postmaster.aol.com/spf/:
AOL will begin using SPF records to maintain our whitelist in the near
future. If you want to remain on AOL's whitelist, you will need to
establish an SPF record for your domain as AOL will begin to query
whitelisted IP addresses from a domain's SPF record shortly. Without an
SPF record, your whitelist entries may expire.
What do you mean by "establish an SPF record"? This just means putting an
SPF record in for your domain right? Is there something AOL-specific that
must go in this record?
It means exactly what it says: publishing a TXT record with the magic number
'v=spf1',
that complies with the draft SPF specs on spf.pobox.com. There are no
AOL-specific
modifiers or mechanisms in SPF.
Ok, that's what I thought.
And you are right, giving *preferred* treatment for successful
authentication (without other tools like a whitelist) is a bad thing to
do.
Ok, but if you already have a whitelist, what's the point of having SPF?
AOL maintains a list of domains that have indicated to AOL that they are
bulk mailers, and are willing to comply with AOL's rules for bulk
mailers. One of those rules is that if AOL gets too many 'this is spam'
complaints about messages from a domain on the list, that domain is
removed from the list. Currently, AOL maintains a private list of IPs of
these senders, entered by hand by the senders, which they call their
'whitelist'. AOL wants to replace a private, proprietary list with a
distributed list accessible to anyone.
Others also already do this, but don't need a special protocol to
implement it. Further, it is still a proprietary, AOL-specific whitelist.
It seems to me that this would be beneficial to other ISPs competing
with AOL, because they would gain access to a large portion of the data
AOL has collected.
I don't see how this puts anyone else on an equal footing. Its rather like
the "standards" published by Microsoft. Microsoft makes the same claims
about those standards, too. But we know that in fact they benefit
Microsoft.
One can easily use DJB's rbldns for a DNS whitelist, with practically the
same Sendmail/etc rule as a DNS blacklist. There's only a trivial config
difference between DNS whitelisting and DNS blacklisting.
If you use SPF to maintain your whitelist, then anyone who creates an SPF
record will get whitelisted. Do you mean that only certain domains will be
whitelisted? If so, then it seems to me that SPF is just a proprietary
protocol for AOL to maintain its whitelist, and a way for AOL to
discourage outsourcing. Neither of these things seem to be in the
interests of the IETF or the internet community.
Only domains that ask to be whitelisted by AOL get whitelisted. They must
currently enter
their outgoing MTA data into AOL's database by hand. AOL is moving from a
proprietary
database accessible only to them to a standardized, public, distributed
database of
senders. This puts the rest of the email-receiving world closer to even
footing with AOL.
I think its not public (the whitelist). Only _some_ domains that use SPF
will have privileges. This is just a whitelist.
SPF publication appears to be a new ADDITIONAL requirement to enter and stay
on the
whitelist. If anything, it replaces the manual entry of MTA IP addresses,
which means less
work for legitimate senders of all stripes.
As for discouraging outsourcing, SPF makes it pretty easy to list
outsourcers, via the
include mechanism.
Its also pretty easy to deny outsourcers, via refusing to use the include
mechanism. This is my main objection. Outsourcing isn't something that
AOL or anyone else can be allowed to have any control over. Giving such
control to AOL or others is an actual harm.
The rest of my objections about failure to work, failure to stop spam, etc
aren't actual harms, they just mean the SPF effort is a waste of time.
Your waste of time doesn't ordinarilly harm anyone, except in this case,
as above.
That said, you need more than just a whitelist. If you subject any email
to less filtering, and it is known from which domains you do that with,
you will encourage abusers to conduct abuse from those domains, and your
users will get more junk. Whitelisting has to be done carefully, and its
more of a response to abusive blacklists by selectively rejecting the
blacklist than a rational response to spam.
If those 'known whitelisted' domains all use SPF, then how will abusers
conduct abuse from those domains?
That's easy. They conduct abuse the same as they do now: They can
either forge other users at that whitelisted domain, or they can use
disposable accounts at the whitelisted domain, or they can steal accounts
from the whitelisted domain.
Your misconception here is well-known and unfortunately common. The is
misconception that somehow the abusers can't choose to use the relays of
the ISP they are using. But of course they can, and of course the ISP's
relays offer the same benefits as open relays. But in fact, most abuse
doesn't use the ISP's relays, nor does it use open relays. At present,
most abuse connects directly. But that's just a rather arbitrary choice
made by the abuser. They can always choose to use the ISP's relays, and
they always have 3 basic choices of how to abuse the ISP's domain. They
connect directly at present merely because its easier than figuring out
the correct ISP relays.
What your misconception boils down to is the assumption that abusers
don't make some arbitrary choices about how to send email and can't adapt.
This has been common to every failed anti-spam technique.
We also whitelists, but only around particular filter flaws: One of things
we do is count each occurance of an IP address in headers. Too bulky, and
messages with that IP address get queued instead of delivered. This test
has to be disabled for common addresses (192.168/16, 172.16/14, and 10/8)
as well as our own mailservers and those of large ISPs. This only modifies
_this_ particular test, and only because _this_ test is invalid if there
is genuinely a lot of email from a particular IP address. But I would not
want to whitelist just any domain that has an SPF record. That would be
bad.
You can do whatever you like on your mail servers. AOL is telling the world
of email
administrators that if they want their mail to continue to be delivered to
AOL, they must
publish SPF records. They are not lowering any barrier, they are raising
another one.
First, they aren't actually doing that. They are just 'maintaining a
whitelist'. Second, they aren't ever likely to do that, for exactly the
same reason they don't require the foolish in-addr check. Not everyone
will have the records, and AOL won't be able to refuse email because of
that. All they can do is choose to lessen the filtering applied to those
with SPF records. That's a lowering of the barrier.
Oh, and they will also be able to interfere with outsourcing.
Interference could come in the form of outright refusal or extortion of
fees, service interference through "unfortunately" unreliable records,
etc.
But I have some doubts about the overall validity of that bulkiness test.
If the test were valid, it might in fact be helpful if Large ISPs did
identify their outbound servers. However, finding out this information is
not hard, if only you monitor your mail queues. It hardly seems worth the
trouble of an IETF protocol.
And if an ISP suddenly brings a new MTA infrastructure online, they should
wait until
individual administrators see it being rejected in their logs and enter it by
hand?
Senders should bear the burden of identifying themselves.
Not senders: recipients. The recipients monitor their logs and when they
see a bulky sender IP, they check it out. The reason that this might not
be helpful is that abusers can adapt their sending pattern so as not to
appear bulky. Thus, such a check might not be valid.
You seem to indicate that every domain that runs an MX should take
responsibility for identifying legitimate sources independently, without
the benefit of information that those senders can publish, and that
other ISPs have already gathered.
Yep. The reciever can't just trust the sender isn't sending spam.
Non-abuser can publish information. Abusers can publish the same
information. Imagine a Bank that asked its customers for their credit
rating.
Subjecting /un-authenticated/ messages to increased scrutiny might be
justified to do though (as Microsoft has announced they will start
doing).
I'd like to draw your attention to relativity:
Subjecting /un-authenticated/ messages to _more_scrutiny_ is the same as
subjecting authenticated messages to _less_scrutiny_
How about authenticated messages are treated exactly as all messages
currently are, while
unauthenticated messages receive more scrutiny than all messages receive at
present?
Same thing. Abusers can "authenticate" themselves, and get less scrutiny.
There is no reason to conclude that MSN will have less abusers with SPF
than without SPF. In fact, once the word is out that MSN is subjected to
less scrutiny, abusers will seek out MSN accounts. This means that AOL
gets more spam and MSN gets more abusers. Good plan. Then maybe MSN and
AOL will then halt services, and abusers won't be smart enough to go to
other providers....
--Dean