ietf
[Top] [All Lists]

Re: "Principles" of "Spam-abatement"

2004-03-17 16:27:29
On Wed, 17 Mar 2004, Dean Anderson wrote:

On Wed, 17 Mar 2004, Robert G. Brown wrote:

  a) Preparse the header so that the entire delivery path is the primary
content of the message, with the message itself added (header intact) as
an attachment.

This is true now.  Many people don't know how to parse the headers, but it 
obeys a specific syntax and is machine parseable.

Of course.  To both -- many people don't know how to parse the headers.
I'd estimate some 499,950,000 of the half a billion users of mail.  So
the CAPABILITY of parsing the headers exists, but not even AV vendors
(who should know how and know better) do it.

  b) Return the complaint as mail to postmaster of the originating
network with a special error code that would allow it to be counted and
digested easily.  One doesn't want to create a new kind of DoS attack on
postmaster, and making it easy and automatic to return a complaint COUNT
of some sort on otherwise identical content can help prevent that while
making it easier for SP admins to deal with mounting complaint levels.

This is possible now, and SpamCop does this. The problem with SpamCop is
that they alter the message, making it useless.  SpamCop complaints are
routinely deleted as a result.  I usually forward them on to customers
with an FYI that we don't consider such to be a valid complaint, but they 
may want to be aware of what someone said.

So what you are saying is, that this CAN be done and even is being done,
but it isn't done correctly and consistently and isn't widely available.
I agree.  In fact, I think that this is potential work for the IETF --
define a way for it to be done correctly and consistently (which
wouldn't be too difficult, I imagine) via a protocol.  Then SpamCop
could fix their tool, SpamAssassin could add a compliant interface,
Joe's Spam Seal (not yet written) could be written to comply with the
protocol and abuse@ or postmaster@ addresses could actually AUTOMATE a
response process based on consistent completely reported problems.  This
lowers their cost and makes it easier and more likely that they'll take
effective action.

As you say, if you get insufficient information, there is little that
you can do even with the best of will as the manager of an ISP with too
little time and too many things to fill it.  You probably don't have
time or energy to engage in the "please resend your complaint with the
full header and message attached" game (known to administrators
everywhere, and not just in regard to mail:-).

  c) Work out what to do about relays and proxies, again to prevent
man-in-the-middle DoS more than anything else.  One site should not be
able to generate mail that it "forwards" with fictitious headers and
evil content so that it appears to come from some hapless but innocent
network.

Relays don't add ficticious headers, nor do they add evil content.  They
may place their (true) headers on top of ficticious headers.  They cannot
verify that the headers given are accurate.  This is true of all relays, 
open or closed.

Proxies are another story. I don't know of any genuine reasons to run open
proxies, though closed proxies (web caches) are clearly useful. I don't
know of anyone claiming they are useful. But neither does that mean they
have no use.  However, as a service provider, I would say this much:  If a
customer found a useful reason to have an open proxy, then I would only
insist that they keep logs of its use. This is easy to place in a contract
or AUP.

Again, precisely.  So what this means is that if I set up a mail server
and send a lot of spam from it all with a header that is forged UPstream
from my host, I obscure my host as its true point of origin.  I can pick
on a hapless host somewhere far away and make it look like the spam
originates from THEIR network, and one would have to compare traffic
logs on the two hosts and/or intermediary hops to determine which of us
is lying.

This makes for an interesting "attack" -- send egregious spam purporting
to come from somebody you hate or some network you are in competition
with, in a messages with multiple relays.  Somebody with a big view of
the hand might finally figure out what was happening, but proving it
would be, well, "difficult" doesn't begin to describe it, given that all
the log data on both sides could be easily forged.

Again, there might be work for the IETF here that could help IF this
becomes a problem or as a preemptive measure now.  There are ways to do
this -- signing a key with a local secret and adding it to the message,
for example -- that even if you didn't have a universal PKI for all
participating hosts would make forging headers for openly fraudulent
reasons easy to catch.  You can forge the upstream route, but you cannot
forge the upstream keys, and the last common host whose key can unlock
their MTA-added tag is left holding the responsibilty bag.

  d) Take steps to ensure that SPs run a postmaster address, and maybe

There is already a BCP for this. Rarely is this a problem.

In the US.  Try hitting the postmaster of 7.197.76.17, and good luck
communicating with the postmaster of the brazilian relay in my previous
reply.  (This is picking nits -- actually I agree that what can be done
largely has been done, but it would be lovely to be able to extend
overseas with a little more ease, to get a bit poetic...;-)

come up with things like Jeff Chase was proposing to continuously
measure their responsiveness to spam/virus-class bounce messages and
globally blacklist the worst (least responsive) offending SPs.  Etc.

How would you define "responsiveness"? Does it mean getting an 
auto-responder message?  Does it mean getting a message saying something 
had been done?  You cannot give out certain information about customers.  
Basically, all you can do is send an auto-responder message and a notice 
that the ticket was closed.  That doesn't indicate what happened, nor does 
it indicate who the spammer was, or if the ISP agreed they were a spammer.  
Sometimes the action is obvious if, say, a website disappears. But how do 
you know they took action against a dialup customer?  You can't.

No, I think that responsiveness has to be measured by the level of spam
reported as originating within the site -- results.  I don't think this
is that difficult, actually.  Vernon pointed out that once earthlink got
serious about controlling spam, the reduction in traffic was very
apparent.

If the IETF has any role here (and it may not) it might be to codify the
process of blacklisting ISPs that aren't serious as they are revealed.
You've pointed out several times that abuse of blacklists is pretty easy
as things stand.  It shouldn't be.  And people like you who have bad
experiences with the previous non-process should be the ones who end up
agreeing that whatever schema that might be proposed is fair and
equitable and not likely to punish ISPs who are doing their best.

Right now enabling SPs are insulated from any kind of RFC-based
responses or complaints about spam because MUA's and MTA's have no
predefined protocol for generating such a response in a constructive
way.  

???  I'm not sure what you mean. By the time you -read- a spam, it is
delivered, and the SMTP protocol has long since finished.  Spam isn't the
only kind of abuse that an ISP responds to. The abuse@<isp> works pretty
well, since you can forward the complained-of message. There are some
things I'd like MUA's to do, such as include the whole headers(some MUA's
do, some MUA's don't), but that isn't an RFC issue, either.

Why not?  SPAM is a major networking problem.  Reporting spam and
viruses (at the MTA or MUA level, as filters can work at either one) is
only going to be effective if it is easy to report spam and if the
reports are consistent and contain the information needed to combat
spam.  Most MUA's cannot help users to report, and users definitely
cannot help themselves.  If MUA writers tried to add a reporting
capability now, they'd botch it by coming up with a dozen different
formats and effectively precluding the creation of automated tools on
the other end to facilitate processing the complaints at minimal cost.

Perhaps not an RFC, but a standard is most definitely needed, and it
needs to be both articulated and discussed in an open RFC-like process.
And there are already RFC's that address this, e.g. RFC 2635.  It is
just that this RFC reads like spam for the beginner (intended for users,
a sort of anti-spam howto) and doesn't spell out anything like a uniform
protocol for reporting spam TO the abuse@ address it suggests MTAs
maintain.  Lacking this (and due to the fact that nobody reads RFCs but
techie nerds; others don't even know what RFC stands for) the document
isn't horribly useful to a software designer wanting to actually design
TOOLS that might be portable and consistent.

Most complaints/bounces that are automatically generated by
antivirus software or reported by humans (I've read plenty of both:-(
are hopeless and de facto useless without several rounds of
communications, and sometimes not even then: the humans don't even know
what a mail header IS and often have no way of knowing or suspecting
that the From address is bogus or sending in the real header so it can
be parsed by the SP postmaster.  Antivirus software developers should
know better (damn it!) but even THEY don't bother to parse the header or
include the header in the stupid bounces they generate, or validate
any sort of correspondance between originating host and From address.

Yes, it would be good to send the entire headers.   

But users should know that the from: header can be forged. They should
also know some other things about email and the internet, such as don't
open attachments you aren't expecting. If you get an unexpected
attachment, ask the person if they sent it. This is an education problem.

Users SHOULDN'T have to know ANYTHING but that if they fill in an
address here and write a message and click this little button it goes
where you want it to.  The fact that they do is a failure in design.  It
may be a failure we cannot fix, but it is a failure.  Kids send mail.
Old ladies send mail.  Stupid people can send mail.  The process should
be robust and idiot proof AT THE TOP LEVEL, although that is indeed not
an IETF issue, it is a software design issue.

The IETF issues are to places where by specifying particular ways of
handling certain problems according to a clearly spelled out protocol,
they enable software designers to write software that FIX these flaws
that is consistent and interoperable.  Reporting spam would clearly be
much more effective if it were done in a standardized way.  If
standardized report formats existed, software designers would have a
clear roadmap for adding reporting features to MTAs that my kids, that
old ladies, that stupid people could use without even knowing that a
"mail header" exists.  It enables other software designers to write
tools for the other end to preparse traffic to abuse@ addresses into
efficient, condensed reports so that instead of getting 1000 different
messages reporting one spam incident (which yes, one would be tempted
to pipe into /dev/null) one gets ONE message reporting 1000 different
complaints of ONE spam incident (with many targets).

Get it to where ISP admins can whack those moles efficiently and easily,
and they might just have at the mole-heads in their subnets with a will
and a way.

   rgb


              --Dean



-- 
Robert G. Brown                        http://www.phy.duke.edu/~rgb/
Duke University Dept. of Physics, Box 90305
Durham, N.C. 27708-0305
Phone: 1-919-660-2567  Fax: 919-660-2525     
email:rgb(_at_)phy(_dot_)duke(_dot_)edu