ietf-asrg
[Top] [All Lists]

Re: [Asrg] Re: "worm spam" and SPF

2004-12-05 20:43:52
On Dec 05 2004, gep2(_at_)terabites(_dot_)com wrote:

 But anyhow, the fact that they "don't follow standards properly"
doesn't really affect my proposal one way or the other.  My proposal
should still work, and one can set the filter as aggressive (or not)
as is appropriate for their situation.

What does "how they interpret ASCII" have to do with my proposal?

The problem in a nutshell is this: when your filter receives a mail
and analyses it to see if it contains prohibited content such as HTML
(to take an example), what criteria does it use? If the criteria are
different than those applied by the mail reading program downstream,
then there will be a discrepancy. Either the filter blocks mail when
it shouldn't (which is often called a false positive), or it doesn't
block some mails which will turn out to contain HTML (possibly
malicious) anyway.

There are some standards for when a mail contains HTML and when it
doesn't but they are often ignored by mail reading software, and often
ignored by spammers, and also ignored by legitimate senders depending on
how the programmers who wrote their mail programs interpreted the gray
areas in standards. 

So unless your filter uses the exact same heuristics about when a
message contains HTML and when it doesn't, there will be discrepancy
as above. Either you accept that there will be an amount of false
positives, or you accept that your users will see HTML content even
though your filter thought it only contained plain text.

The only sure way to protect users against HTML attachments is to
prohibit them from using mail software which displays HTML, so at
present if that is your goal for unknown mail, you would certainly
prohibit Outlook, OE, Lotus Notes, etc., and in the future you would
allow those programs if they guarantee to not display HTML, XHTML,
XML, flash, ActiveX, etc.

It MIGHT have more to do with the proposed ASSOCIATED use of a good
antispam content filter (like Spam Assassin) but my proposal really
has only VERY limited need (itself) to interpret the contents of the
E-mail, beyond looking at attachments and HTML tags.

What you perhaps don't realize is that an attachment marked plain text
but containing HTML tags is often displayed as HTML by mail reading
software anyway. Some software reads plain text, looking for anything that
resembles a web address and generates a clickable URL (thereby turning
the plain text into HTML). Some software accepts even malformed HTML
tags, which aren't considered HTML by any standards.

My point is that when you say "looking at attachments and HTML tags",
you are glossing over a difficult problem, because while ordinary
users tend to produce mail where attachment types and HTML are clear,
spammers purposefully make those aspects as hard to identify as they can,
by exploiting gray areas.

Moreover, you might not realize that SpamAssassin is bundled with 
bulk mail sending software precisely so that spammers can design their
emails against SpamAssassin and tweak them until they pass the tests
performed by SA, and only then start spamming. 

Furthermore, typical spam messages are 2-3K in length, well below
your proposed limit.

Yes, and that causes NO problems for my approach.  I'm NOT proposing
that SIZE be used to discriminate spam/nospam.  I *am* saying that,
along with antispam/antivirus/antiworm goals, it ALSO makes sense
that recipients be able to prevent abusively large messages from
being sent to them (and perhaps chewing up their limited-size
ISP-provided Inbox) from people they don't know and don't trust.
(Limiting the size DOES, however, provide an additional way of
controlling incoming worms (from untrusted/unknown senders), most of
which are significantly larger than 25K).

You are proposing that size be used for discrimination. What you are not
doing is proposing that it be used exclusively. There are two avenues
for worms under your scheme. Worms can get smaller in size, or worms can
stay the same size and spread via trusted senders, by looking for 
regular correspondents in address books etc. Fifteen years ago viruses
were less than 1K in size, there is no reason beyond lazyness why worms
need to be around 25K. 

Throughput is generally not a problem for spammers and virus
authors, since they as a rule don't pay for it, or at least they pay
for a vanishingly small percentage of the total bandwidth their
actions cause to be consumed.

It is a problem. Once a spamming campaign is started, it's a race
against time before spam defences recognize and try to block the
campaign, so speed matters. Also, the spammer economy is
structured. Your typical spammer rents facilities from people who own
zombie armies, open relay lists, bulk mail sending software etc.  The
costs for spammers are not always negligible, and I expect they are in fact
as high as the market will bear.
 
 It would be ENTIRELY possible to implement it entirely within an
E-mail client (such as Outlook or Outlook Express for example) and
wouldn't require ANY changes whatsoever from the ISPs or the wider
Internet infrastructure at large... 

Sure. I'm not arguing you shouldn't implement it, I'm arguing that
the scheme isn't as foolproof as might be hoped. The devil is in the
details, because the high level concepts in which you describe the 
scheme do not map perfectly into the low level concepts required for 
implementation.

Some problems with whitelists: 

Once you've whitelisted all your friends and colleagues, you depend on
how good *their* spam defenses are. If they receive a worm, it can
travel to you freely. And worm writers realize this, so it is one of their
priorities.

If you have a finely grained scheme which requires giving permission
for all sorts of attachments on a case by case basis, you'll be investing
a lot of time into maintaining your rules and regulations, even if the
software can handle them in the blink of an eye. This depends on how much
and how varied your mail is (the problem is worst for customer service
type employees), but it's clearly a case of diminishing returns.

Do you intend to share whitelist data with colleagues? If so, how
does the software deal with conflicts?

Finally, what happens to the emails which are blocked by the whitelist?
Are they silently thrown away, do they collect in a folder which must
be checked periodically, do they pop up a message informing the user?
The last two schemes don't reduce the amount of work a user must do,
as each message must still be scanned. 

-- 
Laird Breyer.

_______________________________________________
Asrg mailing list
Asrg(_at_)ietf(_dot_)org
https://www1.ietf.org/mailman/listinfo/asrg