ietf-asrg
[Top] [All Lists]

Re: [Asrg] Testing content filters

2003-03-18 20:39:37
From: Matt Sergeant <msergeant(_at_)startechgroup(_dot_)co(_dot_)uk>

One thing the anti-virus people got right is the ability to test 
anti-virus engines by passing through a non-viral file that is by 
de-facto detected as a virus by the vendors engines. This file is 
called EICAR, and is a simple .COM executable that does nothing but 
trigger AV engines.

In order to provide something similar for spam content filters, I 
created something similar for spam. We put support for this into the 
SpamAssassin project and notified some other creators of spam content 
filters. Unfortunately they all ignored us.

I think this would be a good thing to support in content filters.

We called this thing "GTUBE" (Generic Test for UBE), and you can find 
the email that should be stopped at 
http://spamassassin.planetmirror.com/dist/t/data/spam/gtube.eml (note 
the headers are abitrary - it is the long string in the body of the 
message that should be detected).

The email is designed in such a way that it should not be munged by any 
gateways.

I urge other vendors of anti-spam content filters to support this. It's 
trivial, but it gives users the power to test that their filter is 
working.

I also think this would be something worth standardising through the 
IETF.

That sounds like a good thing to have, but nothing that should involve
the IETF (not to mention the IRTF).  The IETF cannot standardize anything
in less than 6 months and often neeeds a year or two.  By the time such
a test case were standardized, the spammers would have stopped some
behaviors and started others, making the RFC of only theoretic or historic
interest.

More fundamentally, you don't want to standardize the test case, but
to publish or exchange the current version of the test data.  IETF
standards are about ensuring interoperability, and are cast in stone.
It takes almost as much work to issue a replacement RFC as to create
the original, but you'd want at least quarterly updates for your test.

That the business models of anti-virus and anti-spam services differ
might be a more serious and fundamental problem.  Anti-virus protection
needs to be perfect,and so AV vendors can cooperate.  Anti-spam
protection is allowed and expected to be imperfect, so anti-spam
vendors are able to keep some of their patterns or whatever secret
to compete in customer tests.

This is not to say that you can't publish a non-working-group
Informational RFC, but that you should not be disappointed when it
joins the 95% of RFCs that are write-only/read-never.

As an asside, note that contrary one reading of the suggestion,
effective ilters need to deal with the modifications inflicted on mail
by gateways.  For example, somethings seem to encoded and decode
quoted-printable...never mind why or whether they should.


Vernon Schryver    vjs(_at_)rhyolite(_dot_)com
_______________________________________________
Asrg mailing list
Asrg(_at_)ietf(_dot_)org
https://www1.ietf.org/mailman/listinfo/asrg