ietf-asrg
[Top] [All Lists]

Re: evaluating proposals against requirements (Re: [Asrg] requirements for a proposed solution + notion of consent)

2003-03-09 10:53:28
In <20030307192548(_dot_)A4583931(_at_)exeter(_dot_)ac(_dot_)uk> Adam Back 
<adam(_at_)cypherspace(_dot_)org> writes:

Those requirements sound good to me, it may not be possible to satisfy
them all simultaneously but at least they express the problem so
people can measure solutions against them.  I've numbered them, let's
measure a few of the current proposals against these:

a) should minimize spam to some acceptable level

I think it is a mistake to only consider proposals that reduce spam to
"some acceptable level".

For example, the proposal to have MTAs give a temporary delivery
failure status the first time it recently has seen a sender-recipient
pair may only reduce spam by 20%, but I think that is still useful.

If you find 5 independent tests that each reduce the spam by 20%, when
combined you will have reduced spam by 99.968%.  Granted that most
spam tests will not be independent, so in practice you will have to
find a lot more than 5 such testes to get that level of spam
filtering, but the point is that every little bit *DOES* help.

I like the spamassassin approach that uses many different techniques
to check for spam.  Each test may only filter a small amount of spam,
and some things like their checking for forged email from
AOL/Hotmail/etc and DNS blacklists can give false positives on that
particular test, but over all, it does an excellent job of filtering
spam.  SpamAssassin is far from a perfect filter, there are lots of
things it still does wrong, or could do a lot better, but SpamAssassin
is the most effective tool I've found.

Bayesian filters work in a similar manor.  Each word may not be a
reliable test of spam/ham, but when combined with the results from
many other words, the overall is effect is quite good.


For what it is worth, I would say that SpamAssassin would rate as
follows:

a, b, c, e, f, g, i, j

It currently fails 'd) should be easy to use' if the ISP doesn't do
the installation and also provide a nice front end to do
blacklisting/whitelisting.  (SpamAssassin's automatic
blacklists/whitelists are good, but sometimes you need explicit
control.)

SpamAssassin also requires more CPU power and disk space to hold the
per user autowhitelists and bayesian analysis.  I personally think
this cost is far below the cost of peoples' time, but YMMV.


-wayne


_______________________________________________
Asrg mailing list
Asrg(_at_)ietf(_dot_)org
https://www1.ietf.org/mailman/listinfo/asrg