-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1
Justin Mason wrote:
Chris said:
Okay, suggest how a spam/ham collection can be used to measure the
effectiveness of the following techniques that are, or can be, used in
an anti-spam solution:
1) grey listing
2) sender/sender domain verification
3) Challenge/response
4) SPF and DKIM
5) PKI
6) CSV
7) Non-existant users
8) DCC or other distributed checksumming methodologies.
Similarly DNSBLs.
The effectiveness of all of these, except possibly for greylisting and
C/R, can be measured accurately. We are doing it in SpamAssassin ;) --
here's how.
First, you accept every message, and record the "real-time" data
points regarding how the message was listed against those services,
and/or how it *would* have been rejected at SMTP transaction time (if
at all). However, you don't reject, you accept everything.
Then, later, provide a way for hand-sorting to take place, and compare
the results of the hand-sorting with what the various other techniques
would have done with those messages.
Hey presto, you've now got a way to compare accuracy and effectiveness
of those techniques! Simple as that.
Do you see any provision for that with _any_ of the spam/ham
collections? Can you do that with a pre-existing spam/ham collection if
the technique you were trying to test _wasn't_ being collected at the
time the spam/ham collection was being made?
This points up my very point: in order to do a proper sampling of the
effectiveness of a technique is that you do it in real time, at the time
the emails were sent. In other words, on real mail streams.
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.2.3-nr1 (Windows 2000)
Comment: Using GnuPG with Thunderbird - http://enigmail.mozdev.org
iQCVAwUBRC8bx53FmCyJjHfhAQL1cgP/eFRHoOrKLE4QKfHIDAmyRMpx3+CDEzKD
J3wBSECeOy2NdgIEO42ivKiATmWCLP1sqLLCV5bCeEiZ2vLVyua1at3n28ProgUE
f8kUxawXbk8phRFbklBGk3YxXskrQkKsCb/bj+OzV840j3eVeR8xrpS3iAP2+tYz
jqaLUvcnoo0=
=hlXg
-----END PGP SIGNATURE-----
_______________________________________________
Asrg mailing list
Asrg(_at_)ietf(_dot_)org
https://www1.ietf.org/mailman/listinfo/asrg