--On 21 December 2009 19:24:06 +0100 Matthias Leisi <matthias(_at_)leisi(_dot_)net>
wrote:
Am 21.12.09 18:46, schrieb Nathaniel Borenstein:
distinction, it's whether the users can. If the users can't use the
client's two buttons with sufficiently low error rates, then the
resulting data can't possibly be useful to the admins. In other
When I was responsible for spamfilter operation at a former job, error
rates of "human spamfilters" were considerably higher than FP rates of
any possible solution. This is not scientific evidence, but it
illustrates it nicely:
I nice story. It says that we must not block mail on a single report,
especially a report from a different user. That's fair enough, but it's not
what anyone is suggesting.
The current question is how can we make it easier for users to report spam
to administrators. The question of what administrators do with the reports
is secondary - though it might inform the kind of information that they
want to gather.
It was very important for the CEO of the company not to lose mail. It
was thus decided that his assistant would go through his inbox and spam
folder and use "Mark as Spam" and "Mark as Not Spam" buttons to clean up
things.
This assistant was very diligent, and highly capable at her job.
Nevertheless, she had a surprisingly high error rate -- for every
hundred mails which she marked as (not) spam, she mis-categorized maybe
five to ten mails.
I did not watch the other users as closely, but they had similar error
rates. The feedback loop from the users could never have been used for
automated actions due to the low quality (maybe it would have been
possible with a larger user base and appropriate statistical analysis).
The buttons were still an important source for fine tuning of the filter
and to find emerging trends in spammer behaviour. Nothing more, but
nothing less.
-- Matthias
_______________________________________________
Asrg mailing list
Asrg(_at_)irtf(_dot_)org
http://www.irtf.org/mailman/listinfo/asrg
--
Ian Eiloart
IT Services, University of Sussex
01273-873148 x3148
For new support requests, see http://www.sussex.ac.uk/its/help/
_______________________________________________
Asrg mailing list
Asrg(_at_)irtf(_dot_)org
http://www.irtf.org/mailman/listinfo/asrg