Ian Eiloart wrote:
...
These error rates are most of the time bigger than what can be achieved
by spam filters. So it's probably a bad idea to consider that user
feedback is reliable. User interface shall be as simple as possible.
But wait, we're talking about messages that the spam filter hasn't
rejected. Additional data *has* to be useful.
Sure, I've just mentionned that user feedback isn't reliable. There are many
reasons. One thing to
do to mitigate user feedback lack of reliability is to make user interface as
"good" as possible.
The false positive rates are only a problem if the admin stupid enough
to consider a single report as definitive. If you deliver a message to
100 users, and three report it as spam, then you probably take no
action. If 20 report it as spam, then you need to take a closer look.
Agree, but if 20 report it as spam, but if the other 80 reported it as ham or
if they didn't
reported anything these are different situations.
I certainly don't think a 7% error rate is enough to determine that
users should not be given the opportunity to distinguish between
unwanted mail and reportable junk.
Sure but... I don't think about a binary decision : using or not user feedback, nor trying to
correct user feedback till have a perfect clean feedback. The idea behind Cormack paper is to handle
user feedback the same was as a noisy signal. Try to correct it trivial errors, but accept existence
of noise and make your system as insensitive as possible to noise.
I made an experiment with some users recently. In this particular experiment, I noticed some obvious
things : people are concerned about wanted and unwanted mail - ham and spam isn't their problem;
appreciation for wanted/unwanted messages varies not only from user to user but also depends on the
moment. The latter means that, at different times, the same messages won't be judged the same way by
the same user. Also, error rates aren't the same for different kind of messages, even inside the
same class.
I don't generalize this and, unfortunately, I don't have numbers reliable enough to publish, but I'd
like to have. I don't believe in opinions which aren't supported by reliable numbers.
You can also combine reporting rates with your bayesian content analyser
or spamassassin score, or with your reputational score for the sender
domain, etc.
Sure, but even after that, you still won't have a perfectly clean user feedback.
--
_______________________________________________
Asrg mailing list
Asrg(_at_)irtf(_dot_)org
http://www.irtf.org/mailman/listinfo/asrg