ietf-asrg
[Top] [All Lists]

Re: [Asrg] Adding a spam button to MUAs

2009-12-21 09:33:06
Ian Eiloart wrote:

...


These error rates are most of the time bigger than what can be achieved
by spam filters. So it's probably a bad idea to consider that user
feedback is reliable. User interface shall be as simple as possible.

But wait, we're talking about messages that the spam filter hasn't rejected. Additional data *has* to be useful.

Sure, I've just mentionned that user feedback isn't reliable. There are many 
reasons. One thing to
do to mitigate user feedback lack of reliability is to make user interface as 
"good" as possible.


The false positive rates are only a problem if the admin stupid enough to consider a single report as definitive. If you deliver a message to 100 users, and three report it as spam, then you probably take no action. If 20 report it as spam, then you need to take a closer look.

Agree, but if 20 report it as spam, but if the other 80 reported it as ham or 
if they didn't
reported anything these are different situations.

I certainly don't think a 7% error rate is enough to determine that users should not be given the opportunity to distinguish between unwanted mail and reportable junk.

Sure but... I don't think about a binary decision : using or not user feedback, nor trying to correct user feedback till have a perfect clean feedback. The idea behind Cormack paper is to handle user feedback the same was as a noisy signal. Try to correct it trivial errors, but accept existence of noise and make your system as insensitive as possible to noise.

I made an experiment with some users recently. In this particular experiment, I noticed some obvious things : people are concerned about wanted and unwanted mail - ham and spam isn't their problem; appreciation for wanted/unwanted messages varies not only from user to user but also depends on the moment. The latter means that, at different times, the same messages won't be judged the same way by the same user. Also, error rates aren't the same for different kind of messages, even inside the same class.

I don't generalize this and, unfortunately, I don't have numbers reliable enough to publish, but I'd like to have. I don't believe in opinions which aren't supported by reliable numbers.

You can also combine reporting rates with your bayesian content analyser or spamassassin score, or with your reputational score for the sender domain, etc.

Sure, but even after that, you still won't have a perfectly clean user feedback.


--

_______________________________________________
Asrg mailing list
Asrg(_at_)irtf(_dot_)org
http://www.irtf.org/mailman/listinfo/asrg