On Apr 05 2006, John Levine wrote:
Not really. I get spam complaints all the time for mail from lists
that I know perfectly well that they signed up for and confirmed.
"Oh, I don't want that any more." For the ones that aren't totally
redacted, my setup turns them into unsubs so they don't get any more
mail for that particular list, but I don't think it's fair to count
mail as spam if it depends on reading the recipient's mind in
real-time.
Isn't that perilously close to saying that if they decided something
in the past, they're not allowed to change their mind?
Of course not. You can change your mind all you want, but you can't
expect me to know that you have done so until you tell me. So if the
user unsubs and the mail keeps coming, then it's spam, but if he asked
for it and hasn't asked for it to stop, it's legit.
What if he likes to read the list messages on a single topic, but
doesn't want those messages that deal with another topic?
Or take the ASRG list itself. For argument's sake, I might be only
interested in micropayment-type systems, and don't want to see any other
threads such as announcements on IETF drafts. When signing up, did I
know that I was going to get IETF announcements? Am I wrong to claim
the list sends me some spam? Did the list description on the ASRG homepage
change at all from the time several years ago when I subscribed and now?
Consent is not that simple either. If I unsubscribe, I lose the micropayment
threads, if I stay subscribed I get spam announcements :)
Testing spam filters is a really, really hard problem, for all the
reasons that people have mentioned, basically that you can't test
without perturbing the system you're testing, and you can't capture
enough state to rerun the same test more than once. So you have to
estimate based on complaint rates from bounces and the like.
Sure is, or nobody would make a big deal about it. I don't agree that
perturbing the system is necessary, however.
Then I have to say that you're not thinking very clearly about the way
that spam filters work. As soon as you do or don't deliver a message,
or delay it by greylisting, or query a DNSBL, or any of a zillion
other things that filters do, you've perturbed the system.
That's the spamfilter operating, not the QA test of how well the
spamfilter is judged to be working.
It is obvious that some spamfilters can change spammer behaviour
directly. We've been discussing (at least I think we have) whether the
act of measuring the operation of a spamfilter itself changes the
spammer behaviour. And my claim is that the act of measuring the
operation of the spamfilter need not affect spammer behaviour
directly.
There's also a subtlety with predicting the effect of a putative,
nonexistent spamfilter if its operation directly changes spammer
behaviour, but that's another thing again.
In the end, testing is about making the most of the information
that's actually available.
Well, sure, but it's also about knowing whether your synthetic tests
have any relevance to the real system you're trying to model.
If you can obtain FP and FN numbers, then I believe you have a
valuable summary of accuracy for that system, but it won't tell you
about other aspects such as cpu load etc. One thing at a time please.
--
Laird Breyer.
_______________________________________________
Asrg mailing list
Asrg(_at_)ietf(_dot_)org
https://www1.ietf.org/mailman/listinfo/asrg