On 4/4/06, Laird Breyer <laird(_at_)lbreyer(_dot_)com> wrote:
I agree, and I'm not proposing that. A typical statistical hypothesis
test has the option of being inconclusive. But here we're not even
measuring the same FPs so it's moot.
A ROC curve is a "Receiver Operating Characteristic". Imagine you have
a spam filter sytem and a panoply of subsystems and parameter values
to choose from. For each possible setting, you measure the (FP%,FN%)
pair on some data. Then you plot these points in a square of side
length 100%. This gives a picture of the overall quality of the spam
filter as you insert or remove the subsystems and parameters.
It allows you to compare different spam filters over various operating
ranges, and gives insight into the accuracy tradeoffs you make when
you replace one subsystem with another.
This tutorial may be a little technical, but the important things are
the pictures.
http://www.cs.bris.ac.uk/~flach/ICML04tutorial/ROCtutorialPartI.pdf
This is a readable introduction with a lot of detail.
http://home.comcast.net/~tom.fawcett/public_html/papers/ROC101.pdf
Which brings us back to tagging. In the tagging vision, the end-user gets
to adjust their own sliders and weights based on verifiable analyses done
by upstream expert systems.
--
David L Nicol
Should the bike shed have bunks? Or maybe cots?
_______________________________________________
Asrg mailing list
Asrg(_at_)ietf(_dot_)org
https://www1.ietf.org/mailman/listinfo/asrg