"Sauer, Damon" <Damon(_dot_)Sauer(_at_)BellSouth(_dot_)com> wrote:
Believe me, I am *really* trying to understand what it is that we are
measuring and for what purpose.
We wish to gain confidence in our understanding of the current
impact spam has on our networks. Some statistics exist, but as Dave
Crocker pointed out in the meeting, they're small samples, self
reported, and are insufficient for achieving a high level of
confidence. (That's a loose paraphrase.)
So we need a survey, with a large number of respondents. The survey
needs to determine how much spam the respondents are seeing, and what
impact it has on their network.
The purpose of any survey should be to collect data which is
objective, and requires no subjective judgement on anyone's part.
This will make the survey data much more useful, and will give us more
confidence in the results of any subsequent data analysis.
There is NO need for any "control" in such a survey. We will be
asking for statistics about the current traffic on peoples networks.
We will NOT be asking people to compare their networks to others. We
will NOT be asking about people's opinions about their network
traffic.
As I see it, these measurements are in part, useful as a analysis of the
advantage or disadvantage of different types of anti-spam measures.
I disagree.
We have no way of knowing the effectiveness of anti-spam measures
used by the respondents. There is no question we can ask about that
effectiveness which will give us a meaningfull answer. The
respondents will be using many different anti-spam methods, some home
brewed. This makes it nearly impossible to determine what is meant by
"anti-spam measure". We will therefore NOT know what the data means,
so the data will be useless, and any analysis will be a waste of time.
In contrast, we can ask the simple question: "How many emails did
you decide were spam?"
The answer to such a question is objectivem and it's an easy
measurement to make.
In addition, a survey with the goal of measuring the effectiveness
of anti-spam solutions will be socially contentious. (Or more so...)
Since just *having* anti-spam measures is a way to deter spam, I think that
showing these numbers is important.
How many spammers give up spamming a host just because they have
anti-spam... and why?
How do you expect to be able to tell?
Any other suggestions on how we can uniquely analyze the data?
Let's not talk about data analysis until such time as we're sure we
can gather meaningful data.
Alan DeKok.
_______________________________________________
Asrg mailing list
Asrg(_at_)ietf(_dot_)org
https://www1.ietf.org/mailman/listinfo/asrg