-----Original Message-----
From: Yakov Shafranovich [mailto:research(_at_)solidmatrix(_dot_)com]
Sent: Wednesday, July 23, 2003 12:47 PM
To: asrg(_at_)ietf(_dot_)org
Subject: [Asrg] 0. General - Dealing with proposals
In light of the fact that the requirements documents, the evaluation
framework and the consent model are all works in progress, I
think that we
need an interim policy or framework on how to deal with new
proposals. The
research agenda
(http://www.irtf.org/asrg/asrg_research_agenda> .htm) states
that " Some proposals need to be revisited
with regard to the requirements,
the evaluation model, and the consent framework. A template
for proposals
would also be useful."
Are the documents mature enough at this point that they can
be used for
evaluating proposals?
I would like to see another turn on the requirements doc first. I think that
there was some feedback that has not made it in yet. The evaluation
discussion in Crocker's draft is mature enough to guide people's thinking as
they propose approaches. I do not know if it provides a view that is
quantitative enough to allow comparison between systems. At the same time,
while we want to require authors of proposals to go through this thought
process, we have to be careful that the evaluation framework is not so
heavyweight to dissuade folks making proposals. I do not think that the
consent framework has matured to a point of clarity for many folks at this
point.
Also, proposals seem to cause the most
problems on
the list and tend to start flame attacks more than any other
topics. This
just re-enforces the need for some kind of policy on how
proposals should
be dealt with.
Would someone like to volunteer to write up such policy
and/or a template
for proposals as well?
I think that the first step is to require an evaluation according to a
determined model in part to encourage thoughtfulness. From there, we are
able to have a productive conversation analyzing the proposal. From an email
I sent to list about a month ago:
"As a solution is proposed, it should state the classes or types of unwanted
messages that it will be effective against. With proper measurement and
characterization work, we should be able to understand what percentage of
the current spam volume that describes. Additionally, the system must state
the assumptions upon which it is based. This allows analysis of the
robustness of the system in the face of countermeasures that may reverse
these assumptions."
I think that this needs to be part of an evaluation model. So perhaps on the
way to creating a template, the first step is to build upon this thinking
and the evaluation discussion in Crocker's draft to create a separate spam
technology evaluation model document as mentioned as item 8.a. on the ASRG
research agenda. Again, volunteers?
_______________________________________________
Asrg mailing list
Asrg(_at_)ietf(_dot_)org
https://www1.ietf.org/mailman/listinfo/asrg