Re: Outstanding reviewers

hi SM, all,

In my previous message on the subject
(http://www.ietf.org/mail-archive/web/ietf/current/msg83204.html) I'd
considered using the datatracker to improve the "assignment" or at least
tracking the of coverage of reviews of documents. I'd thought of this
primarily as a method to reduce final review workload (in the context of
CHANGING THE JOB), but this would be another very good reason to add
reviewer information to the datatracker.

More inline.

SM wrote:
> At 15:52 17-10-2013, John C Klensin wrote:
> > I had promised I was through for today, but I just had a wild,
> > crazy, suggestion that follows on the retirement of the xx99 and
> > xx00 subseries of RFCs.
> I missed the above the first time I read the message.  Fortunately, I
> read the message again and I noticed "wild, crazy". :-)
>
> RFC authors get credit in the IETF system.  Reviewers don't get much
> credit in the IETF system even though the system would not work without
> reviewers.  The American Physical Society has an Outstanding Referee
> program to express its appreciation to reviewers.  The reviewers are
> selected based on the quality, number, and timeliness of their reviews.
>
> There was a comment from Loa Andersson: the public has a "right to know"
> who reviewed a document. 
This is a very good point.

> There are times when I wondered who let a
> draft through as I was not comfortable asking a person to review a draft
> if there hasn't been any effort to review it previously.  Some of that
> information may be available in the datatracker but it is freeform
> text. 
This is a problem with any approach to review tracking using the
datatracker -- if it's not done consistently nobody will come to rely on
it. More automated tool support would help with consistency.

> Some changes would have to be done for the document shepherd to
> be able to add who reviewed a draft and a link to where the review was
> posted.
A first pass at this would simply note reviewer, date, document
revision, and sections covered, along with the content of the review
(which, if incoming references to mailing list archives are considered
to be stable, which they should, could in most cases simply be a URL).
That would be sufficient for identifying reviewers, and would probably
do a lot to support triage of well-reviewed from poorly-reviewed
documents before they get sent up to the IESG.

> It may be possible to use the data (see previous paragraph) to identify
> outstanding reviewers by area. 
Yep.

> The IETF can express its appreciation by
> publishing a yearly list or an xx99 list (see comment from John Klensin).
Regardless of how the recognition gets disseminated, there's a lot of
value in starting to collect the data.

Cheers,

Brian

signature.asc
Description: OpenPGP digital signature