[Top] [All Lists]

Confirming vs. second-guessing

2008-03-16 16:19:30

unsure how the confirming body confirms the candidate without also being
apprised of this information.  

This seems to go to the heart of a long-standing dilemma in the IETF:

    Is it the job of a reviewing body to pre-empt lengthy and diligent work or 
is it the job of a reviewing body to the work was done diligently and 

These are very different jobs.

Whether Nomcom or a working group, a decision process over a long period of 
represents extensive research, deliberation, and balancing among trade-offs. 
This is something that simply cannot be replicated by another person or body 
spending a few days or even weeks on "review".

If they are not replicating the decision process, they are doing something else.

What should it be?

If a reviewing body that is doing no original research and is working over a 
very brief period can legitimately second-guess the protracted effort of a 
working group or a Nomcom, then why bother with the first group?  It would be 
dramatically easier and faster to simply have the "reviewing" bodies make all 
the choices in the first place.

The necessary difference, here, is between reviewing that a primary body did 
work competently, versus re-doing the underlying work.

Certainly we want the review process to be substantive.  That is, meaningful. 
So what questions should it be asking?  It's easy to go through some pro forma 
material to establish that the primary group really was diligent.  Fine.  But 
what about going to deeper issues?

Perhaps a review of the primary group's competence requires going to all the 
source material?  Why?  that really reduces the primary group to merely 
assembling source data.  Is that what we want?  Probably not.

What if the reviewing body believes it has essential data that might have been 
missed or considered inadequately?  Now we essentially go to the possibility 
that the reviewing body really does its own source of research information or 
insight to it.  After all, the reviewing body comprises knowledgeable folk.

It seems to me that that extra insight should serve to ask the primary group to 
explain how it considered the special issues.  In fact, that kind of dialogue 
can be a very good way to establish the substance of the primary group's 

Primary groups tend to be defensive about this dialogue and reviewing bodies 
tend to be indelicate and/or vague when asking these questions.  And the 
exchange adds delay, but I think that these are the price of having meaningful 
review. Just as the primary group's effort needs to be respected, so does the 
reviewing body's.

As long as we have no consensus about the nature of the job to be done by a 
reviewing body, we are going to suffer with its thinking can can reasonably 
second-guess primary bodies.


   Dave Crocker
   Brandenburg InternetWorking
IETF mailing list