[Top] [All Lists]

Re: Confirming vs. second-guessing

2008-03-16 19:54:33

Michael StJohns wrote:
Is it the job of a reviewing body to pre-empt lengthy and diligent work or
 is it the job of a reviewing body to the work was done diligently and

I think you're missing a "decide if" before "the work" in the second line?

Yeah.  See.  We can start with something we agree on.  But then, agreeing on my 
having missed something is no challenge.

I think this is kind of a slanted (sorry) statement of the problem. I'd put
it more like:

"Is it the job of the reviewing body to make an independent decision on the
candidates suitability, or is it the only job of the reviewing body to
protect the process irrespective of the actual nominations?"

Glossing over the linguistic subtleties, I think that the recent reality behind 
"make an independent decision" is that it is functionally identical to 
"second-guess".  My choice of language is certainly biased, but then I was (and 
am) trying to emphasize the impact on the entire process.

People who do line work are undermined if their work is constantly subject to 
reversal, unless the work is legitimately flawed.  It's not the job of the 
review process to replicate the work, but to confirm it's quality.  There is a 

Perhaps by "independent" you meant that it must have substance rather than be 
automatic approval.  If so, we certainly agree.

Where we differ is about methods.

These are very different jobs.

Whether Nomcom or a working group, a decision process over a long period of
time represents extensive research, deliberation, and balancing among
trade-offs. This is something that simply cannot be replicated by another
person or body spending a few days or even weeks on "review".

The Nomcom has to winnow through a pile of candidates, discussion, gathering
information, discarding and ultimately selecting the one person (or for IAB
group of persons) that it is recommending for selection. That takes lots of
time and effort.

Taking the information which applies only to those candidates, reviewing it,
and making a decision, hopefully takes less time given the appropriate

Sampling error.  Experimenter bias.  Something like that...

To pursue your model, the review committee is given a subset of the source data 
and none of the history involved in doing the winnowing, and is somehow 
to be able to perform as well or better than the primary group?

These sorts of decision processes are dominated by trade-offs.  The review body 
has none of that protracted context that is leads to the decision among the 
trade-offs. Nor can that context be replicated easily or even very well.  
why a review has to have more to do with verifying that the process of the 
primary group was diligent. You don't get that by looking over source data.

I do add that a review body might believe it holds special knowledge -- that 
source data -- that it deems important, and it well might use that in its 
dialogue with the primary group.  This is more than mere process review and I, 
for one, think it entirely appropriate. As a basis for dialogue.

In the case of Nomcom work, it's pretty rare, however, for that dialogue to 
result in a reversal, since Nomcoms really are typically quite diligent -- 
and/or the review body does not focus on the right basis for reversal...

(I think "quite rare" translates to "never" but of course we'll never know.)

Put another way, the Nomcom is a search committee, but the hiring authority
resides in the confirming bodies.

Yes, that is the myth.  And that's why it is labelled "Nominating" Committee. 
But it really is only a myth, since the repercussions of an outright rejection 
are pretty onerous.

Even if we skip over my potentially unpleasant assessment of reality, I'll 
repeat my above observation about the effect of undermining folks who do 

If they are not replicating the decision process, they are doing something

The rest of this message is sort of ignoring the whole "winnowing" process
done by the Nomcom.

I was certainly not intending to and am pretty sure I didn't.  It's all about 
that protracted context.  That's more than simply forwarding a nomination.  
about juggling trade-offs.

What I suspect does tend to get ignored in these kinds of discussions, is how 
the IETF is different from the places we work as a day job:

    In the IETF the people who are in charge are not really in charge.  All the 
approval stuff distracts from the reality that work is done by people who 
volunteer and those people are not "hired" by the decision-makers.

    Yeah, those decision-makers name people to particular jobs, and those jobs 
are important and difficult.  But all of this hinges on a community 
constituency's deciding to work on the topic. The decision-makers come in to 
play only after that community constituency has developed.

    It is an inherently grass-roots model, quite the opposite of "working for" 
an organization.

    The other thing that is missed is that the Nomination process does not 
forward multiple candidates for a position.  It forwards one.

    This is entirely different from the hiring activities most of us are used 
to.  Again, it should serve as a very bright flag that this process is quite 
different from the model we see in companies.

Since I mostly don't agree with the premise the reviewing bodies are
"repeating" the Nomcom's job if they consider candidates qualifications, I
don't really have comments on the rest of the message.

Offhand, I don't think that there is a problem for the review body to "consider 
candidate's qualifications".  It is a question of what material is reasonable 
for them to use to make that assessment.

As long as we have no consensus about the nature of the job to be done by a
 reviewing body, we are going to suffer with its thinking can can
reasonably second-guess primary bodies.

And on this we agree.

Begin and end in agreement.  This is goodness.



   Dave Crocker
   Brandenburg InternetWorking
IETF mailing list

<Prev in Thread] Current Thread [Next in Thread>