Re: "straightforward, reasonable, and fair"

> Keith - thanks for the pointer to "Harrison Bergeron".   
> Coincidentally, I was trying to recall this story in a conversation  
> recently and had
> forgotten the details and the author...
>
> But, I don't see how it applies here.  I'm not claiming "Nobody was  
> smarter than anybody else."  Yakov explained it better than I have:  
> "for each AD there is more than one person in the IETF who is more  
> technically astute than that AD. So, why should the IETF decision
> process favor opinion of such AD more than the opinion of these  
> other individual who are more astute that the AD ?"
Ralph,

While for each AD there may be more than one person in the IETF who  
is technically astute than that AD, it is difficult to reliably  
identify those people, and those people will typically _not_ be the  
other individuals who are reviewing a document in Last Call.   
Technically astute people who are able to review a significant number  
of documents probably end up on IESG sooner or later anyway -  
presuming that they have the other skills and attributes that make  
for good ADs.  The other thing is that given a set of people who are  
technically astute in a general sense, the ones who specialize in a  
particular activity probably end up being better at that activity  
than their peers.  So for instance the ones who specialize in  
reviewing large numbers of documents and trying to ensure some  
consistency between those documents are likely to be better at that  
skill than the other IETF participants.  It is not expected that ADs  
know more about each WG's topic than the WG.  But WGs sometimes have  
trouble seeing the forest for the trees.  It's part of the AD's job  
to examine things from a broader perspective.
There is no way to have meaningful external review of WG output  
without somehow "favoring" the opinion of those doing the review.   
And in order for such review to maintain some consistency between  
documents, there need to be a relatively small number of selected  
reviewers.  Peer review cannot work as well, both because the average  
clue level of a larger number of reviewers will inherently be lower,  
and because peers will not be as adept at looking at the document  
from a big picture perspective.
One inherent consequence of this for IETF is that it is possible for  
a single AD to have a large say in whether a document is found  
acceptable.  This is both good and bad.  Often a single AD will spot  
a significant problem that nobody else has seen.   Sometimes that  
problem is a subtle problem that others in IESG and/or the WG don't  
immediately accept or understand.  The other side of this coin is  
that when a single AD gives a document a good review, other ADs are  
less likely to state an objection to that document.   Giving a single  
AD a large say often has the effect of speeding a document's approval.
In other cases an AD will act capriciously or out of unreasonable  
prejudice, and there are sometimes accusations that an AD acted out  
of malice.  IETF and IESG processes try to provide multiple remedies  
to those situations.   It may be that the processes need to be  
tweaked or that other remedies are needed, and we should constantly  
be looking for ways to make our processes better.  But I don't see  
how to dispense with either external review or giving a single  
reviewer's opinion a lot of weight, at least on an initial review,  
without drastically reducing the quality and/or volume of our output.
Thinking about the disagreements between ADs and WGs that I've seen,  
they generally fall into two (overlapping) categories:
1. There is a legitimate difference of opinion about how to reconcile  
the conflict between the interests of the community of users that the  
WG hopes to serve, and some other community of users.  In this case I  
think "favoring" IESG is the right thing to do, because WGs  
frequently produce results that will cause problems for other  
communities - though naturally both IESG and the WG should attempt to  
work out a compromise that serves both sets of interests while  
minimizing the conflict between them.
2. The AD has a legitimate concern, but the change suggested by the  
AD is either (a) lame or (b) worse than the problem the AD wishes to  
remedy.  Part of this problem is that in practice the AD has  
relatively little ability to fix a document.  No matter how bad the  
document is, the most an AD can generally manage to do is to insist  
on small changes to the text.  Yes, in theory the AD could insist  
that the document undergo significant revision, but the pressure from  
both inside and outside the IESG to move even bad documents along is  
considerable.  [*].  Sometimes there's no way to fix a document with  
small textual changes.  But because of the pressure to either approve  
a document or to ask for fairly small changes to the document text,  
ADs sometimes suggest small changes to the text that end up being  
poor fixes.
The best cure for both of these problems, I suspect, is not to give  
ADs less "favor" in reviewing documents, but to identify such  
problems earlier, at a time when there is less investment in the WG's  
output and less sense of urgency to get the document out the door.
The connection with the Vonnegut story is this: an insistence on  
making things "fair" has the effect of discouraging excellence.
Keith

[*] Bad documents take more time to review than good documents --  
both because they tend to be poorly written and because the AD is  
struggling to think of a simple fix while reading the document -- and  
nobody on IESG wants to review a bad document over and over.

_______________________________________________
Ietf mailing list
Ietf(_at_)ietf(_dot_)org
https://www1.ietf.org/mailman/listinfo/ietf