Re: call for ideas: tail-heavy IETF process

On 05/15/2013 09:07 PM, Pete Resnick wrote:
> I initially replied to address Keith's comment. But a few things on 
> Joe's:
> On 5/15/13 7:41 AM, Keith Moore wrote:
> > On 05/15/2013 10:39 AM, Joe Touch wrote:
> > > On 5/14/2013 9:54 PM, Keith Moore wrote:
> > > > Publishing broken or unclear documents is not progress.
> > > Broken, agreed.
> > >
> > > Unclear, nope - please review the NON-DISCUSS criteria, notably:
> > >
> > > The motivation for a particular feature of a protocol is not clear 
> > > enough. At the IESG review stage, protocols should not be blocked 
> > > because they provide capabilities beyond what seems necessary to 
> > > acquit their responsibilities.
> > > The DISCUSS isn't there to make documents "better" - that's for 
> > > COMMENTs.
> Exactly right. Sometimes we forget; it's a good thing to remind us.
>
> > > A DISCUSS there to catch a set of problems and to *block* the 
> > > document's progress until that problem is resolved.
> Mostly correct. However:
>
> - If there is only one AD who wishes to DISCUSS and no other AD agrees 
> with the DISCUSS holder, at the next telechat the document is 
> unblocked. (See <http://www.ietf.org/iesg/voting-procedures.html>.)
> - Even if others agree with the DISCUSS, the chair can be asked to use 
> the alternate procedure, which requires 2/3 of non-recused ADs to 
> agree to publication.
> Which is to say, if there is only a single AD "blocking" a document, 
> that block is essentially a 2 week affair if you are willing to push 
> the point. No need for negotiating; if the WG decides that the AD is 
> totally off base, tell your sponsoring AD that you're waiting the two 
> weeks. People (unfortunately IMO) don't push the point nearly enough.
I think it's very unfortunate that IESG has adopted rules that work this 
way.   Part of IESG's job is to provide independent review of WG 
output.   It that review can be circumvented merely by waiting two 
weeks, that's a bug in the process.   And if an AD raises a DISCUSS 
about a matter of technical or document quality (or for that matter, 
about a process violation), and the WG isn't even willing to discuss the 
point, but instead relies on the two week timeout, I think that's 
grounds for appeal to the IAB.
> (For the record, the IESG has *never* used the latter of the two 
> procedures.)
> That said, I am also of the view that there is a third way, but I have 
> never seen a WG attempt it:
> DISCUSS should in fact require discussing. 
We agree on that much.
> Assuming there was some good faith effort on the part of the WG to 
> figure out what the AD was on about and they really assess that the AD 
> has it wrong, a WG could say, "Sorry, we got this right. You are 
> confused (or wrong). We are not changing the document. We are done 
> discussing." At that point, I am of the opinion that the AD cannot 
> hold the DISCUSS any longer. The AD must move to ABSTAIN. 
I disagree with this in the strongest possible terms.  I believe that 
for IESG to have rules that insist on or encourage voting ABSTAIN when 
the AD really means "this document is not acceptable", is both 
irresponsible and dishonest on the IESG's part.  I also believe that it 
tends to mean that ADs who didn't actually review the document get more 
say than ADs who did.   The proper thing to do in that case is for 
either side to appeal to the IAB.
> The discussion is, for all intents and purposes, over and to continue 
> to DISCUSS is (IMO) an appealable offense. Then it is a matter of the 
> IESG deciding whether there are enough ADs supporting the document 
> (YES or NO OBJECTION) to count as consensus of the IESG. We ostensibly 
> use 2/3 of non-recused ADs to mean "consensus", since (I think the 
> theory goes) if you can't get 2/3 of ADs to agree that it's OK to 
> publish a document, that is a sign of a lack of rough consensus in the 
> IETF (and probably a serious problem in WG operation). Indeed, if the 
> ADs are so off of their rockers that more than 1/3 of them are against 
> a perfectly reasonable document, it's time for the appeals and recall 
> procedures to be used.
I don't think IESG voting should be thought of as a consensus process, 
except perhaps in the deadlock breaking procedure.   The typical number 
of reviewers on the IESG is too small for a consensus process to be 
meaningful.   With so few thorough reviewers outside of the WG, you need 
a procedure that at least initially assumes that all review comments 
have to be taken seriously.
> Now, as to Keith's comments:
>
> > I strongly disagree with what the NON-DISCUSS criteria say. DISCUSS 
> > isn't just for blocking documents.   And document quality is as 
> > important (in the sense that poor document quality can lead to as 
> > many interoperability or other problems) as technical correctness.
> > Why are people trying to sabotage IESG?
> I'm sorry Keith, but your last line is rubbish. To claim that what 
> people in this thread are talking about amounts to an attempt to 
> sabotage the IESG is the height of hubris. 
"sabotage" is probably not the best word I could have chosen.   But I do 
have the strong impression that people are clamoring for IESG reviews to 
be toothless, and that frustrates me.   Every time I attend an IETF 
meeting and look at what a broad spectrum of WGs are doing, I find that 
most of the ones I see are producing output of very dubious quality.   
While trying to fix this after WGLC almost never works well, the threat 
of IESG rejection of WG documents is one of the few incentives that WGs 
have to actually consider interests outside of their narrow areas.   
External review after a WG has completed its work is absolutely 
essential.   And in our current process, the only even marginally 
effective mechanism that we have for that is IESG review.
I understand that ADs can sometimes be capricious and sometimes simply 
be wrong, and I'm supportive of the idea that there should be some way 
to keep a single AD from holding things up.   But expecting ADs to 
change their reviews just because others don't agree with them is 
completely inappropriate.   So is letting people who didn't thoroughly 
reviewed a document have more sway in the review process than people who 
have reviewed it.
> In my opinion, the IESG has 2 jobs as far as document review goes: (1) 
> Check for serious technical problems that the WG might not have 
> considered and make sure they get considered; and (2) Make sure that 
> our processes have been followed to assure that the WG products are 
> truly the product of (IETF-wide) consensus and fit within the policies 
> of the IETF. I am *positive* that I have failed to limit my DISCUSSes 
> on documents to one of those two reasons; I expect I will err and do 
> so in the future. I should be corrected. We reject kings, and should.
I agree that we should reject kings.   But we should also reject 
stubborn working groups and stubborn authors.   We shouldn't presume 
that a WG knows better than the IESG.  The two have different 
points-of-view, but neither is inherently superior to the other.

> As for the rest: The IESG should be looking for technical correctness 
> problems and identify those that do not appear to have been properly 
> considered by the WG, but trying to police document quality beyond 
> that, including all of those listed in the NON-DISCUSS criteria, 
> really are out of bounds for the IESG to be concerning themselves 
> with. Sure, there are issues of clarity that can result in 
> interoperability problems, but every one of those I have seen is 
> because the words written can be read in a technically incorrect way. 
I disagree with this also.   Ambiguities in technical specifications 
that could affect interoperability need to be corrected no matter who 
finds them.   But I've rarely seen significant delays result from fixing 
minor ambiguities here and there.   The bigger delays come from the 
occasional documents that are so poorly written that small changes can't 
clarify them.    Those documents, frankly, need to be rewritten.   
(Perhaps the best thing that IESG could do in such cases is approve such 
documents as Experimental and commission a rewrite with a competent 
technical writer as co-author.)
>  Our documents are never going to be perfect and attempts to make them 
> so actually lessen the quality of our output over the long term.
I think I understand what you mean, and probably agree.   But the 
problem with stating things in that way is that it can be taken to mean 
that we should just write things as quickly and sloppily as possible.
Keith