Responding to various people in one e-mail.
To summarise, we have procedures that say what kinds of Discusses are
appropriate, and personal engineering preferences are not. Legitimate issues
should be raised, however, and in the case of most big issues, the right
approach would be to send a document back to the working group for revision.
But overall, my perception is that the ADs take their task very seriously, and
try to do the right thing when performing their IESG reviews. I think they make
a significant contribution to the quality of our documents.
That being said, a couple of other things are also obvious.
First, we have a very tail-heavy process. I get a lot of comments about that,
and you all know it too. Moving more of the problem finding to earlier parts of
the process is necessary (but also not easy).
Secondly, I know from personal experience that feedback on what kinds of things
get raised in the IESG review is very useful. Even if we are trying to do the
right thing, we make mistake, and perhaps more importantly, our understanding
of what kinds of discusses are useful keeps evolving. Please tell us when you
think we are doing the wrong thing (be it not raising an issue or raising it in
the wrong category).
Finally, today too many big revisions are handled as a discussion between the
authors and ADs. Big discussions should be done publicly in the working group,
and in some cases the right action is for a document to be sent to the working
group so that it can be reprocesses and returned.
And then to the specific responses.
Fred:
In my opinion, some individual ADs seem to, from their behavior, feel that
they have not done their jobs unless they have raised a "discuss". The one
that took the cake for me personally was a "discuss" raised by a particular
AD (who shall remain nameless) that in essence wondered what he should raise
a "discuss" about in my document. There were a couple of errors in that; he
told me later that what he had done was opened the comment tool and typed
that question, and subsequently accidentally hit the equivalent of "send" -
the question wasn't intended to go out. But the question itself is telling:
the issue was not "does the document meet the requirements of BCP 9", it was
"what comment shall I raise"?
And this is of course completely against our rules in the Discuss criteria
document. But accidents do happen. If there were an intentional "I owe you"
Discuss, that is obviously something that should go away, and authors and other
ADs should complain about it.
Also, in my opinion, IESG review that raises a certain number of issues
should not result in the document sitting in the IESG's queue for a few
months while the authors go back and forth with the AD or the GEN-ART
reviewer pounding the document into someone's idea of "shape" without working
group involvement. I personally would prefer that simple matters get sorted
out between the ADs and the author, but complex changes or additional content
requested by the AD should result in the document being sent back to the
working group.
I'm not saying it doesn't serve a purpose. I'm saying that I know of drafts
that have been nearly rewritten during such back-and-forth, so what popped
out was largely unrelated to what went in. In such cases, I think the
document should have been returned to the working group with comments, not
worked on privately.
I agree 100% with this.
Adrian:
Examples are useful because they give the IESG something to chew on. If you
don't call us when we do "bad stuff" we might never know.
Indeed. For what it is worth, my own idea of what kinds of things I should as
an AD complain about has evolved quite a bit. As a new AD I was overly eager,
ready to defend the Internet against… bad documents. In some cases filing
Discusses that in retrospect I probably should have filed as Comments. Over the
years I've come to the realisation that most things are additional Comments for
the authors to take into account, but that real brokenness should still cause a
Discuss to be raised. But for such development to occur, you need to gain some
perspective. Part of that process is getting feedback.
I believe that the clue is in the word "Discuss" and I hope that I (at least)
am
willing to listen when the response is "we thought about it, it's not a big
problem."
+1
Brian:
Seeing randomly selected drafts as a Gen-ART reviewer, I can
say that serious defects quite often survive WG review and
sometimes survive IETF Last Call review, so the final review
by the IESG does serve a purpose.
This is one of the big issues. I think we all agree that our process is
tail-heavy. More review and more problem discoveries occur at the end. This is
trivial to see, but the crux is how that could be changed.
Dave:
the tendency of ADs to inject spontaneous, late-stage personal engineering
preferences, which might have been reasonable to add to the original working
group discussion but realistically have no place in a final review
And personal engineering preferences are against our rules for Discusses. But
beauty is in the eye of the beholder, and so are bugs :-) Is a particular spec
bug an legitimate, serious technical problem, otherwise somehow against agreed
IETF consensus, or is it an engineering preference?
But what is tiring about this line of justification is its continuing failure
to balance the analysis by looking at the costs and problems that come with
it. Does it provide regular and sufficient benefit to justify its
considerable costs?
Agreed.
Jari