On 12/23/2015 09:21 PM, Dave Crocker wrote:
Anecdotal. Mine. Over enough years to represent a pattern. (I'm not
alone in this, but I'm reporting my own experience) In 25 years, not one
single RFC I've worked on had a serious problem caught by an AD, though
many were eventually discovered to have serious problems
When I was an AD, I caught several serious problems with drafts after WG
or IETF Last Call.
The problem I saw wasn't the inability to catch things so much as the
inability to get them fixed. By the time a draft got past WG Last
Call, it was generally so exhausted that it could not agree on how to
make any significant change to a document, even in response to a
legitimate and serious concern. In many cases, if there was a simple
fix, it was quickly and happily implemented. But in far too many cases
where there was a serious problem, the most that an IESG member could
hope for was to get some weasel words inserted in the text or prepend an
IESG note - neither of which really addressed the problem.
It was entirely too much work to do all of that review. After being on
IESG for four years, it was about ten years before I could stomach a
detailed review of any lengthy technical document. While on IESG I
tried getting others to do reviews for me, but I found that I still had
to read the documents under review to make sense of the reviews I was
getting. If I had to do that again today, I'd probably try to give the
reviewers more guidance.
But mostly I think that WGs need to have their work subject to formal
external review much earlier in the process than Last Call, particularly
when the WG's work has the potential to impact other WGs or other areas.
Keith