One small addition...
--On Monday, December 21, 2015 16:39 -0800 Dave Crocker
<dhc(_at_)dcrocker(_dot_)net> wrote:
3. We have no discipline to reviews and no documentation
to give careful guidance.
Reviews are a random walk of well-intentioned people
who have varying skills, varying attention-spans, etc. The
results are, therefore, random.
Perhaps worse, some of the area reviews are not random but are
from people who, when they are engaging in full disclosure,
admit that they didn't understand the spec and therefore
reviewed what they did understand and maybe engaged in a bit of
nit-picking to show that they really tried to read it. I've
occasionally seen almost exactly the same behavior from a few
ADs. Maybe more than "occasionally" depending on how critically
one evaluates the reviews.
Beyond a certain point, those types of reviews make us feel
good, but may make a negative contribution to document quality
by convincing others, who might have less time (or not draw the
short straw) and/or better expertise and insight to conclude
that they don't need to read the document because lots of others
are doing so.
In other words, it is merely a certainty that serious problems
will slip through. Every review is good to do, but no
specific review is the savior of us all. (The ultimate
fallacy of a savior model is requiring expenditure of scarce,
strategic area director resources on late-stage reviews...
Expensive resources, frustrating delays, minimal benefits. Who
wouldn't want all that?)
Exactly.
The main benefit of a cross-area review is that it is a cold
reading by someone with no history with the effort. Fresh
eyes.
I would have said "no history with the effort and at least some
substantive knowledge and interest in the subject matter". "I
don't have a clue but was designated as the Foobar Area
Reviewer" reviews are usually not very helpful (although
sometimes we get lucky).
...
We could move that requirement to Proposed, but that would
merely make Proposed essentially the same as Full. And the
barrier to Proposed is already too high.
Instead we should understand that we cannot and should not try
to demand or expect documents that are perfect. We should
demand 'good enough' and let the outside world evaluate and
feed the results back to us.
I'm not certain we are getting to "good enough" quite often
enough. Moreover, what was "good enough" when the expectation
was that people would not deploy Proposed Standards in products,
at least without understanding that was a risk and treating it
as such, may not be "good enough" when Proposed Standards are
not only deployed but the community's attitude seems to be that
a piece of specification that turns out to be a bad idea has to
be truly catastrophic to modify the spec in an incompatible way,
...
The problem is our mythology before that and the misguided
expectations and barriers created as a result.
Very likely... at least a major part of the problem.
john