Re: Content-free Last Call comments

After reading some of the criticisms, I wonder if folks who think 
they've been disagreeing with me are going to get to the end of this 
message and say, "Oh, if that's all he's on about, who cares?" But *I* 
of course think there is an important issue in here. Anyway, back into 
the breach. David's message seems like a good launching point.
On 6/12/13 3:43 AM, Dave Cridland wrote:

> The IETF consensus on the charter is handled by Apathy Is Assent 
> rules, so claiming that this consensus call becomes the default is an 
> interesting argument to make.
Partially tangential: We in the IETF use the phrase "consensus call" in, 
AFAICT, a unique (and IMO goofy) way. It implies that first we have a 
discussion, and then at the end of the discussion we engage in a short 
act we refer to as a "consensus call", the completion of which *creates* 
the consensus. That isn't a consensus call. That's just an anonymous and 
imprecisely counted vote. I think I've got a new section for my 
consensus document to work on: "Consensus isn't the destination; it's 
the journey." (I should get a job writing text for fortune cookies.)
But back to the topic at hand:

> I strongly feel that positive statements have value, as they allow the 
> community to gauge the level of review and consensus...
Reviews are good. We need to do them. They are of value. They should be 
encouraged. People should post the results of reviews. They should brag 
about them as they see fit. No argument there.
Public statements of negative reviews (with specific issues identified) 
are really important. If issues are identified that then go unanswered, 
that breaks consensus.
Public statements (even short ones) of positive reviews have value in 
that we want to know that reviews have taken place. That's useful. Mind 
you, such statements don't help you "gauge consensus": Sure, you *might* 
conclude that one more person is part of the consensus, but of course 
the number of people doesn't matter to the consensus, and the consensus 
hasn't moved at all -- you haven't resolved a new open issue. (There is 
a separate question about what to do when documents haven't gotten 
serious review prior to IETF Last Call and what that means for 
consensus. More on that in a moment.)
So, none of the above things get me torqued. Reviews: Good. Public 
statements of reviews: Good. Issues to be resolved: Good.
Here's what gets me torqued, that I think is useless, and starts people 
thinking that they are voting:
"I support publication of this document."
"I do not support publication of this document."

Even prefacing those with "I reviewed the document" doesn't help. (Of 
course the "I reviewed the document" part itself is potentially useful 
if the person bothers to tell us the result of the review, like "it's 
technically sound" or "it solves a problem that needs solving" or "it's 
technically crappy *and* here are the reasons why".) But the "I do/don't 
support this document" part is what gets my goat. It contains the 
apparent implication that someone is counting noses and that "support" 
matters. It shouldn't. What matters is that we as a community have 
worked this thing out, that it solves a problem that we have identified 
is real, and that nobody in the community has found serious fault in it.
It's not the shortness of the statement that's the problem (though 
sometimes, especially with negative reviews, it is a problem); it's 
treating the call for consensus as a vote that bugs me. It's not useful 
to simply "support" a document.
> If, on the other hand, only objections are sought, then the text 
> (which simply asks for "comments") also needs changing. And the 
> GenArt, AppsDir, and SecDir reviews should only be send when they have 
> objections to publication, of course.
Nope, I'm good with all that. In fact, the GenArt review says exactly 
what it should: It provides "generalist reviews for the General Area 
director (currently the IETF Chair), providing an additional set of eyes 
for documents as they are being considered for publication." It's not 
looking to "gauge consensus". It's looking for a review.
That said, I do believe that objections are *way* more important than 
comments saying, "It's fine". Once something has gone through a WG, the 
presumption is "It's fine." Additional eyes are good, but they're good 
precisely because they might find problems.
That brings us to one last thing which I mentioned above: What if 
documents *aren't* getting serious review prior to IETF Last Call? This 
goes back to our "tail heavy" discussion. The IETF used to be very good 
about getting lots of review (sometimes even cross-area review) *during* 
the consensus building process in the WG. We now have many instances 
where WGs are not really working toward consensus to solve a problem; 
they're doing reviews of a single person's idea of how to solve a 
problem. Even worse, we are seeing many individual submissions that are 
not the product of consensus building at all, and when that 4-week Last 
Call kicks in, we are at best getting reviews, but not building 
consensus around the need for the document. Solving that problem by 
tail-loading more reviews does not address the real issue.
pr

--
Pete Resnick<http://www.qualcomm.com/~presnick/>
Qualcomm Technologies, Inc. - +1 (858)651-4478