ietf
[Top] [All Lists]

Re: "Discuss" criteria

2006-12-30 00:32:08
Hi Dave,

Joel and Sam already sent responses that I agree with,
but I wanted to add a couple of things. First of all, most
of the things that the IESG looks for in its review are
judgment calls. Its our job to think about whether, for
instance, underspecification problems in the document
are so bad that it calls for a Discuss even when we know
that interoperable implementations exist. Such a
situation should be rare, and I think most of us value
implementation experience very highly in all stages
of the IETF process. But I remember at least one
case from this year where "look, my implementation works"
claims from the proponents of a protocol were countered
with "no, its broken" from others, and it turned out that
the latter people were right. In any case, if the things
that IESG looks for weren't judgment calls, I would be
calling the tools team to work on the Discuss tool
and have the IESG do something else...

Secondly, when one of us makes a Discuss, there
is significant pressure to keep it within the Discuss
criteria guidelines, make it actionable, and explain
it in a believable manner with justifications. The
first test is whether the rest of the IESG believes
in it. Brian, for instance, gives a lot of feedback on
the other AD's Discusses, and this results often
in changes or removal of the Discuss. To survive
a laugh test in the telechat a Discuss has to have
some substance. Of course at the end of the day
we may disagree on whether there is a problem,
but my experience is that Discusses that stay
have well explained justifications.

Also, you wrote:
Meta-point:

     Something quite basic that is missing from the draft on
     Discuss Criteria is a requirement that any Discuss not only
     explain its precise normative basis, but that it give a clear
     statement of what actions must be taken to clear the Discuss.
The Discusses have to be actionable in the sense
that author must know what specific problem there
is, and he needs to be able to fix it in this document.
But I hope you are not saying that the AD has to find
the fix...

Finally, I think we should just ship the discuss criteria
document. We tend to hold on to documents for too
long in the expectation that we might find an
improvement later. But I think the right documentation
format for this type of documents is IONs.

--Jari

Dave Crocker kirjoitti:
Dave is probably correct that the specific criteria are of broader
interest than just ADs, WG chairs, editors, and process wonks, and
might become even more perfect with broader review, but that's
another issue.

And, since the criteria are public, I'm sure the IESG would be
interested in feedback on the criteria, especially now that WGs and



Meta-point:

     Something quite basic that is missing from the draft on
     Discuss Criteria is a requirement that any Discuss not only
     explain its precise normative basis, but that it give a clear
     statement of what actions must be taken to clear the Discuss.


From the draft:

* The specification is impossible to implement due to technical or
clarity issues.

When this assertion is made, is it sufficient to cite existing
implementations
based on the current version of the specification?  Is the AD at least
required to explain the assertion in detail?


* The protocol has technical flaws that will prevent it from working
properly, or the description is unclear in such a way that the reader
cannot understand it without ambiguity.

Will demonstrations of interoperability be sufficient to counter this
claim?


* It is unlikely that multiple implementations of the specification
would interoperate, usually due to vagueness or incomplete
specification.

Will demonstrations of interoperability be sufficient to counter this
claim?


* Widespread deployment would be damaging to the Internet or an
enterprise network for reasons of congestion control, scalability, or
the like.

Beyond the simple assertion of this claim, what requirements are there
for the AD to substantiate it?


* The specification would create serious security holes, or the
described protocol has self-defeating security vulnerabilities (e.g.
a protocol that cannot fulfill its purpose without security
properties it does not provide).


* It would present serious operational issues in widespread
deployment, by for example neglecting network management or
configuration entirely.

There is often a failure to distinguish between new and peculiar
problems created by a particular specification, versus general
problems that already exist.

A classic example of this is citing basic DNS problems, for
specifications that are merely consumers of the DNS and, hence, are
not creating any new problems.


* Failure to conform with IAB architecture (e.g., RFC1958 (Carpenter,
B., “Architectural Principles of the Internet,” June 1996.) [2], or
UNSAF (Daigle, L., “IAB Considerations for UNilateral Self-Address
Fixing (UNSAF) Across Network Address Translation,” November 2002.)
[3]) in the absence of any satisfactory text explaining this
architectural decision.

This is an interesting item.

1.  At what point in time did publication of IAB preferences take on
the force of law?

2.  Note that the list gives some examples, but does not supply a
complete list. How are working groups to know which IAB documents have
been declared normative standards for all IETF work and which have not?

3.  Why is the IAB allowed to create normative standards that cover
all IETF work, without requiring that they first gain IETF-wide approval?


* The specification was not properly vetted against the I-D
Checklist. Symptoms include broken ABNF or XML, missing Security
Considerations, and so on.

* The draft omits a normative reference necessary for its
implementation, or cites such a reference merely informatively rather
than normatively.

* The document does not meet criteria for advancement in its
designated standards track, for example because it is a document
going to Full Standard that contains 'down references' to RFCs at a
lower position in the standards track, or a Standards Track document
that contains only informational guidance.

* IETF process related to document advancement was not carried out;
e.g., there are unresolved and substantive Last Call comments which
the document does not address, the document is outside the scope of
the charter of the WG which requested its publication, and so on.

* The IETF as a whole does not have consensus on the technical
approach or document. There are cases where individual working groups
or areas have forged rough consensus around a technical approach
which does not garner IETF
 consensus. An AD may DISCUSS a document where she or he believes
this to be the case. While the Area Director should describe the
technical area where consensus is flawed, the focus of the DISCUSS
and its resolution should be on
 how to forge a cross-IETF consensus.

This is perhaps the scariest of the criteria.  It says that a
knowledgeable, motivated constituency can spend months on solving a
problem that it needs to have solved, and then others who have not
participated in the work can come along and sabotage it.

Yes, I know that is not the intend of this criterion, but it is what
the effect will be -- and in some cases already is.

Once upon a time, the rule in the IETF was that working group
consensus was what mattered, absent a clear technical basis for saying
that a specification "would not work".

That is quite different from now saying that after a working group is
done we somehow must be assured that the specification has acquired an
IETF-wide politically correct acceptance.

If this last criterion is meant to be taken as it is stated, there is
a pretty straightforward basis for believing that ever bringing new
work to the IETF is a very questionable decision.

The criterion presumes that the IETF, as a whole, must agree on the
One True Solution to a problem.

In the IETF's history, that has been a requirement in some special
cases, but not others.  The default view has been to let the market
decide among choices. Requirement for a single choice has been
asserted only when there is a strong argument that having multiple
choices will cause damage.

d/


_______________________________________________
Ietf mailing list
Ietf(_at_)ietf(_dot_)org
https://www1.ietf.org/mailman/listinfo/ietf

<Prev in Thread] Current Thread [Next in Thread>