ietf
[Top] [All Lists]

Re: document requirements and IETF procedures (was: Re: polishing of draft-ietf-dnsext-rfc3597-bis-01)

2010-02-19 19:38:35
Hi Alfred,

[Follow-up to the IETF discussion list as you suggested. The original message is from the namedroppers mailing list.]

At 06:02 19-02-10, Alfred Hönes wrote:
Regarding openness, I have more serious concerns.
(RFC-to-be's held off by IESG intransparently, etc.... )
But that is off-topic here.

I'm not sure where I last had read that phrase (flame?) above -- and it
essentially doesn't matter here --, I bet it was one of the procedural
drafts from Brian Carpenter in which he tried to overcome the road
blocks to revising RFC 2026 and adopting it to long standing practice.

I am tempted to quote a COMMENT posted yesterday by an AD, with
respect to a technical document in the IESG and the IETF procedures
followed in its (6 years lasting) genesis, and I challenge its more
general applicability to the meta level of IETF procedural standards
as well  (and so I've elided from the quote all keywords related to
the triggering circumstances):

   "[...]
    This would seem to imply that the .... WG has decided to
    deviate from the old IETF operating principle of "rough
    consensus and running code".  For at least some of the
    techniques described in this draft, they are generally
    accepted and widely implemented on key implementations.
    I ask what the reason is for divorcing IETF standards
    from established best practices and actual running code?
    ... RFCs are not sacred documents, they should reflect what
    we want our implementations to do.  But maybe there are
    important use cases for the actual standard ... behavior
    in this space, just that I don't know about them.  Please
    educate me about the background for this decision."

I am going to quote large extracts from RFC 3774.

  "The IETF is unsure what it is trying to achieve and hence cannot
   know what its optimum internal organization should be to achieve
   its aims."

  "Externally, the IETF is often classified with these conventional
   SDOs, especially by detractors, because the differentiation in the
   IETF's mission and processes and the rationale for those differences
   are not clear."

  "A major symptom of this lack is that WGs do not consistently produce
   timely, high-quality, and predictable output."

  "Failure to identify and articulate engineering trade-offs that may
   be needed to meet the deadlines that the WG has set without
   inappropriately reducing the 'fitness for purpose' for the
   intended customers."

  "Continued refinement of the solution beyond the point at which it
   is adequate to meet the requirements placed on it by the intended
   purpose."

We could call it mission creep.

  "Poorly defined success criteria for WGs and individual documents."

  "Lack of auditing against explicit criteria throughout the
   standards development process."

  "Lack of review, especially early review, by reviewers who are not
   directly interested members of the WG, and by subject matter
   experts for topics related to, but not necessarily the immediate
   focus of the document."

  "Lack of criteria for determining when a piece of work is
   overrunning and/or is unlikely to be concluded successfully,
   either at all or within an acceptable time frame."

  "One problem which the IETF does not appear to suffer from is
   excessive bureaucracy, in the sense that transfer of information is
   generally kept to the minimum necessary to accomplish the task.  It
   is important that any changes introduced do not significantly
   increase the bureaucratic load whilst still recording sufficient
   information to allow process improvement."

I note that the IESG provides a wealth of information nowadays through its minutes. The same cannot be said of Working Groups. The minima nowadays is XMPP logs.

  "One area of particular concern is the tendency for protocols to be
   assessed and issues resolved primarily through static analysis of the
   written specification rather than by practical experiment with
   'running code'."

It's one thing to read a specification. Implementing it provides a different view as we can come across use-cases we would never have thought about.

  "The current hierarchy of Proposed, Draft, and Full Standard maturity
   levels for specifications is no longer being used in the way that was
   envisioned when the stratification was originally proposed. In
   practice, the IETF currently has a one-step standards process that
   subverts the IETF's preference for demonstrating effectiveness
   through running code in multiple interoperable implementations."

I guess that it should be called ossification instead of stratification.

  "It is widely perceived that the IESG has 'raised the (quality)
   bar' that standards have to pass to be accorded a PS status."

That's what happens when the RFC brand is considered as more important.

  "There appears to be a vicious circle in operation where vendors
   tend to deploy protocols that have reached PS as if they were
   ready for full production, rather than accepting that standards at
   the PS level are still under development and could be expected to
   be altered after feature, performance, and interoperability tests
   in limited pilot installations, as was originally intended.  The
   enthusiasm of vendors to achieve a rapid time to market seems to
   have encouraged the IETF in general and the IESG in particular to
   attempt to ensure that specifications at PS are ready for prime
   time, and that subsequent modifications will be minimal as it
   progresses to DS and FS, assuming effort can be found to create
   the necessary applicability and interoperability reports that are
   needed."

Maybe by trying to achieve perfection at Proposed Standard, the IETF has involuntarily encouraged that. It is difficult to make changes after that as there are deployment considerations then.

  "The periodic review of protocols at PS and DS levels specified in
   are not being carried out, allowing protocols to persist in
   these lower maturity levels for extended periods of time, whereas
   the process would normally expect them to progress or be relegated
   to Historic status."

We accept long-lived experiments instead of putting in a timer. Experimental status is a facing saving device. Proposed Standard is the equivalent of "the IETF said so". We reinterpret the text as a holy book. Naturally, that leads to divergent interpretations. That gives a new meaning to protocol maturity.

  "the specification is not updated to reflect parts of the
   specification that have fallen into disuse or were, in fact, never
   implemented."

As the years go by people forget why the text was written in such a way.

  "Although there may be large attendance at many WG meetings, in
   many cases, 5% or less of the participants have read the drafts
   under discussion or that have a bearing on the decisions to be
   made."

  "Little or no response is seen when a request for 'last-call'
   review is issued, either at WG or IETF level."

Fortunately there is the GEN-Art review and reviews by directorates from specific areas or else the cross-area review would be in name only. Maybe consensus should be redefined as "lazy consensus".

 "The IESG members are overworked which is bad for their health,
  humor, and home life"

 "It appears that the IETF is showing a disturbing tendency to
  turn IESG 'rules of convenience' into rigid strictures that
  cannot be violated or deviated from."

 "Newcomers to the organization and others outside the affinity group
  are reluctant to challenge the apparent authority of the extended
  affinity group during debates and consequently influence remains
  concentrated in a relatively small group of people."

The old-boys (and girls) club is sometimes perceived as a subset of that affinity group.

  "Participants are frequently allowed to re-open previously closed
   issues just to replay parts of the previous discussion without
   introducing new material.  This may be either because the decision
   has not been clearly documented, or it may be a maneuver to try to
   get a decision changed because the participant did not concur with
   the consensus originally."

  "One cause that can lead to legitimate attempts to re-open an
   apparently closed issue is the occurrence of 'consensus by
   exhaustion'."

  "The IETF culture of openness also tends to tolerate participants who,
   whilst understanding the principles of the IETF, disagree with them
   and actively ignore them.  This can be confusing for newer
   participants, but they need to be made aware that the IETF does not
   exclude such people."

Regards,
-sm
_______________________________________________
Ietf mailing list
Ietf(_at_)ietf(_dot_)org
https://www.ietf.org/mailman/listinfo/ietf

<Prev in Thread] Current Thread [Next in Thread>
  • Re: document requirements and IETF procedures (was: Re: polishing of draft-ietf-dnsext-rfc3597-bis-01), SM <=