ietf
[Top] [All Lists]

Re: Last Call: <draft-housley-two-maturity-levels-06.txt> (Reducing the Standards Track to Two Maturity Levels) to BCP

2011-05-31 11:24:47


--On Thursday, May 05, 2011 09:13 -0700 The IESG
<iesg-secretary(_at_)ietf(_dot_)org> wrote:


The IESG has received a request from an individual submitter
to consider the following document:
- 'Reducing the Standards Track to Two Maturity Levels'
  <draft-housley-two-maturity-levels-06.txt> as a BCP

The IESG plans to make a decision in the next few weeks, and
solicits final comments on this action. Please send
substantive comments to the ietf(_at_)ietf(_dot_)org mailing lists by
2011-06-02. 
...

Hi.

I made several comments about earlier versions of this document
then stopped because, while the incremental versions have all
included small improvements, they have addressed few of the
basic issues and the document seemed to be riding something of
a juggernaut.  Because it is now in Last Call, I want to
summarize and update those comments, in large measure because I
don't want my near-silence since late January to be taken as
agreement.

This proposal is problematic in several ways.  The first is
obviously the most important, but the others suggest changes
that might be useful whether or not that main provision is
adopted.

(1) Excessively-high requirements for Proposed Standard and
dropping the three-level model.

        The document asserts, I believe correctly, that one of the
        problems with the IETF Standards Process is that the
        relatively low bar established for Proposed Standards in
        RFC 2026 has evolved into a very high bar and that we
        should return to that lower requirement level.  Yet it
        effectively proposed to solve that problem by reducing the
        number of steps in the standards process to two.  No
        evidence has been offered that whether or not the standards
        process is two levels or three has anything to do with
        difficulties completing Proposed Standards or getting them
        to a satisfactory level of completeness and correctness.
        The logic on that subject in the document appears to me to
        quite similar to a physician saying "We agree there is a
        problem with your hand.  We don't know how to treat it
        either surgically or medically, but we should do something,
        so let's amputate your foot".  That really makes no sense
        and is probably bad for the patient (only probably... one
        really never knows).

        In addition, draft-housley-two-maturity-levels-06 does not
        discuss any mechanism for getting from the current
        as-practiced requirements for Proposed Standard back to the
        requirements of RFC 2026, a schedule for doing so, nor
        whether it is necessary to warn readers about the level of
        scrutiny applied (it may not be, but there has been little
        or no discussion of that subject in the IETF and there is
        no such discussion in the document).  If this were a
        technical specification, unless there were an expectation
        that a transition will occur immediately and by magic, that
        omission would be a "known defect".

        Suggestion: The level of completeness and quality required
        for a Proposed Standard, at least insofar as those
        requirements exceed the requirements in RFC 2026, are
        entirely in the hands of the IESG, relevant WGs, and the
        Last Call process.  Whether those requirements can be
        reduced to the level required by 2026 has nothing to do
        with this document because there is no change to the 2026
        requirements.  If we are serious about making that change,
        I recommend that the IESG:

        (i) Put this document aside temporarily
        (ii) Announce to the community (presumably via an IESG
            statement) that, as a general matter, documents
                        submitted for Proposed Standard will not be
                        required to meet conditions beyond those imposed
                        by 2026.  Note that 2026 allows imposition of
                        additional requirements, especially requirements
                        for implementation experience.  Whether a
                        particular WG should strive for a higher quality
                        standard for its documents should be treated as a
                        management issue (e.g., it would rarely make sense
                        to downgrade documents that were nearly finished
                        at a higher level), but reviews on Last Call or
                        from within the IESG that suggest a higher level
                        of perfection should probably be ignored unless
                        they provide reasoning for the greater requirement.
        (iii) The above implies that different areas, and
                        perhaps even different WGs within an area, will end
                        up with different practical requirements based, at
                        least in part, on their perception of the risks
                        implied by a less comprehensive document.  I see
                        that as an advantage and recognition of reality,
                        not a problem. 
                (iv) After a couple of years -- probably the minimum
                    needed to achieve a steady state given documents
                    already under development and targeting a greater
                        level of requirements than 2026 anticipated -- the
                        IESG should review the situation and make a
                        recommendation to the community as to whether
                        further changes are needed to the standards process
                        based on experience with this revised way of
                        dealing with proposed standards. 

        Note that the model described above is different from the
        one in draft-bradner-restore-proposed-00, especially with
        regard to schedule.  draft-bradner-restore-proposed-00
        suggests a fixed transition schedule; these notes propose
        to make transition a management matter (and, indirectly, a
        test of the IESG's seriousness about restoring the intended
        Proposed Standard definition); as noted above,
        draft-housley-two-maturity-levels-06 does not address this
        issue.  After a few months of consideration, I now believe
        that "management matter" approach provides both a better
        transition plan and long-term framework than trying to
        force documents into a single level of completeness and
        scrutiny.  The other comments and suggestions in
        draft-bradner-restore-proposed-00 could be considered
        useful supplements to the suggestions above. 


(2) Elimination of the "downref" requirement for
standards-track documents.

    The original motivation for the "no normative references to
        documents at lower maturity levels" rule was to be sure
        that the documents referenced were of adequate quality to
        be used, effectively, as part of the document under
        consideration.  If one thinks about it, having a document
        that the IETF certifies as representing an implemented,
        deployed, and useful specification dependent on something
        that is no more that an untested probably-good idea makes
        little or no sense.  Referencing Proposed Standards that
        have survived the level of scrutiny that has prevailed in
        the last several years should not be nearly as
        problematic, but removing the downreferencing requirement
        at precisely the same time that we intend to get Proposed
        Standards back to the 2026 level creates exactly the risk
        that the rule was intended to prevent.

        Suggestion: The community had this almost correct in RFC
        4897 in that the community should be able to review
        proposed downward references.  RFC 4897 more or less
        assumes that the review should require explicit community
        review and signoff.  That might be excessive; a more or
        less default review might be sufficient.  But we should
        preserve the community's ability to conclude that a
        downreferenced document is inadequate for the purposes of
        the referencing document, if only through a Last Call
        response.  The change made in
        draft-housley-two-maturity-levels-06 appears to eliminate
        "the referenced specification is of inadequate quality to
        be used as part of this specification in a normative
        reference".  It seems to me that is a bad idea and probably
        even an unintended side-effect.

        Note that this suggestion should be as effective at
        eliminating downward references as a "major cause of
        stagnation in the advancement of documents" as the proposal
        in draft-housley-two-maturity-levels-06 if, in fact, that
        is a major issue.


(3) Maturity level, document editorial quality, and RFC Editor
issues 

        The rather cavalier assumption that a three-stage standards
        process somehow makes it less possible to apply the
        Proposed Standard criterion of RFC 2026 omits another
        important reason why the bar for Proposed Standards has
        become so high.  It would be a reasonable (and arguably
        historically accurate) statement of the traditional
        editorial quality requirement for Proposed Standard
        documents is that they needed to be good enough to be used
        for implementation among cooperating parties, i.e., clear
        enough that an implementer with access to WG mailing lists
        (or authors) could implement them and evaluate document
        quality for implementation.  Perfect English writing style
        was not a requirement, nor was sufficient clarity that one
        was assured of being able to do an interoperable
        implementation in a "clean-room" environment relative to
        the authors and discussions about development of the spec.
        Viewed from the editorial quality perspective, the Draft
        Standard level was the one at which document quality was
        expected to be improved to meet those much higher standards
        of editorial quality and technical completeness and
        clarity.  RFC 2026 requires "no known omissions" for
        Proposed Standard, not "no omissions" or strong assertions
        of completeness.

        One of the significant obstructions to getting Proposed
        Standard documents published quickly is that the IESG has
        insisted on a rather high level of technical quality,
        usually doing the editing itself rather than trusting the
        RFC Editor staff to get that right.  WGs and the community
        have responded to the expectation of IESG insisting on
        documents of very high editorial quality (and sometimes the
        expectations that documents will never advance beyond
        Proposed Standard) by nit-picking editorial quality
        internally and on IETF Last Call, creating additional
        delays.  If we are going to get Proposed Standards
        published more quickly and under 2026 criteria, the IESG
        will need to do some self-examination about these editorial
        document quality issues and perhaps either settle for lower
        quality at Proposed, shift more of the editorial burden to
        the RFC Editor, or both.  In any case, if documents are to
        be less than editorially perfect at the Proposed Standard
        level, there needs to be provision, or at least an
        understanding, for assuring that documents meet a higher
        standard at subsequent maturity levels.  Those provisions
        are absent from draft-housley-two-maturity-levels-06, as is
        even a statement of the possibility that they might be
        relevant...  another "known omission".

        Note that accepting a less formal, and less
        formally-correct, editorial style is consistent with
        publishing "Proposed Standards as soon as rough consensus
        is achieved".  Spending weeks or months sorting out
        editorial issues is not.  

        In addition to the above, Long-duration MISSREF entries in
        the RFC Editor queue are a major source of the claim that
        downward referencing requirements are a significant
        impediment to advancement of documents.  However, if those
        entries are examined, a significant fraction of them are
        actually either missing references pointing to I-Ds (for
        which the present proposal provides no help at all) or a
        mechanism for identifying a set of documents that have to
        be published together to make sense.  As long as we
        periodically approve and publish a single logical standard
        as a collection of documents that may go through a WG or
        the IESG at different rates, there will be a need to hold
        the first-approved of those documents until the
        last-approved one is ready.
        draft-housley-two-maturity-levels-06 does not make
        provision for whatever additional RFC Editor states are
        needed in the IETF Stream to cover that problem.  While not
        major, it is another sign that, if the present proposal
        were being evaluated as a technical specification, it would
        be found to contain known omissions.

    
(4) Discussion of the STD document series in Section 6.

        It has been pointed out several times, in recent months to
        suppress discussion of proposals that might be alternates
        to some or all of this one, that there are many issues with
        how the IETF develops, approves, documents, and manages
        standards that are not addressed in this specification and
        that the community cannot reasonable expect to address all
        at the same time.  The question of what should be done
        about the STD numbers is just one entry on this very long
        list.  Examination of other IETF procedural RFCs indicates
        that we rarely, if ever, include a discussion of a single
        option that we have decided to _not_ pursue (even if only
        in the near term).  The only apparent justification for
        including Section 6 in this draft is that there was a
        specific proposal in an earlier version, a proposal that,
        as the draft indicates, did not get community consensus.
        The section should be dropped from this document before
        publication.  If the community wants to have a series
        discussion of STD numbers, I suggest that
        draft-klensin-std-numbers might be one contribution to that
        discussion.


(5) Patents

    The specification says (last bullet in Section 2): 

    > * If patented or otherwise controlled technology is
        >   required for implementation, the separate
        >   implementations must also have resulted from separate
        >   exercise of the licensing process.  

        While this text is copied from 2026, things have changed
        and the requirement has been interpreted very loosely,
        particularly to leave interpretation of "required" in the
        hands of the implementers.

        If it is repeated in a spec that is approved now and that
        claims to reset and clarify Draft Standard requirements, it
        can easily be read as a requirement that the community has
        to somehow evaluate whether claimed controlled technology
        is actually "required for implementation".  That may be
        obvious in some cases; it may be highly controversial and
        dependent of validity of patent claims (something that can
        normally be resolved only by the courts) in others.  The
        IETF has been repeatedly advised, both by legal counsel and
        by various incarnations of IPR WGs, to avoid straying into
        the territory in which it takes a position on whether an
        IPR claim is valid.  Consider a scenario in which some
        company, say ChuckCo, decides that an IETF Internet
        Standard for Alice's proposal would put them at a
        competitive disadvantage.  It is merely necessary for
        ChuckCo to file an IPR claim against the technology in
        Alice's spec, even a obviously spurious one, assert a
        completely irrational and punitive licensing model that
        cannot be exercised in a reasonable way, and then sit back
        and watch the IETF paralyze itself over this provision as
        written.

        Suggestion: I believe that advice of Counsel should be
        sought about this language and, if feasible, alternate
        language produced that either turns the requirement into an
        "if feasible" request or that places the responsibility for
        evaluating whether the claims are applicable on the
        implementers and not on the IETF or IESG.


Conclusion: 

        This document is not ready for approval.  It is justified
        in terms of one issue but offers no evidence that it will
        help at all with that issue and instead proposes a solution
        that is associated with another set of issue that may not
        be critical-path.  For that primary issue (speeding
        documents into Proposed Stanard), it fails to address known
        issues and, more important, is not required at all.  While
        "no known omissions" is not formally a requirement for IETF
        procedural documents to be published in the BCP series, it
        is nonetheless a useful guideline and this document does
        contain such omissions in several areas.  It removes an
        important criterion for evaluating document readiness for
        publication when the document depends on other
        specifications that are unclear or worse.  It makes no
        provision for transition.  It contains spurious and
        almost-irrelevant text.  And it potentially opens an IPR
        can of worms that the IETF has been warned against.

thanks for listening
   john


_______________________________________________
Ietf mailing list
Ietf(_at_)ietf(_dot_)org
https://www.ietf.org/mailman/listinfo/ietf

<Prev in Thread] Current Thread [Next in Thread>
  • Re: Last Call: <draft-housley-two-maturity-levels-06.txt> (Reducing the Standards Track to Two Maturity Levels) to BCP, John C Klensin <=