Hi again John,
Substantive response this time...
On 01/14/2013 01:05 PM, John Leslie wrote:
The IESG <iesg-secretary(_at_)ietf(_dot_)org> wrote:
The IESG has received a request from an individual submitter to consider
the following document:
- 'A Fast-Track way to RFC with Running Code'
<draft-farrell-ft-03.txt> as Experimental RFC
The IESG plans to make a decision in the next few weeks, and solicits
final comments on this action.
I'm pretty darn uncomfortable _ever_ picking a fight with any
sitting AD, But I feel obligated to say this seems like a terrible
idea to me.
As a background, I'm a long-time believer in "rough consensus" for
Proposed Standard and "running code" for advancement along the
standards track. I do not believe the two mix well.
I don't get why they don't mix well to be honest. And I don't
get it from your mail (so far). But I think you're maybe
interpreting the draft differently from how its intended so
let's see if we can figure it out.
I'd also point out that this experiment is optional. And
the intent is that if the experiment were a success that
the fast track process would always be optional - to be
used or not as WG chairs choose, with the co-operation of
authors/editors and ADs, pretty much any of whom could
basically force the draft to be processed in the normal
manner.
Some of your concerns read as if the goal were that all PS
RFCs would be processed this way. That's not the goal and
that's stated in the draft.
A standard needs to be as simple as possible (but no simpler);
running code needs to be complex.
Stephen Farrell wants to "speed up" our process by adding an
incentive for "having" running code. Specifically:
]
] This memo describes a way to fast-track some final stage reviews for
] a working group draft when the working group management and area
] directors believe that the existence of one or more implementations
] serve as a practical review of the engineering choices made by the
] working group.
Actually, now that I read that carefully, it doesn't even need to be
"running" code. :^(
Well I reckon its pretty clear when you read the whole thing,
but text tweaks are welcome.
] This experiment needs an implementation that matches the specification
] contained in the draft.
(Verifying that has proven to be surprisingly hard. One guesses
that the "match" may not be rigidly enforced...)
Understand, IMHO there is much to be gained by testing the
interoperability of independent implementations of a specification;
but that is a long-winded process, hardly anything which could be
done during the "fast-track" Stephen is proposing.
It may be a long process. I think that'll depend on the
subject matter. Could well be that this isn't suitable for
things where sensibly checking the implementation matches
the draft is too complex. And maybe that's a good thing
(if it encourages us to write simpler specifications for
example).
] It is up to the WG chairs and responsible AD to satisfy themselves
] on this point within the normal bounds of trust
"Normal bounds of trust" isn't particularly compatible with our
paradigm of transparency of process...
I don't even think it means the same thing to the rather limited
set of people named here.
Yeah you may be right. Can't recall who suggested that phrase
(maybe it was me, but it doesn't sound like me:-) but it could
be improved on I guess.
We do also trust WG chairs and ADs for lots of stuff already, so
would it be better if it said, e.g.:
This experiment needs an implementation that matches the
specification contained in the draft. It is up to the WG chairs and
responsible AD to satisfy themselves on this point.
There is no implication here as to the licensing of the
implementation whether open- or closed-source. ...
] and without any implication about the licensing related to an
] open- or closed-source implementation.
Any mention of licensing seems out of place here.
But it only says that licensing can be anything. I do think
that's worth saying but maybe could be better split out as
above.
] Note that the existence of an implementation is not sufficient to
] ensure that the draft will automatically pass working group or IETF
] last call, nor that it will receive IESG approval.
Nor should it be, of course...
Getting to the devil-in-the-details:
]
] 1. Any Working Group Last Call (WGLC) or Area Director (AD) review
] ... will run in parallel with the two-week IETF Last Call
] 2. The IETF last call announcement message must say that the draft
] in question is following the fast-track process...
] 3. The draft itself should note (e.g. in the introduction) that it
] has been fast-track processed and reference this RFC.
] 4. Only comments that would be "DISCUSS-worthy" according to the
] IESG Discuss Criteria [DCRIT] need be handled during last call.
] Other comments can be handled or not, at the authors/editors
] discretion.
This is actually normal for IETF-wide LastCall (though it leads
to raised tempers when seemingly valid comments are ignored).
It is not normal for WG LastCall; nor is it even compatible with
"rough consensus" -- by which we mean that any new questions raised
have been considered to the satisfaction of a substantial majority
of the participants.
Well WGLC isn't part of 2026, and others have argued that that means
that there's no need for this to even be a process-experiment. That
seems to be at odds with your position.
My take on this is that yes, this is a proposal for an experiment
that changes what we do now with WGLC, and its one I think we ought
try. I think we just disagree on whether that's a good thing to
try or not.
] 5. The responsible AD for the WG concerned makes the decision as to
] whether changes are required as a result of review comments, and
] determines whether or not those have been completed. If
] significant change or extended discussion is required, or if
] changes are not complete within two weeks after the end of fast-
] track last call, then the draft is returned to the WG by the
] responsible AD and the document is withdrawn from the experiment.
This is not an easy read. :^(
First, it centralizes the decision in one person (whereas normal
WGLC process has two or more WG Chairs judging consensus of the group).
I disagree. All the WG chairs must already want to use the fast-track
experiment so they presumably are already ok with the draft at the
start of IETF LC. So the only thing the responsible AD is judging is
whether the IETF LC comments are such that going ahead to IESG review
is ok, which what happens now.
Second, it centralizes the decision in one person whether changes
in the document properly address any WG or IETF-wide comments. This
kind-of matches current practice in the IESG for Area Director
comments; but does not match the theory -- and we could on very short
notice regress from the level of trust which allows this to happen
now.
I don't get what you mean there about the theory.
Third, the two-week drop-dead time limit is unworkable. Not just
IMHO; in practice it hardly ever happens that everything resolves
itself that quickly.
You might be right. But I think if some WG chairs and authors/editors
ask for fast-track then they also need to be willing to work quickly
to handle IETF LC comments, and they ought only do this where they
expect it to work and are willing to try make it happen. So I'd
argue to keep this timer. (The duration of the timer I agree might
be wrong.)
AD DISCUSS comments tend to bunch up in the
hours before the telechat, and frequently need to be clarified
_after_ the telechat.
I don't know what you mean about AD DISCUSSes - IESG review isn't
changed in that respect and the draft does not require authors to
handle those any differently from today.
Fourth, it's not really clear what happens after "the document is
withdrawn from the experiment." It would appear at first blush that
it must now start a separate WGLC, followed by a new IETF LastCall.
I would never consent to this process for a document _I_ cared about.
You are correct, the intent is that the draft goes back to the WG
and is then processed as normal.
] 6. As soon as the responsible AD has confirmed that the authors/
] editors have made any changes required as a result of fast-track
] last-call, then the document should enter IESG review and be
] placed on the next IESG telechat agenda that is more than one
] week in the future.
Oh... Stephen wasn't talking about the IESG process at all!
I'm even more confused now.
I'm confused that you're confused!
I guess this means that one person (the responsible AD) takes a
look at any and all comments during the "combined" LastCall and
responsible-AD-review and makes a unilateral decision of what
changes the authors/editors should make (on short notice); then
things open for IESG-member DISCUSS comments.
Right. That's the idea. But note the WG chairs kick off the
process.
This would seem to preclude the current practice of putting
drafts on the IESG agenda _before_ the end of IETF LastCall. I'm
only getting more confused as to how this speeds up our process.
It may be that the set of drafts for which this is useful and the
set that are reasonable to put on a telechat agenda before the end
of IETF LC are disjoint. But its an interesting, though I'd say
minor, point that might come up during the experiment if we try it.
] 7. Given the reduced time associated with fast-track processing, the
] responsible AD may not have time to carry out sufficient review
] prior to IESG evaluation to be confident in balloting "Yes" for
] the document. Draft progression during and after IESG review is
] otherwise unaffected, so a "Yes" ballot is needed from some AD.
I _do_ understand that part, thought it's a bit obscure to those
who haven't been IESG members: no document gets approved by the
IESG without at least one "Yes" ballot-position; and Stephen wants
to be able to decline to ballot "Yes" on a document he (as
responsible-AD) has almost unilaterally re-written. (This would
be a really-bad idea, IMHO.)
No, that's not the intent, nor I think would it be what'd happen;
"almost unilaterally re-written" would not be appropriate I entirely
agree, but I don't see any AD doing that after IETF LC now, and I
can't see it happening with this experiment.
Today, the convention is that the responsible AD ballots yes to
kick off the process of IESG review. This is just saying that
that AD might start IESG review with a different position (I
would guess a no-objection, though there are exceptions to
everything;-) because that AD hasn't had time to fully evaluate
the draft. That AD might e.g. have asked someone else to check
the implementation (allowed by this draft) but might want to
do a fuller review during IESG evaluation before presumably
balloting yes. Requiring the yes ballot up front is just a bit
too demanding with the fast track timers.
] 8. If at any point in the fast-track process the responsbile AD does
] not act in a timely fashion then the IESG Secretary should ensure
] that the draft enters IESG evaluation and is scheduled for the
] the soonest IESG telechat that is more than one week after the
] end of the two-week post-IETF LC period. That is, if the
] responsible AD does not act, then the default action applies
] which is that the draft enters IESG evaluation and is placed on
] the earliest IESG telechat.
(This is too complicated for me to try to explain in this email.)
Maybe you'll come back to it?
====
There's a lot more I could review, but I won't in this post.
Were I an AD responsible to ballot on this, I'd ballot "abstain".
"Abstain" means "There's no point trying to satisfy me," and is
the way an IESG member essentially votes "No."
I don't believe "running code" is even particularly helpful in
run-of-the-mill Proposed Standards.
Right. I guess that's where we disagree.
I've seen it _seriously_ harm
the documents in cases where the "running code" has shipped and
there is reticence to change it: no matter how many weaknesses are
pointed out, you can't get WG consensus to change the spec.
I agree that happens. I don't think this'll change how often.
But if it made that happen more often then yes, we should probably
declare the experiment over and not make this part of the
formal process. I could argue that this might reduce that
problem, on the basis that the draft and code are more likely
to be the same, but I've no empirical basis on which to base
that argument. (Same as you wouldn't if you argued that this
experiment will make it happen more often.)
In cases where "running code" has proven helpful, it's my
honest opinion that the spec has become way too complicated (and
thus the "running code" has proven which parts are unworkable).
Hmm. I've had somewhat the opposite experience, so I guess
YMMV. I've not gone to dig up examples though but in DKIM I
think we ended up with a simpler spec. due to some folks
doing good work with running code before and during the WG's
processing of the drafts.
I don't believe it's possible to "fast-track" "runnng code";
nor do I believe it would be desirable were it possible. Code
needs to deal with complex cases that, for the most part, don't
need to be standardized in the first place. But code which
doesn't deal with these is useless in the marketplace.
That may be right, but we're not talking about fast-tracking the
code.
IMHO, we designed a separate "Proposed Standard" step to get
a specification published quickly without delving into the many
questions that arise when writing actual code.
That's fair. I think this experiment partly reflects one way
in which the standards process theory and IETF practice have
diverged over the years. The bar for PS is mostly now much
higher than was the original intent. I think that in some
cases, this experiment might lower that bar a little.
There are plenty
of Standards-Development Organizations that start from "code".
We don't need to become one of them.
We also have issues with RFCs for which there is no code and
never will be;-)
But more to the point, I think that in a lot of cases where
the IETF has done a good job, there has been running code
before the WG even started, so I disagree that starting from
code is bad. Rubber stamping stuff whether based on code
or not would be bad, but isn't part of this experiment.
Cheers,
S.
--
John Leslie <john(_at_)jlc(_dot_)net>