Hi Thomas,
On 01/22/2013 09:31 PM, Thomas Narten wrote:
FWIW, I share Joe's concerns. And Stephen's responses don't really
change my mind.
Ah well. I'm willing to keep trying:-)
This document seems to have a bit of missing the forest for the
trees. In the overall scheme of things, I don't believe the draft will
materially help, and is at best a distraction from dealing with
meaningful problems.
Do you mean the experiment or if this became a permanent
part of the process? Your comments read as if you're concerned
mostly about the latter, and you make no mention of the
former.
As it happens, I agree that this is not going to produce
a huge improvement. But its not intended to, and modest
improvements (as opposed to modest proposals:-) should also
be ok. If not then we're in a worse process-rut than even I
think.
The crux of the issue is that any attempt at "fast tracking" is
fundamentally about short-circuiting some aspect of our review
processes.
I'd prefer s/short-circuiting/speeding-up/
We should only do that very carefully.
For example as an experiment? I think we are being careful.
In almost all cases,
individual judgement is needed as to when it is appropriate to do
that. ADs/WG chairs already have some flexibility here. e.g., a WG can
skip WG LC if they think its not needed. And nothing stops an AD from
going to WGLC before doing a careful review. But I'd rather that be
done because the case at hand justifies such an optimization, not
because there is some "running code" checkmark that allows a special
case to be fast tracked..
This document risks substituting process for judgement. I'd rather see
more of the latter and less of the former.
Disagree. The WG chairs exercise judgement to kick off the
process. The IESG exercise judgement as usual with no changes.
Some specifics from the doc:
In summary this experiment collapses three stages of the publication
process to run concurrently: working group last call, AD review, and
IETF last call. Additionally, the experiment will only require
IMO, it is almost never appropriate to collapse WG LC and IETF LC. In
cases where it is appropriate to do so, there presumably isn't a need
for a WG LC to start with. And an AD has always been able to start an
IETF LC without doing a detailed review. The point, is judgement is
needed, and all such cases are best handled on a case-by-case
basis.
What's wrong with the judgement of a set of WG chairs? AFAIK, ADs
are no wiser in general and this one is certainly dumber than a
lot of WG chairs.
Surely, we don't need a document to justify doing this in those cases
where it makes sense...
IETF last call. Additionally, the experiment will only require
issues raised during these three stages to be addressed if they meet
the IESG's Discuss criteria. The former is not formally a
process
and later:
4. Only comments that would be "DISCUSS-worthy" according to the
IESG Discuss Criteria [DCRIT] need be handled during last call.
Other comments can be handled or not, at the authors/editors
discretion.
The above can easily become a way to dismiss legitimate review
comments.
Equally, gratuitous nit-picking can be used to try kill something
with a thousand cuts. Neither are at all common that I can see,
during IETF LC.
I think incorporating the discuss criteria should be an interesting
part of the experiment. (I've found them really helpful in reviewing
documents these last couple of years.)
No doubt, when a AD or reviewer suggests "this needs
fixing", the proponents will hold up this document and say "you
shouldn't do this, per the RFC -- you're violating the spirit of this
document, only really really critical stuff needs to get addressed..."
No thanks. That is not what the IETF is about.
I don't see how you get to "No doubt..." I do entirely doubt that'll
happen for most all editors.
If it is really urgent to get a document done, it is far better to
take steps to make sure the editors are engaged and responsive. That
is more likely the real issue!!!
Urgency isn't a part of this. Going faster doesn't imply urgency.
It is understood that the savings in time for the end-to-end delivery
of a proposed standard or experimental RFC may be modest, however,
the modifications to normal IETF process will also serve as an
indication of the importance that the IETF places in running code.
Running code is valuable. Always has been, always will be. But we need
to resist the temptation of making "running code" more equal than
other criteria or putting it on a pedestal as some sort of holy
grail.
I agree. And this draft doesn't do that. Or are you saying it does?
Running code by itself is just a sound bite. The importance of
running code is what it tells us about a protocol specification. Just
the mere fact that there is running code doesn't mean there is
anything particularly enlightening to learn from an
implementation. For example, "running code" of a router function in
software doesn't necessarily say anything as to whether the code can
be implemented efficiently using ASICs, which may be the real
requirement.
Sure. The proposal says that this won't be suitable for all drafts.
In response to other comments we had the idea of building up text
about that as part of a wiki.
5. The responsible AD for the WG concerned makes the decision as to
whether changes are required as a result of review comments, and
determines whether or not those have been completed. If
significant change or extended discussion is required, or if
changes are not complete within two weeks after the end of fast-
track last call, then the draft is returned to the WG by the
responsible AD and the document is withdrawn from the experiment.
The two week figure is too stringent.
Possibly. But I think its a useful quid-pro-quo - if the authors
want fast track review but aren't willing to do fast edits then I
think they ought not get fast-track review.
I think the timers in general are something that an experiment
may show to be ok or not. And if not, then changing 'em would
be easy if we wanted to continue this after the experiment.
The above is too prescriptive to
be practical. For starters, how will the IESG telechat align with the
ending of LC?
That's covered in the draft. Could be it needs to be better
stated. (Basically, next telechat that's more than one week
after the authors have done their changes, if they got those
done within the two weeks after the end of LC.)
Or what if an IESG member issues a defer (or will those
be disallowed?).
No change there. IESG evaluation is entirely unaffected by this.
The draft says that.
Section 4 has a bit too much detail and process in it. I'm sure it has
both too much and too little actually. I.e., it probably misses some
corner cases, and then what do you do given how prescriptive the rest
of the section is?
That's fair. Its hard to get the right balance there. I think
some of that text could be better on a wiki, as discussed in
response to some earlier comments.
But overall, I see this document at best as a no-op. However, I fear
that it can be used to short-circuit our review processes in a way
that doesn't help the IETF and that won't really speed things up
enough to justify the downsides.
I think the worst case is nothing happens but that if this
does get tried, we'll learn something about our processes and
(of course:-) I reckon we ought do the experiment.
Cheers,
S.
Thomas