Hi,
On Apr 19, 2012, at 22:38, Eliot Lear wrote:
I do not support such a view, and it is not supported in a plain reading
of RFC 2026. What's more, it's not how researchers work. Researchers
naturally move on. If we are looking to further push researchers away
from the IRTF, this is a good way to do it.
Whether or not an experiment is active is also not how research works.
Sometimes work is taken up after years of gaps. ILNP is a good example
of this. Had 8+8 been published in a timely fashion as an experimental
RFC, we would have had obsoleted it, only to then unobsolete it?
I agree with Eliot. I think the IESG is interpreting things into 2026 that
aren't there. Not all Experimental RFCs describe what - with my research hat on
- I would call a scientific experiment. Actually, I can't come up with single
one that does.
Experimental RFCs specify protocols and protocol extensions *for
experimentation*, just as Standards Track RFCs specify protocols and protocol
extensions for widespread implementation and deployment. But they do *not*
(usually) describe the actual experiments that are to be performed, and I know
of no one that would go so far as discuss a timeline for any such experiments.
The assumption that an Experimental RFC always describes an actual scientific
experiment is simply wrong, and since this assumption underlies the proposed
IESG statement, it renders it ineffectual at best and actually harmful at worst.
Experimental RFCs get published for all kinds of reasons. On the IETF side,
this may range from actual proposals for protocol evolution for the community
to play with, to things that failed to get Standards Track consensus but that a
WG didn't feel like abandoning outright, and some WGs use them as a pre-PS
step. The proposed IESG statement completely misses that reality.
And obviously, Experimental RFCs exist on other Streams. In terms of whether
any IESG statement would even apply to IRTF and Independent Stream RFCs, that
is not at all clear to me. In any event, it would have been good to CC the
IRSG, since they are not included on the wgchairs list. If the IESG believes
that their statement would extend to RFCs on other Streams, they should REALLY
have engaged us much earlier, rather than with a six-day deadline for comments.
On 4/19/12 10:31 PM, Adrian Farrel wrote:
We have developed the following draft IESG statement. This does not
represent a change in process,
Actually, this *does* represent a change in process. It implies that any
Experimental RFC must only describe an actual scientific experiment, so that
later on it can be judged whether that experiment has succeeded or failed, and
the outcome recorded. This is a much more limited view of what Experimental
RFCs are than I believe 2026 states.
It also requires the proponents of an Experimental RFC to also sign up for
writing another future document describing the "results." (Which per above, I
believe doesn't make sense for the cast majority of Experimental RFCs, because
they do not describe scientific experiments.)
and continues to value Experimental RFCs
as an important part of the IETF process. It does, however, seek to
encourage documentation of the conclusion of experiments.
We are aware that there may be other discussion points around
Experimental RFCs, and we would like to discuss these, but we also
believe that there is merit in making small, incremental improvements.
The IESG would welcome your thoughts on this draft before they approve
the final text on April 26th.
This is simply much too soon.
IESG Statement on Conclusion of IETF Experiments
Experiments are an established and valuable part of the IETF process.
Actually, I would claim they are not. Experimental *RFCs* are, and IETF
participants (and others) may perform experiments with IETF protocols and
proposed extensions (whether they are described in IDs, or RFCs of whatever
status), but the IETF itself does not do experiments.
A number of core Internet protocols were first published as Experimental
RFCs while the community gathered experience and carefully investigated
the consequences of deploying new mechanisms within the Internet.
In the case where an experiment leads on to the development of a
Standards Track RFC documenting a protocol, the new RFC obsoletes the
old Experimental RFC and there is a clear conclusion to the experiment.
It's not so simple. Consider for example TCP F-RTO. The Experimental RFC is
4138. It defines F-RTO for both TCP and SCTP. TCP F-RTO was moved to PS in
5682, but 4138 was not obsoleted, because the SCTP F-RTO is still considered an
experiment.
4128 was published in 2005. 5682 was published in 2009. So clearly SCTP F-RTO
has failed the "experiment"? Not really. With RTCWEB strongly considering SCTP
as a transport framework for the data channel, it's actually very likely that
SCTP F-RTO will move to PS in a few years too. So if we had declared the
experiment closed, would we have done good or harm?
However, many experiments do not lead to the development of Standards
Track RFCs. Instead, the work may be abandoned through lack of interest
or because important lessons have been learned.
Or, nobody feels a great need to do the work to republish an Experimental RFC
as PS, although it is widely implemented. Again, 4138 was Experimental for four
years, although all major stacks implemented and enabled it by default. Why
took it so long? Because the community had other things to work on and the
original document was fine.
It is currently hard to distinguish between an experiment that is still
being investigated, and an old experiment that has ceased to be of
interest to the community.
Very true. That is why it is very hard to come up with any generic rules here,
as this IESG statement is attempting to do. (And never mind that many
Experimental RFCs don't actually describe experiments.)
In both cases an Experimental RFC exists in
the repository and newcomers might easily be misled into thinking that
it would be helpful to conduct more research into an abandoned
experiment.
An abandoned experiment is not a failed experiment. Old ideas aren't
necessarily bad ideas, and many IETF technologies that lie dormant for a long
time all of a sudden gain traction. (Example: SCTP in RTCWEB.)
In view of this, the original proponents of experiments (that is,
authors of Experimental RFCs, and Working Groups that requested the
publication of Experimental RFCs) are strongly encouraged to document
the termination of experiments that do not result in subsequent
Standards Track work by publishing an Informational RFC that:
- very briefly describes the results of the experiment
- obsoletes the Experimental RFC
- if appropriate, deprecate any IANA code points allocated for the
experiment
- may request that the Experimental RFC is moved to Historic status.
So is the intent here RFC database and IANA registry housekeeping? In that
case, focusing on Experimental RFCs is simply misguided. There are many more
standards-track RFCs that have "failed" and would also need to be obsoleted and
have their codepoints reclaimed.
And unlike for Experimental RFCs, we actually do have 2026 text that talks
about standards-track maturity, etc.
If there is no energy in the community for the producing such an
Informational RFC, if the authors have moved on to other things, or if
the Working Group has been closed down, Area Directors should author or
seek volunteers to author such an Informational RFC.
If I am counting things correctly, almost 1500 Experimental RFCs have been
published so far, many by working groups that no longer exist. That's a lot of
RFCs to write by volunteers (ADs really should have better things to do...) for
little gain.
Lars
smime.p7s
Description: S/MIME cryptographic signature