Re: Running code, take 2

On Fri, Dec 14, 2012 at 1:36 PM, Alessandro Vesely <vesely(_at_)tana(_dot_)it> 
wrote:
> On Fri 14/Dec/2012 09:49:30 +0100 Yaron Sheffer wrote:
> > to clarify, my proposal only applies to Internet Drafts, and clearly
> > states that the implementation section should be removed from the
> > document before it is published as RFC.
> One place where an Implementation Status may help is IETF LCs:  People
> are asked to comment on I-Ds whose development they didn't track, on
> topics they may have only a working knowledge of.  Code licensing and
> IPR details could quickly convey whether a given standardization
> process is being gamed, for example.
>
> > Formally, we don't want non-permanent stuff in RFCs.
> We have the Errata and the Outcomes, AFAIK.  The Errata is freely
> writable, somewhat impractical, and doesn't address this purpose.
>
> > And realistically, even if we had an implementation wiki, it is
> > unlikely to be kept up to date once the RFC is published.
> The Outcomes /is/ a wiki, http://trac.tools.ietf.org/misc/outcomes/.
> No wonder it's not updated:  It is difficult to find it from the
> relevant RFCs (let alone specific sections thereof), and it is not
> freely writable.  Anyway, it addresses only a small subset of the
> Implementation Status.
>
> While we need no formal procedure to write the Implementation Status
> section of an I-D, we probably need to specify how the content of that
> section can be used to set off the wiki page for the new RFC,
> and how that wiki should work.  Would that help running code?
<brainstorming_mode>
This just came to my mind... Why not doing a little step more and
making the wiki page a kind of "directory" to available
implementations?  Maybe each entry could have a "license type" field
(e.g. open source, commercial, ...) and "compliant flag" assuming
values like "fully compliant," "partially compliant," "unknown,"
"what kind of weed did they smoke before coding this?!?" :-)  If the
protocol allows it, the compliant flag could be assigned on the basis
of running a suitable set of test vectors (possibly described in the
RFC), otherwise it could be assigned after a community discussion
(maybe inside the WG that produced the protocol, it still existing).

This kind of "implementation directory" could be useful not only to
the IETF, but also to people who needs to implement a protocol and
would prefer to use something ready.  A new entry in the directory
could be written by anyone (or proposed via e-mail).  The author that
writes the entry cannot set the compliant field which will be
"unknown" by default.  If there is some evidence that the
implementation is compliant (e.g., the software is used inside a
widely used browser and it did not cause any meltdown so far:-), the
status could be set to "probably compliant." The upgrade to "fully
compliant" would require running tests and/or community discussion.

Finally, I would also add to the implementation page a reference to
any available test vectors (if the protocol allow them).  I find thme
are quite useful during the implementation phase.
</brainstorming_mode>

Riccardo