On 11/9/13 8:40 AM, Randy Presuhn wrote:
"Running code" means it's the implementations (not necessarily
the implementors) that should carry the weight. Yes, for
protocols to be implemented the implementors (eventually)
have to agree, because otherwise the protocol doesn't get
implemented. But "running code" is really the code, and
not just the opinion of those who might write the code.
...
I have very mixed feelings about this. Yes, it is the way
some working groups function. However, implementors are
in the best position to comment on the how difficult it is
to implement the specification correctly, as well as how
elements of the specification contribute to (or reduce)
implementation complexity. Both of these have very real
consequences for interoperability, security, and ensuring
a diverse ecosystem of independent implementations. The
mere fact that someone was able to get something to run
as specified should not carry more weight than their
assessment of how much unnecessary pain or overhead (CPU,
memory, whatever) results from quirks of the specification.
Yup. I'll reply to the rest of Larry's message in a bit, but this part
is complicated enough that I'm not sure I want to dive into it in the
document. I agree with Randy that assessment of other factors (CPU,
memory, whatever) is of course weighed against the "mere fact that
someone was able to get something to run as specified", and if someone
clueful calls this out, we tend to listen. But keep going with the
example: If some smart person does call it out, the implementer might
say, "Baloney! I got this implemented in 2K of code on an old 100Mhz
embedded processor." And the smart person might respond, "Yeah, but on
an embedded processor, you can play all sorts of games to get this
running in that space; on a real processor on a general purpose piece of
hardware, this is going to be a nightmare." And so on. At the end of the
discussion, the chair (or whoever) is going to have to sort the wheat
from the chaff: What's a showstopper and what's just a preference? Is
the implementation cited just a one off and not representative of the
type of implementation that the groups output is chartered to work on?
Is the assessment that it's going to take too much CPU a wild guess or
does it have some real backing? Do we need to look at some benchmarks?
To figure these things out, hopefully the group will have come to its
own consensus about the facts at issue, but if the consensus is going to
be rough, the chair is going to have to make a call on what's legit.
Blindly saying, "We're going with the running code" without seriously
evaluating that running code is about as useful as saying "We're going
with what Mary Smartimplementor claimed" without seriously evaluating
the claim.
Engineering is hard. We've got to do that. Coming to consensus over the
engineering tradeoffs is also hard; there's no one-liner like, "We
always go with the running code". As the document says, the "running
code" bit of the mantra is that we value actual practical experience
over theoretical models. But running code is not itself some sort of
magical thing. We still have to do the rest of engineering and come to
consensus on whether we've gotten it "right". And I do think the
document says that much. I don't think it should go much further.
pr
--
Pete Resnick<http://www.qualcomm.com/~presnick/>
Qualcomm Technologies, Inc. - +1 (858)651-4478