ietf
[Top] [All Lists]

Re: I-D ACTION:draft-etal-ietf-analysis-00.txt

2002-03-29 18:20:02
Peter Deutsch <pdeutsch(_at_)earthlink(_dot_)net> wrote:

The implications for this seem clear enough. It seems to imply that the
amount of traffic per protocol the activity goes on to generate is a
reasonable milestone for any IETF activity. This doesn't mean the POISED
list (or heck, even the IETF general list ;-) should be shut down (and
putting aside the question of the increase in SMTP traffic they generate
:-) The point is that we should recognize that such activities are
clearly overhead, and we should all be doing our best to minimize such
overhead whenever we can.

I don't feel comfortable with the notion that the work of a WG should be
judged according to adoption of its protocols, particularly in terms of
traffic generated.  All protocols are not equal; some have limited
utility by design, as they serve a limited community.  I also don't think
that time spent in pre-use is necessarily overhead; this diminishes the
value of producing clear documentation.  If it takes longer than some
might desire for RFCs to be published, but the overall clarity of the RFCs
is improved (regardless of their utility), I think that's time well spent.

So, if people agree that traffic measurements have value as a metric,
then presumably the first derivative of traffic volume over time is also
a reasonable indicator of the future takeup rate for new protocols.
Measuring some of the other things being discussed here (RFC counts,
engineer-hours spent in meetings, messages to a mailing list, count of
pastries consumed) would all seem to me to be measuring overhead
activities, not core to the organization. This is not a bad thing to
understand, but would not seem to be the most important metric in our
arsenal.

But past performance is not always an indicator of future performance.
Take the first couple of years of HTTP traffic, for example.

Instead of allowing us to conclude that we're doing fine and those pesky
users are letting us down by not deploying our favorite new toy, it puts
the responsibility for convincing users to adopt our work clearly on our
own shoulders, which is where I think it belongs. Just as any good
business person knows the the difference between a good technology and a
good product, we should acknowlege the difference between a good technology  
and a good solution. If people don't deploy it, we are the ones who failed.
We didn't build it in such a way that it was a better alternative for the
user and it wasn't used. Anything else is noise, not signal.

But protocol popularity is not something intrinsic to the protocol itself.
Application development plays a huge part.  Again, using HTTP as an example,
the immense popularity (and utility) of web browsers drove HTTP traffic.
Some protocols need a killer application before they see a lot of utility,
which will in turn create more interest/demand in improving the protocol --
and more WGs and/or WG activity. :)

As a simple test, if we were to find that the percentage of traffic on
the net using IETF developed/endorsed protocols turns out to be falling,
it would imply that the organization's influence is waning, which would
be something we might want to investigate.

This need not necessarily be considered a failure of the IETF.  It might
be an indication of the maturity of the IETF, in that other standards
bodies/companies/users can use IETF protocols/services/BCPs as a foundation
for whatever it is they're trying to do.

--gregbo