On 15 Feb 2015 15:34, "Michael Richardson" <mcr+ietf(_at_)sandelman(_dot_)ca>
wrote:
Dave Cridland <dave(_at_)cridland(_dot_)net> wrote:
> 1) AUTH48 is interesting as a metric; it's clearly an indicator of a
> document actually getting published, and one that doesn't get
delayed
> outside the author's control unlike publication. Moreover, anyone
going
> through AUTH48 has had to deal with an IESG telechat on their
document;
> that's extremely useful experience in leadership selection. Possibly
> "last call" and "telechat" are also good enough - and interesting
in as
> much as a document that "fails" last call is arguably as important a
> contribution as one that goes on to publication.
So, instead of AUTH48, you'd pick an earlier state like IESG Review.
(see https://datatracker.ietf.org/doc/help/state/draft-iesg/, but
actually I think write-up and the diagram are old and missing some
states...)
That (perhaps correctly) skips things that go through the Independent
Submission Stream.
I agree that including the authors into the state above solves the
problem of
a document which is so good that it needs no review and no tickets.
That all sounds good. And yes, I think an independent stream submission is
reasonable to skip. I did consider suggesting that only IETF consensus
documents should count, but that is probably too narrow.
> 2) I think the document uploader concept is good, but limiting the
> documents to only those which get scheduled time in a WG session
isn't
> so good; some WGs don't meet often, or even at all, and a document
> that's so well crafted that it doesn't need discussion in a face to
> face meeting is a really good document in my opinion. Clearly we
don't
> want arbitrary documents either. Any WG document would seem a "good"
> contribution, and I suspect any document with a shepherd assigned
> should be safe, too, since that effectively implies an expectation
to
> publish.
The problem with shephard assigned is that it could be very late in the
process, and a document which *does* get a lot of discussion and revision,
represents a lot of contribution of effort. It is not unreasonable to me
that a single document occupies the entire IETF "life" of 3-4 persons.
Yes, true. As I said, I think measuring contribution is a very hard metric
to find, and there's an implicit suggestion lurking here that we're after
long-term, quality contributions - I'm not sure I disagree with that
incidentally.
But in general terms, any document that has any of those characteristics -
it's WG document, one that's getting WG "time" by some measurement, or one
that has a shepherd assigned - seems likely to be good for a document
uploader metric.
It might be interesting to have the tools adapted so the uploader can note
which of the authors have actively contributed this time around, too, to
avoid the situation where two co-authors have to share uploading.
Similarly, it might be useful for WG chairs to be able to tag a document as
having been discussed within their WG.
> 3) This works for WGs that use a ticketing system and have
> meetings. Not sure what percentage that actually is.
A number of WG chairs don't like the current ticketing system, but
increasing
I think that WGs are going to use some ticket system.
And, thinking about it further, we probably want to promote the use of a
ticket system anyway. In addition, I'm more comfortable with measurable
metrics.
Again, it might be useful for the ticket system to be able to record
whether the ticket resulted in useful discussion, and note who was
involved. Again, this is at the whim of somebody, but I really don't think
excluding whims is entirely a positive thing.
Dave.