There's something inherently wrong with trying to establish criteria for voting
DISCUSS.
My understanding was always that DISCUSS was supposed to be an indication that,
at a minimum, the AD needs to understand the situation better before casting a
yea or nay vote. The resolution of a DISCUSS might end up being a yes vote, a
no objection vote, a request for clarification of the document. It should also
be possible for an AD to say "now that I've understood better, I can't support
this going forward" (for which ABSTAIN is not an appropriate label).
It should never be wrong for an AD to vote DISCUSS, though it's reasonable to
limit the amount of time during which a DISCUSS can stand for a particular
document.
It should also never be wrong for an AD to vote NO if the AD has initially
voted DISCUSS, made a sincere effort to understand the issues, believes that he
does so, and can state 2026 or other documented criteria for why the document
is not suitable for approval.
So my recommendations are:
1. Fix the broken IESG voting system before you try to establish more decision
criteria. I'm serious. The system you have now is just too broken, and has
been broken for a long time. It places too much pressure on ADs to cave in
when a WG produces flawed output that isn't easily fixed. It is weighed too
heavily in favor of approving a document - such that one AD can vote YES,
another vote DISCUSS, no other ADs read the document and all vote NO OBJECTION,
and the AD that votes DISCUSS will be expected to change that vote to ABSTAIN
because nobody else felt like reading it. And the idea that an AD should
ever, ever, be compelled to change his vote to an ABSTAIN is completely
unacceptable. And the "votes" of ADs who haven't read the document should not
count AT ALL.
Here is a stab at better voting criteria:
- The possible votes are: Yes, No, No Objection, Discuss, Abstain, Recuse
- Yes means "I've read it, I believe it meets applicable criteria (2026 or
whatever else is applicable), that there's community-wide consensus to support
the document, and that all issues raised by reviewers (including directorate,
last call comments, etc.) have been adequately addressed."
- No means "I've read it, but one or more of the above criteria have failed to
have been met." The criteria must be documented.
- No Objection means "I've read it, and I think there are issues with it, but I
don't think those issues are sufficient to block the document". The criteria
should be documented, but the AD isn't compelled to do so.
- Discuss means "I have read the document and see at least one potential issue
which needs to be discussed." Discuss should always be raised before voting
No. However, Discuss votes should time out and be replaced by Abstain (if not
explicitly changed by the AD) after say 45 days. (however, nothing should ever
stop an AD from changing his vote to No.) Note that the ADs, WG chairs,
authors, etc. should attempt to resolve the issue offline (not during the
telechat or other IESG meeting) but the discussion should be archived.
- Abstain means "I did not read this", and/or "I don't consider myself
competent to review this"
- Recuse means "I have a conflict of interest with stating a position on this
document"
All Discuss votes must be changed (whether explicitly by the AD or by automatic
timeout) before a ballot is finished.
If there are two or more No votes, all ADs without a conflict of interest are
expected to read the document, and the vote is taken again.
In order to approve a document or standards action, there must be twice as many
Yes + No Objection votes as No votes.
2. Don't overconstrain the use of DISCUSS. In particular, don't ever create a
situation where a reviewer can't cite a problem with a document, regardless of
whether that problem has previously been enumerated. It's fine to have
guidelines for the benefit of new ADs but they should be nonbinding. You need
to have other ways of dealing with the occasional stubborn AD who votes DISCUSS
or whatever without a defensible reason.
This applies to all IESG voting actions, not just moving from PS->DS/IS.
3. I take serious issue with the statement in the draft that IESG reviews are
"reviews of last resort" and the implication that WG reviews are sufficient.
In numerous situations this has not been the case. I'm all for having IESG
place greater reliance on directorates (especially if this is formalized to
some degree), so as to lessen IESG's workload to to get individual ADs out of
the hot seat. But for IESG to presume by default that the WG has done due
diligence is for IESG to shirk its responsibility.
4. When evaluating PS->DS/IS actions, IESG should avoid repeating the PS
review. Instead it should look at what has changed between PS and DS/IS, and
what has been learned as a result of implementation and deployment experience.
The changes have to be minimal and "safe". No new features can be added,
and there needs to be reason to have confidence (ideally backed by analysis and
implementation experience) that any changes made will not impair
interoperability between both PS and DS/IS versions of the protocol.
Design flaws discovered since approval as PS are also fair game, and flaws for
which the implications are better understood than at PS time should also be
fair game. However it's not fair to revisit an issue that was raised at PS
approval time, unless new information or understanding makes that issue of
worse consequence than understood at that time. (e.g. say an issue was raised
at PS time and it was decided it was too minor to hold up the document, but
since then practical DoS attacks had been identified that exploited that
problem...then it's fair to raise the issue. It's always fair to raise an
issue about something that actually causes problems.)
What's also not fair game is to "raise the bar" - to expect the document at DS
to meet more stringent criteria than it was required to meet at the time of PS
approval. IESG needs strong justification (e.g. community consensus) to raise
the bar in this way for standards that are actually deployed. This includes
but isn't limited to security issues.
(Though it might be reasonable to make an exception to this if more than N
years had lapsed since PS approval.)
But IESG should not presume that there are no serious technical issues. It
shouldn't presume one way or the other. And it shouldn't discourage ADs from
raising DISCUSSes about technical issues at all.
Keith
On Aug 30, 2011, at 4:18 PM, Jari Arkko wrote:
During the discussion of the two maturity levels change, a question was
brought up about DISCUSSes appropriate for documents that advance on the
standards track. We discussed this in the IESG and I drafted some suggested
guidelines. Feedback on these suggestions would be welcome. The intent is to
publish an IESG statement to complement the already existing general-purpose
DISCUSS criteria IESG statement
(http://www.ietf.org/iesg/statement/discuss-criteria.html).
Here are the suggested guidelines for documents that advance to IS:
http://www.arkko.com/ietf/iesg/discuss-criteria-advancing.txt
Comments appreciated. Please send comments either on this list or to the IESG
(iesg(_at_)ietf(_dot_)org) in time before our next telechat (Thursday,
September 8, 2011).
Jari
_______________________________________________
Ietf mailing list
Ietf(_at_)ietf(_dot_)org
https://www.ietf.org/mailman/listinfo/ietf
_______________________________________________
Ietf mailing list
Ietf(_at_)ietf(_dot_)org
https://www.ietf.org/mailman/listinfo/ietf