Hi Mark,
Thanks for the prompt response, and sorry for a delay in responding.
On Wed, 23 May 2007, Mark Allman wrote:
1) the document appears to be slightly inconsistent wrt. what it's
actually specifying, or there are nuances that may not be sufficiently
well articulated. The introduction speaks of "..evaluating suggested
CC algorithms that _significantly differ_ from the general CC
principles outlined in [RFC2914]" (emphasis mine). The abstract does
not mention 'significantly differ', and there is Rule (0) which seems
to imply that this BCP could be applied both to the mechanisms that
significantly differ from the RFC2914 guidelines and other congestion
control mechanisms that still honor the RFC 2914 guidelines. Which is
it? Does it have implications on the different 'tracks'?
I think you are reading a little much into this. I have just tweaked
the language in the document to include 'significantly different' in the
abstract and have changed (0) to this:
(0) Differences with Congestion Control Principles [RFC2914]
Proposed congestion control mechanisms that do should include a
clear explanation of the deviations from [RFC2914].
Is that more clear?
Yes (though '..that do should..' seemed weird on the first read).
Section 2 also says "Each alternate CC algorithm published is
required to include a statement in the abstract indicating whether
or not the proposal is considered safe for use on the Internet".
Which documents this applies to is not clear enough: all IETF
documents (through WG or through an AD)? IAB documents? IRTF
documents? Individual RFC-editor submissions? It is not clear to me
whether this document would have the authority (without explicit
discussion within the RFC-editor publications constituencies) to
impose further requirements on non-IETF document streams especially
if it doesn't include 'Updates: 3932' in its header. I suspect the
document only means to affect the IETF RFC publication stream but is
not clear enough about it.
The document is intended to apply to IETF-produced documents. I added
the following to the 'Status' section. Does this help?
Note: we are not changing RFC publication process for non-IETF
produced documents (e.g., those from the IRTF or independent
RFC-Editor submissions). However, we would hope the guidelines in
this document inform the IESG as they consider whether to add a note
to such documents.
Yes.
Section 4 only talks of 'minimum requirements for a document to be
approved as Experimental with approval for widespread deployment in
the global Internet.' and further down "..approval for widespread
deployment". What about the rest? Also, is omitting "in the global
Internet" intentional? The flow of text doesn't go well as it is if
that's the case, and if it's intentional, I'd rather use two
entirely different wordings to make 'encouraged for widespread
deployment' and 'encouraged for widespread deployment in the global
Internet' more clearly distinct from each other. (Also, it isn't
clear if the text is out of sync regarding guideline (2) compared to
what it says in section 3 and what it says here.)
I agree the text is a little weird. I beat it a little. Does this:
The minimum requirements for approval for widespread deployment in
the global Internet include guidelines (1) on assessing the impact
on standard congestion control, (3) on investigation of the proposed
mechanism in a range of environments, guideline (4) on protection
against congestion collapse and guideline (8), discussing whether
the mechanism allows for incremental deployment.
For other guidelines, i.e., (2), (5), (6), and (7), evidence
that the proposed mechanism has significantly more problems
than those of TCP should be a cause for concern in approval for
widespread deployment in the global Internet.
work better?
(Maybe you meant to add 'following' before 'guidelines' in the 2nd
line?)
If 'following' or some such is added, the first paragraph is clearer
on what the authors and reviewers should do: they should read through
the listed guidelines and do what's required there. It might not be
optimally clear whether the author _must_ do as 'shoulds' say in those
guidelines or are those just suggestions. I'd guess you mean either
"must do" or "must do, or justify why you haven't" but not "may do".
What isn't clear enough is what the authors and reviewers should do
for the case of 2nd paragraph's guidelines. Can the authors skip them
completely, and the burden of proof that the proposal causes
significantly more problems on the reviewer?
Or do you mean that the proposers should do everything guidelines (2),
(5), (6) and (7) say, but shortcomings in the results of these
guidelines (e.g., proposal being only slightly more troublesome than
TCP) should not block the approval for widespread deployment in the
global Internet. Also the same point as above wrt 'shoulds'.
I suspect you mean the latter, but this should be clearer.
2) the document requires that 'serious scientific study .. needs to have
been done'. Yet there is no metrics an outsider could use to evaluate
whether this test is met or not. It may be that TCP congestion
control community has de-facto oral tradition what needs to be
evaluated before an algorithm can be looked at seriously, but based on
this proposed BCP, such metrics are not clear enough.
Unless we can define clearer requirements and metrics for evaluation,
perhaps the IETF should not attempt to make such an evaluation but
leave it to the scientific community. I'm not sure how the IETF can
liaise with the scientific community on that, e.g., whether it's
possible to get a consensus statement from IRSG or an IRTF RG or
something that could meet this requirement (if we don't want specify
these criteria within the IETF).
The IETF and the IRTF have worked out a procedure for working on these
sorts of things (and Lars has documented it in an ION).
(Note: this is still a draft version if you're referring to
http://www.ietf.org/IESG/content/ions/drafts/ion-tsv-alt-cc.txt)
This model raises one fundamental question issue of the scope of this
document.
Who should be evaluating section 3 and 4 of this document? Is this
solely meant for ICCRG, the IETF or both? If both, would both parties
do everything described in those documents?
It would be clearer if the reviewer was ICCRG, and the IETF would not
attempt to perform the same review, and the IETF wouldn't be allowed
to second-guess the proposal, e.g., that it's research has not been
done well enough if it was already positively evaluated by ICCRG.
At the end of the day, IMO, the IETF needs to be able to make decisions
about new CC schemes. The document we wrote lays out a set of areas in
which authors of such proposals need to provide information to the IETF
community such that the IETF community can make sound engineering
judgments about the proposals. For a document that intends to live for
a long time it is pretty hard to go beyond "sound scientific evaluation"
and a big-picture list of *areas* where we know we want information.
Maybe including a reference to this (even changing) list of evaluation
criteria would help to mitigate my concern of enabling a 'rock fetch'.
That way the authors and reviewers can look at it in advance to
strengthen their case and afterwards also make an easier evaluation if
the review was fair and according to the documented criteria.
Also, checklist (2) has "A minimum goal for experimental mechanisms
proposed for widespread deployment in the Internet should
be that they do not perform significantly worse than TCP
in these environments."
However, 'these environments' refers to a non-exhaustive list of
scenarios. This checklist does not provide adequate information to
evaluate whether sufficient testing in the non-exhaustively defined
"difficult environments" has taken place or not. I do not believe we
can require assessments in various environments unless it's better
specified what which environmets that various refers to.
You are right that we may want to better enumerate things. But, we
cannot necessarily illuminate all such environments that one might
encounter. I hacked out the following text (to be added, not replacing
anything that is there now).
While it is impossible to enumerate all possible "difficult
environments", we note that the IETF has previously grappled
with paths with long delays [RFC2488], high delay bandwidth
products [RFC3649], high packet corruption rates [RFC3155],
packet reordering [RFC4653] and significantly slow links
[RFC3150]. Aspects of alternate congestion control that impact
networks with these characteristics should be detailed.
Is that better?
Yes, thanks.
What I want to avoid that when someone comes up with a proposal, folks
that are for some reason opposed to it will try to seek specific (but
globally not necessarily compelling) scenarios where the proposal
doesn't behave very well, and argue that that's a major shortcoming.
I think above should help on making the process more deterministic
though with some experience it could maybe go even further.
3) The first evaluation criteria includes ".. should be evaluated when
competing with standard IETF congestion control."
What is that standard IETF congestion control referred to? RFC 793
plus RFC 1122? These are the only two IETF _Standard_ specifications
I can think of. Or does this include proposed standards as well? What
constitutes "IETF congestion control" or "standard congestion control"
(in another place) that a CC algorithm should be evaluated against?
Please do not cite the TCP roadmap RFC on this. It's Informational
and not an IETF consensus statement, and as such has no authority to
define what constitutes "standard congestion control".
Added refs:
Proposed congestion control mechanisms should be evaluated when
competing with standard IETF congestion control
[RFC2581,RFC2960,RFC4340].
OK.
4) The first evaluation criteria also includes "We also note that this
guideline does not constrain the fairness offered for non-best-effort
traffic."
I don't understand your point here. All we are saying is that if---by
some means---we know that some flow is not best effort then it can be
subjected to some other criteria then it need not be constrained by the
guideline.
What I try to say is that I can't figure out how operationally and
practically this could be achieved. Do you have examples in mind how
how this could apply in some specific scenario?
If not -- and it isn't a practical concern right now -- maybe we
should not overengineer the BCP to address best-effort vs
non-best-effort at all.
If yes, there are some approaches to this, I'd be interested in
hearing how they'd work, but more even more than that, I'd require the
proposers of such a mechanism to provide an evaluation how reliable
that knowledge of best vs non-best-effort is.
What I'm afraid of is that the specification would say that it's
applied for traffic with certain DSCP values but typically that alone
is not IMHO a very reliable indicator of BE vs non-BE because DSCPs
can be rewritten and used for other purposes than those envisaged by
the proposer of the mechanism.
5) Normative references is empty, yet this is a BCP document which, among
other things, referred to "standard congestion control" without
providing a reference (as raised above). Please move some
informational references over here and/or add applicable references.
I have now put the references to RFCs 2581, 2914, 2960 and 4340 under
normative references.
(And, splitting the references was the dumbest thing we ever did---it
causes no end to the haggling.)
(Well.. Some will argue that the current IESG's policy requires
such a split. Some have interpreted that in certain classes of
Informational documents, a split may not be necessary, but I
believe Standards Track and BCPs do need it.)
2. Status
==> this title seems too short, and a somewhat longer and a more
descriptive one would be useful. Actually the section seems to be a
mix and match of different topics and a better organization might help
the document considerably. E.g., if there was a numbered list or
subsection of different "publication paths" and descriptions of each
the scope of the document and the intent of this section would likely
be much clearer.
I changed the title to "Document Status", but I am not inclined to
re-arrange it further because lots of people have read this now and
nobody has complained.
The major caveat here is that *I* hacked out the changes described
above. Sally may want to wordsmith or just plain disagree.
OK. I'd have liked to change it more but I realize it's a bit late in
the process and the other clarifications made above should help
already.
--
Pekka Savola "You each name yourselves king, yet the
Netcore Oy kingdom bleeds."
Systems. Networks. Security. -- George R.R. Martin: A Clash of Kings
_______________________________________________
Ietf mailing list
Ietf(_at_)ietf(_dot_)org
https://www1.ietf.org/mailman/listinfo/ietf