ietf
[Top] [All Lists]

Re: Genart last call review of draft-ietf-bmwg-ipv6-tran-tech-benchmarking-06

2017-04-29 02:50:06
Hello Robert,

Thank you very much for your detailed review.

We have revised the draft according to your comments.

The result was uploaded as:

https://tools.ietf.org/html/draft-ietf-bmwg-ipv6-tran-tech-benchmarking-07

Please find inline a punctual overview of the changes.

On 4/26/2017 7:05 PM, Robert Sparks wrote:
Reviewer: Robert Sparks
Review result: Ready with Issues

I am the assigned Gen-ART reviewer for this draft. The General Area
Review Team (Gen-ART) reviews all IETF documents being processed
by the IESG for the IETF Chair.  Please treat these comments just
like any other last call comments.

For more information, please see the FAQ at

<https://trac.ietf.org/trac/gen/wiki/GenArtfaq>.

Document: draft-ietf-bmwg-ipv6-tran-tech-benchmarking-06
Reviewer: Robert Sparks
Review Date: 2017-04-26
IETF LC End Date: 2017-05-02
IESG Telechat date: Not scheduled for a telechat

Summary: Essentially Ready for publication as an Informational RFC,
but with some minor issues to address before publication

This document is (with exceptions noted below) straightforward and
easy to follow

Minor Issues:

Section 8 is very confusing - I suspect it has been made so by
removing things that were in it earlier. Right now it claims to
provide additional tests, but the only content is about testing things
with firewalls, along with a statement that this document is only
targeting network devices that do not have a firewall function. I
think you can keep most of the text here (except the statement that
you aren't talking about things with firewalls) and remove the
confusion by changing the section heading to something like "Tests in
the presence of a firewall function".
- we have removed the second paragraph , which was indeed confusing.
The point, which may be clearer in Section 1.1 is that we need
additional benchmarks for stateful IPv6 transition technologies,
which may be (or not) labeled as having a firewall function.  
It's unclear how to apply the formula in Section 10.2.1 to the results
that come out of, say, Section 7.3.2, where you are reporting a
(minimum, median, maximum) tuple. Some discussion about the
applicability of the tests where you recommend against reporting a
single number to the methods in this section would help.
+ we added a note in the reporting format part stating: "

*In the case of benchmarks for which more than one value is reported
(e.g. IPDV Section 7.3.2), a column for each of the values SHOULD be
included.*

"
 It would also
help to point out that the Xpd result can be go negative (it will go
negative for things that become smaller as the number of flows
increase, and positive for things that get bigger). If I read this
correctly, if throughput (for example) goes to 0 as n increases, Xpd
will go to -100%. Similary if latency doubles as n increases, Xpd will
go to +100% (and will go to +200% if latency triples).
+ This was one of the important points which needed clarification. Thank
you for pointing it out.
In order to keep the  performance degradation results in the Lower is
better  direction, we proposed a secondary calculation formula for
higher is better benchmarks (e.g. Throughput). This makes, in our
opinion, the results easier to interpret.

Nits:

There are several places where you point to Section 6 where I think
you meant to point to Section 7. See the Procedure: line in 10.2.1 for
an example. Also make sure 9 is correct where you say "Sections 6
through 9" in section 12.
+Corrected

Please double-check that you meant "larger MTU" in the last paragraph
of 5.1. It might be correct, but I find the paragraph confusing.
We find this part to be correct. However, keeping only the 1280 +
overhead recommendation might make it less confusing.
Do you think that would be a viable solution?
In 8.1 you meant to point to 5.2, not 5.3

Please try to simplify this sentence: 
  "The duration of each trial SHOULD be at least 60 seconds to reduce
the potential gain of a DNS64 server, which is able to exhibit higher
performance by storing the requests and thus utilizing also the
timeout time for answering them."
+corrected

(Style comment - please feel free to ignore): Consider deleting "as
well" everywhere it occurs in the document - most of the places it is
used, the sentence works just as well, and sometimes better, without
it.
+corrected
Typos:
end of section 5.1.1: Appendix A1 should be Appendix A
+corrected

Best regards,
Marius

<Prev in Thread] Current Thread [Next in Thread>