Tue, Jun 13, 2017 at 03:03:07PM -0700, Joe Touch:
Hi, all,
...
Title : The Harmful Consequences of
Postel's Maxim
https://tools.ietf.org/html/draft-thomson-postel-was-wrong-01
I completely agree with John Klensin that a test suite defines the
protocol standard (warts and all).
This may be, even if a pair of implementations were required company,
but is that wholly negative? The test suite is not stuck in time; it can
evolve to test things that its authors had not anticipated, its own bugs,
or evolution of the protocol itself. It is also more likely to catch bugs
and protocol design flaws than interoperability testing alone.
At the very least, an open test suite allows everyone to test against a
know baseline, before they test interoperation with other implementations.
For example, a rigid test suite that fails when an reserved field is not
its prescribed value, would be an asset for future development that
allocates the field.
Certainly the mentioned JSON problems would have benefitted from a test
suite and interoperability testing. Clearly, peer review and
interoperability testing are insufficient.