You’re entitled to your opinion, but I entirely disagree. I thought each
of those made an important point and highlighted some areas where consensus
is broadly held. I appreciated Russ’ composition of the issues and think
he deserves our thanks.
-Tim
On Wed, Nov 6, 2013 at 6:23 PM, Dave Crocker <dhc(_at_)dcrocker(_dot_)net>
wrote:
Folks,
An IETF hum is a method of surveying a group for its views. Unfortunately
the hums that were taken at the end of this week's IAB plenary do not
permit any meaningful interpretation.
Here's why...
Surveys are extremely sensitive to the phrasing of the questions, the
phrasing and range of the response choices, the sequencing of the
questions, and the context of the asking. Get any of these wrong and you
can get the wrong information, or even just the appearance of information
-- that is, misunderstandings -- but nothing actually useful.
A common response to such a concern is "well, at least we'll get some
answers", but that's like saying "well, at least we'll get some noise."
The fact that the noise is misunderstood to be signal does not actually
make it signal.
The different phrasings of a question can produce very different
understandings by responders. The challenge is to formulate a question
that is likely to be interpreted similarly amongst responders (and the
person asking.) It's also a challenge to ask a question that captures
something that is actually meaningful (and was intended) rather than merely
sounding good.
The offered response choices can bias the responses. A set of choices
like (Good, Excellent) obviously leaves out (Bad, Don't Care, Don't Know.)
Or they can have bias in their phrasing by making some choices more or
less appealing (Could be better, Excellent), rather than equivalent
vocabulary in tone (Bad, Good). So it's a challenge to make sure that
choices cover the proper range and with equanimity to the alternative
choices.
A sequence of questions also needs to be carefully orchestrated. For
example today's questions took as a given that surveillance is an attack.
Due diligence might expect establishing that relationship explicitly. And
yes, it is possible that some IETF attendees do not see it as an attack.
Another example of sequencing is dealing with subtleties and complexities.
For example some anti-surveillance mechanisms are certain to defeat
popular operational management mechanisms. Do we care about the tradeoffs?
Lastly, environmental context can encourage or discourage candor. Examples
include the genders of the asker and respondent, any relationship they
might have, or the presence of others. Would you really provide candid
answers about possible problems with your sex life when being asked with
your partner present? Amongst a group of co-workers? Your parents?
The hums asked at the plenary were problematic along each of these lines.
The first question was theatre, essentially making the context political.
By way of example, note the difference between what was asked:
The IETF is willing to respond to the pervasive surveillance attack?
which has loaded language with 'pervasive' and 'attack', versus a more
neutral and purely technical question meant to cover the same basic concern:
The IETF is willing to improve its specifications to be more resistant
to surveillance?
But this isn't exactly a balanced question either. By that, I mean that
the answer really is already known. A good question is one that has a
chance of getting some support for each choice. So perhaps a better
example would be:
The IETF is willing to require adding resistance to surveillance to
all of its protocols?
The questions typically also did not offer "don't know" or "don't care"
choices. Some folk probably knew that they don't know enough yet, limiting
their ability to support the kinds of questions being asked.
The IETF's doing anything privacy-related that is useful is going to
require considering tradeoffs and some of those tradeoffs might reduce the
utility of a service. So the actual choices that will be made might turn
out to be quite different from what was implied by the dominant answers to
the plenary questions.
And lastly, consider carefully the context of the room and ask whether
everyone actually felt completely free to give a "no" hum to the initial
questions. I suggest that the emotions of the room created a strong bias
against no's. Maybe not for you. Maybe not for me. But probably for
many of the folk sitting near you.
We now find ourselves with a set of hums that appears to establish a
direction but which can't survive even basic analysis, as the later
postings on the ietf mailing list demonstrate.
Here's what I suggest: A single, simple, conceptual question that
supplies all of the 'guidance' we can legitimately offer, at this stage:
The IETF needs to press for careful attention to privacy
concerns in its work, including protection against surveillance.
[ ] No
[ ] Yes
[ ] Don't Yet Know
[ ] Don't Care
d/
--
Dave Crocker
Brandenburg InternetWorking
bbiw.net