ietf
[Top] [All Lists]

Re: Meeting Venue Preference Survey

2010-08-30 10:23:18
Folks,

We really need to get these surveys produced by someone with training in survey design. The intent of the survey is quite reasonable, but that the construction of it is not. Survey's are quite sensitive to wording and context. This suffers from serious problems with both.


From the survey:

2. Meeting Preferences

1. It is important to me to have the meeting venue be easy to get to:
It is important to me to have the meeting venue be easy to get to:
> Very unimportant
Slides

What does "slides" mean? I'm guessing it's an extraneous entry, since it throws off the apparent model of a balanced 5-choice set of responses.


2. Do you prefer a meeting in a gateway city,

I believe the underlying problem with this question, as demonstrated by the postings about it so far, is the lack of consistent criteria for defining "gateway" and "secondary".

I'll offer the view that a "gateway" city is a principle hub of international air travel, while a "secondary" city should have at least some international air access. I think that's a useful distinction, but it means that more than one of the examples of secondary, in the survey, really would be classed as tertiary or worse, and there's a reasonable chance that Vancouver would count as primary.

At the least, please clarify the criteria for secondary.

I should note that it's probably still possible to get useful data from that survey question, simply based on respondents' subjective reactions to the terms gateway and secondary. Over the years, including recently, there's been enough chatter about the basic distinction to make the specific lists of cities secondary. Just knowing folks' preferences between gateway and 'other' might be helpful. That said, "primary hub" might be a better choice than "gateway"; I would not be surprised to find some inconsistency in the meaning different people impart to the word.

(There's also some question about sampling for this survey. The main ietf list is widely subscribed to, of course, but not as widely as this survey ought to target. I suggest sending the notice also out to ietf-announce, at the least. Perhaps some other lists, such as for nanog, apnic, and ripe...)


3. Do you prefer going back to venues or trying new venues?

As phrased, this question probably biases responses toward 'new', since they sound more interesting, and possibly biases it strongly.

Presenting a statement of implications about the tradeoffs -- e.g., risks of new, reliability of returns -- would have set the stage for the response much better.


5. Would you be willing to pay a higher registration fee to have the meeting in 
a gateway city?

This is a fundamentally biased (distorted) question. It is predicated on a factual assertion that is unsubstantiated and very probably false.

Gateway cities have many more travel choices and many more lodging choices. This very probably means that total travel costs can be /lower/ than for secondary cities. At the least, this means that the relationship between cost and city 'class' is an open question.

Further, the registration fee is only one of a set of costs. What is important is the total cost, not just the narrow, localized registration fee.

The set of responses provided also is rather oddly constrained.



8. Would you attend if we held the IETF in Africa?
9. Would you attend if we held the IETF in South or Central America?

This is yet another example of a question lacking foundation. What is the basis for having a meeting in a region that produces few IETF active participants? Perhaps the reason is compelling. But a question like this, lacking any premise, can only get a random sampling of spontaneous reactions. And given the way humans provide such reactions, the odds are high that repeating the survey in a month would produce different answers to this question.

d/















--

  Dave Crocker
  Brandenburg InternetWorking
  bbiw.net
_______________________________________________
Ietf mailing list
Ietf(_at_)ietf(_dot_)org
https://www.ietf.org/mailman/listinfo/ietf