Actually this represents currently accepted engineering practice in the
usability field.
According to Nielsen you can identify 80% of issues with small scale
studies of 5 people. The point here is not to create a statistically
representative sample, it is to identity the chief issues that may be of
concern.
You should always do a pre-survey before you do the main run or else you
will fail to ask the right questions.
Now I do have a big problem with the practice of publishing 'scientific
studies' on the basis of small sample sizes. You can get a paper
published in the security usability field on the basis of three samples
of 9 persons each. That is actually one of the larger studies. But that
is a different matter and in any case they have rather more
methodological issues to deal with than sample size.
-----Original Message-----
From: ietf-bounces(_at_)ietf(_dot_)org
[mailto:ietf-bounces(_at_)ietf(_dot_)org] On
Behalf Of Eric Gray
Sent: Thursday, May 08, 2008 8:09 AM
To: Mark Nottingham
Cc: ietf(_at_)ietf(_dot_)org
Subject: RE: IETF Website Redesign Effort
Mark,
Unfortunately, the way I read this, it is effectively
not the latter. The impression I am left with is that 10
volunteers are being asked for feedback on changes. Assuming
this is an iterative process, their collective feedback could
have almost any arbitrary degree of impact on the final
result - depending on how well developed the current plan is
and how responsive the change makers expect to be to feedback.
Hence the value of Brian's observation that there
should be some effort to determine if the general outline of
planned changes is something that most people agree to.
Also, as has been politely hinted at already, the
sample selection method is likely to "poison" the sample.
Why the first 10 starting from a specific message at a
specific time convenient to only a narrow set of time zones?
Why only 10? Why publish the names of the
victim^H^H^H^H^Holunteers? None of these seem to be the sort
of factors one would like to see in getting a reasonable
cross section of the (probably) tens of thousands of people
who use the IETF website on a regular basis (where is that
visitor counter, or the web-site at the moment, anyway?).
--
Eric Gray
Principal Engineer
Ericsson
-----Original Message-----
From: ietf-bounces(_at_)ietf(_dot_)org
[mailto:ietf-bounces(_at_)ietf(_dot_)org] On
Behalf Of Mark Nottingham
Sent: Wednesday, May 07, 2008 9:53 PM
To: Russ Housley
Cc: ietf(_at_)ietf(_dot_)org
Subject: Re: IETF Website Redesign Effort
The message on-announce was worded as if these people would be
*performing* the redesign, but I read your message below as saying
that they'll serve a a pool of testers.
I sincerely hope the truth is closer to the latter...
On 08/05/2008, at 6:11 AM, Russ Housley wrote:
Stephane:
We are inviting the first ten (10) interested members
of the IETF
community who respond to this email to become a part of
the website
redesign team. If you are interested in assisting with
this effort,
please respond to this email as soon as possible.
It is no longer "rough consensus and running code", it is
now "quick
click and no SMTP latency".
This approach was taken at my suggestion.
These early reviewers are so that the Secretariat can get some
feedback about the new design without have a large number
of people
pile on. Think of it as a product test group. Once
these folks are
comfortable that the new design is an improvement, there
will be an
opportunity for broader comment.
Russ
IETF Chair
_______________________________________________
IETF mailing list
IETF(_at_)ietf(_dot_)org
https://www.ietf.org/mailman/listinfo/ietf
--
Mark Nottingham http://www.mnot.net/
_______________________________________________
IETF mailing list
IETF(_at_)ietf(_dot_)org
https://www.ietf.org/mailman/listinfo/ietf
_______________________________________________
IETF mailing list
IETF(_at_)ietf(_dot_)org
https://www.ietf.org/mailman/listinfo/ietf
_______________________________________________
IETF mailing list
IETF(_at_)ietf(_dot_)org
https://www.ietf.org/mailman/listinfo/ietf