--On Monday, April 29, 2013 09:55 +0100 Stewart Bryant
<stbryant(_at_)cisco(_dot_)com> wrote:
The question that people are asking is why the diversity of
the IETF leadership doesn't reflect the diversity of _the
IETF_.
The evidence seems to be that human's are terrible at
"guessing"
statistics, and the only statistics that are reliable as those
objectively gathered and subjected to rigorous statistical
analysis.
I mostly agree with this, but it means that attempts at
statistical measurement of populations we can't really
characterize are irrelevant. In particular, as soon as one
talks about the "diversity of _the IETF_", one is talking about
the participant population. There is no evidence at all, and
some evidences to the contrary, that the attendee population is
a good surrogate (approximation to a random sample, if you
prefer) for the participant population. Making that assumption
by polling or measuring the attendee function and assuming it is
representative of the IETF may introduce far more biases than
most of what we are talking about.
You can often see this in human assessments of risk. It is
also in the nature of statistics that you get long runs of
outliers, and
that only when you take a long view to you see the averages you
would expect. Again Humans are terrible with this, assuming
for example that a coin that comes up heads 10 times in a row
the assumption is that this is bias, and not a normal
statistical
variation that you would expect in an infinite number of
throws.
On the other hand, as a loyal empirical Bayesian, I suggest
that, if I observe a run of 10 heads and, as a result, bet on
the next toss being heads, I am somewhat more likely to carry
home my winnings at the end of the day that you are if you
continue to bet on a 50-50 chance no matter how long the run
gets... _even_ if the rules are normal statistical variation.
Now, after an infinite number of coin tosses occur, you may be
proven correct, but part of the reason for that Bayesian
judgment (or a judgment based on moving average properties of
the time series) is that few of us are going to be able to wait
for that infinite number of tosses.
It would be useful to the discussion if we could see data on
diversity
that was the output of a rigorous statistical analysis. i.e.
one that
included a confidence analysis and not a simple average in a
few spot years.
I agree. But I also suggest that humans are pretty good at
binary comparisons and some longitudinal relationships that do
not involve population samples. For example, with no effort to
compare the population statistics of the IESG with the
population statistics of the IETF (the precise comparison that
is most susceptible to the statistical problems both of us are
concerned about), it is easy to look at IESG membership
longitudinally and observe that, between the early 1990s and
2010, there were always at least one, and often two or three,
women on the IESG. Since then, zero. Now, based on around 17
years of moving average, I feel somewhat justified statistically
in believing that something odd is happening.
I would feel much more justified if we went a couple years more
with no change in our procedures and how we think about things
and the "zero women" trend continued, but that illustrates the
other problems with this sort of analysis and an attempt to base
it on population statistics, especially the population
statistics of experimental design. First, our having these
discussions have, I believe, already increased sensitivities to
the issues and maybe even how the community thinks about it. If
we end up with a woman or three on the IESG a year from now, it
will basically be impossible to know whether that was
-- simply a return to normal behavior after a period of
deviation that could be attributed to statistical
variation or
-- whether it was because this discussion was
effectively a consciousness-raising exercise that
changed how decisions are made.
The second issue is that, as in a clinical trial in which it
becomes obvious (with all of those subjective human judgments as
well as strict statistical ones) that one of the treatment
groups is doing much better than others, it may be socially and
morally unacceptable to continue the experiment in order to get
cleaner statistical results.
--On Monday, April 29, 2013 06:14 +0000 Christian Huitema
<huitema(_at_)microsoft(_dot_)com> wrote:
Certainly useful, but it is easy to inject one's own bias into
such processes, and to overlook other factors. I may be
biased, but I have the impression that the largest source of
bias in IESG selection is the need to secure funding for the
job, which effectively self-select people working for large
companies making networking products.
Or at least large companies and mostly those with a significant
stake in the Internet. I agree with this impression. In
principle, we could separate gender (or other) bias from this by
comparing those who are willing to be candidates with those who
are selected. I don't think that works in practice unless we
require people to get binding company approval before agreeing
to be candidates. We have never done that. Ignoring the other
bad effects it might have, it is a rational strategy for many
people to agree to be candidates after only a minimal discussion
with their organizations and then to have the serious discussion
only after the Nomcom's "please confirm that you are still
really available" note arrives.
Gender may be the least
of the problems there; there are other dimensions of
diversity, e.g. academic vs. industry, network equipment
versus internet service providers, software versus hardware,
etc. Only a fraction of these segments can afford to have
someone working full-time on the IESG. Now, having to work
full time is a bit much for a volunteer position, and we may
want to consider ways to remedy that.
Even within a company, there may be biases about who gets the
approval. On the gender issue, we might be seeing symptoms of
something that is actually positive for the women engineers
although bad for the IETF. A series of analyses[1] suggest
that, in standards bodies that are mostly working on mature
technologies, few companies consider it a good investment to
send design-capable people with good management skills to
standards meetings, much less to have them commit to standards
management activities. When the technologies are less mature,
it makes more sense for organizations to commit design and
implementation talent for standards development and even for
standards management. Marshall Rose memorably suggested that
those trends and others led to SDOs that contain a lot of
"Goers" rather than a lot of designer-implementer "Doers".
If one were to hypothesize (I have no idea whether this is true)
that many of the women with design engineer capabilities who
attend IETF early in their careers have those talents recognized
by their companies and end up in positions that make it
unattractive to send them to the IETF --much to contribute all
of their time for a minimum of two to four years-- then we just
don't see them on the IETF. Even if an exactly equal number of
incoming men have the same talents and the same recognition and
promotion pattern, the smaller number of incoming women would
predict an IESG that was largely or entirely men.
Another possible corollary to Marshall's analysis and the
earlier studies is that the leadership of any SDO will gradually
converge on entirely Go-ers or "professional standardizers",
simply because they are less expensive --in terms of loss of
skilled time for an extended period as well as more direct
costs-- than design or development talent. That particular
issue isn't related to most of what we've been discussing as
"diversity" but it may be another very important and related
issue. I don't think the IETF is there yet but I've wondered if
we are on the downhill slope.
best,
john
[1] These analyses go back long before Rose contributed the much
more colorful interpretation and vocabulary to the discussion
that I've invoked above.