ietf
[Top] [All Lists]

RE: [Gen-art] Re: Gen-art review ofdraft-hartman-mailinglist-experiment-01.txt

2006-05-09 06:39:14

There are certainly some major weaknesses in the current IETF mailing list
management procedures, and those weaknesses are very well-described in this
document.  I agree that we need to address those weaknesses.  However, I am
not sure that I agree with the fix proposed in this document.

This document defines an RFC3933 experiment in which we would temporarily
give the IESG the authority to create new mailing list management procedures
and enact them.  The only hard limitations on this authority are that
posting suspensions cannot run beyond the timeframe of the experiment (18
months), nor can the procedures prevent anyone from reading an IETF mailing
list.  The document does not even limit the type of action that the IESG can
take -- while it only talks about posting rights suspensions, this document
would allow the IESG to define an enact other types of mailing list control
at their discretion.  It explicitly does not require that we use the same
procedures on all IETF mailing lists, as it explicitly allows the IESG to
define different procedures for different lists.  

This document does not define any principles that the IESG should follow in
determining mailing list management procedures, nor does it require any type
of community review or consensus to enact them.  Although it encourages an
IETF Last Call for new procedures, it does not require an IETF Last Call or
IETF consensus to establish a new procedure.  It doesn't even require that
the IESG send a new procedures for community review before enacting it
and/or that they explicitly notify the mailing lists to which the new
procedure would apply.  It merely requires that the new procedure be
published on a public web site.

At first, I thought it was the purpose of this document to allow the IESG to
try out different mailing list management procedures on different IETF
mailing lists for a short period of time, with the goal of picking some
successful procedures that would later be discussed by the community and
potentially reflected in our BCPs.  In other words, I thought that this was
a temporary measure to address the weaknesses in our current procedures and
get some experience with alternatives.  I still thought that the goal would
be to settle on well-defined, community-consensus-based procedures by then
end of this 18 months.  At that time, I supported this document, because I
saw it as a better alternative than living with our broken procedures until
the community could fix them.  I thought that the IESG could end this
experiment if/when we had community consensus on a new set of procedures.  

However, at the last Gen Area meeting, it became clear that if this
experiment is successful, the goal would be to permanently delegate this
level of discretion to the IESG, rather than publishing a specific set of
mailing list management procedures in BCPs.  Our mailing list management
procedures would permanently be defined in a web page that would be
maintained and modified by the IESG, rather than in BCPs that are determined
through a community consensus process.  

It is the nature of RFC 3933 experiments that they are intended to be
adopted as permanent procedural changes if they are found to be successful.
However, this document does not define any criteria for success.  RFC 3933
says:  "At or before the end of the "sunset" timeout, the IESG would either
revise (or cause to be revised) the document into a BCP RFC or the procedure
would expire..."  So, I _think_ what would happen at the end of this
18-month period is that the IESG would decide whether delegating this
authority to themselves was a useful procedural change and, if so, they
would publish a revision of this document as a BCP, permanently delegating
this authority to the IESG.

So, there are two questions I think we need to answer:  

(1) Is the community comfortable with the idea of giving the IESG such wide
discretion to establish and modify our mailing list management procedures? 

(2) Is the proposed experiment well-enough documented that we will be able
to clearly understand whether or not it was successful?

Personally, I would not be comfortable delegating this level of authority to
the IESG.  The former IESG's decisions in this area did not align with my
own principles, and I am not certain that the new IESG would balance
expediency and openness in a way that would be acceptable to me.  I am also
uncomfortalbe with the idea that something as important as our mailing list
management procedures would be documented on a web site that could be
modified without community review and approval.  I understand that I may not
be representative of the larger community in this regard, but I would much
prefer that we fix the problems with our current BCPs through a community
consensus process, resulting in updated BCPs that define our mailing list
management procedures.

I also have a smaller, but more concrete issue with this document.  IMO,
this document doesn't define a well-formed experiment.  In particular, it
doesn't define any criteria or method that that the IESG should use to
determine if the experiment has been successful.  Without such criteria, how
would the IESG decide whether or not to delegate this authority to
themselves on a permanent basis?  It has been recent IESG policy not to
publish experimental RFCs without well-defined success criteria, and I do
not think that this document meets that requirement.

Margaret 



-----Original Message-----
From: Sam Hartman [mailto:hartmans-ietf(_at_)mit(_dot_)edu] 
Sent: Monday, May 08, 2006 2:00 PM
To: Harald Alvestrand
Cc: IETF Discussion
Subject: Re: [Gen-art] Re: Gen-art review 
ofdraft-hartman-mailinglist-experiment-01.txt

"Harald" == Harald Alvestrand <harald(_at_)alvestrand(_dot_)no> writes:

    >> 
    >> The primary reason you want to encourage 
meta-experiments is that a
    >> lot of the hypotheses you want to test involve delegation.  For
    >> example I want to test the hypothesis that the right 
way to solve the
    >> mailing list mess is to delegate it to the IESG.  I 
could delegate it
    >> to the IESG as part of an experiment along with a 
initial procedure.
    >> But if I do that I'm testing a different hypothesis: 
is delegating
    >> something to the IESG with the whole IETF designing the initial
    >> conditions a way to solve the problem.  As you might 
imagine that's a
    >> different hypothesis.
    Harald> I do not believe that is a correct statement.
    Harald> An RFC 3933 experiment by its very nature 
involves three different
    Harald> sub-experiments:
    Harald> - Whether you can get the IETF to agree to 
running this experiment
    Harald> - Whether the bodies that administer the 
experiment can perform the
    Harald> experiment so that we have something to measure
    Harald> - Whether the experiment produces useful results

I agree with this characterization.

I don't see how this characterization is in conflict with mine.


    Harald> I think what you are proposing will lead to 
failure of the *first*
    Harald> experiment, which means that you won't get to the 
other two.
    Harald> If you say "give the IESG the power to produce 
procedures, we're not
    Harald> going to give you even a hint as to what these 
procedures might be",
    Harald> this is what I would characterize as selling a 
pig in a poke, and it's
    Harald> perfectly reasonable for the IETF community to 
ring up a "no sale" on
    Harald> the consensus register.

The IETF certainly may do that.  You clearly believe the IETF 
should do that.  I believe the IETF should not do that.
Ultimately Brian will have to make a consensus call.

I think it is reasonable to delegate comuing up with 
procedures to the IESG.  I think that if we're going to 
delegate that then the procedures should not be discussed in 
this draft.
 If the community wants more oversight, here are some things 
I think would be reasonable to ask for:

* Add criteria the IESG will use for evaluating procedures or for
  creating procedures.
* Requiring proponents of this experiment to submit candidate 
procedures that demonstrate procedures are possible
* Adding explicit review requirements for procedures that the 
IESG proposes.
* Establishing principles the IESG should use

If you believe that one of these options would help, I invite 
you to propose text or direction.

You could also propose an experiment to run with specific procedures.
I would not object to that experiment but I would also not be 
interested in contributing to it.


    Harald> The theory that there is a different hypothesis 
being tested if the
    Harald> IESG sets the initial procedures than if the 
initial procedures are
    Harald> part of the experiment also seems doubtful to me; 
in both cases what I
    Harald> think will happen is that the IESG will start off 
with procedures
    Harald> designed by Sam Hartman, and the big difference 
is that the community
    Harald> will have seen the initial procedures before 
deciding to run the
    Harald> experiment.
    Harald> The whole IETF has never designed anything and 
never will; all design
    Harald> is done by specific people, and (hopefully) 
reviewed by some
    Harald> large-enough fraction of the IETF. That's how 
EVERYTHING in the IETF
    Harald> works.
    >> Since I'm on the IESG I'm actually in a
    >> reasonably good position to negotiate an initial 
procedure that the
    >> IESG will be happy with and that would be similar to a 
procedure the
    >> IESG would come up with on its own.  However we want 3933
    >> experiments--even experiments delegating things to the 
IESG--to be
    >> documents that anyone can write.  So we should require 
that authors of
    >> 3933 experiments demonstrate stakeholder buy-in for 
experiments but
    >> not require that they take actions as if they were the 
stakeholders.
    >> 
    Harald> I do not believe this theory either.

So, the main point of the paragraph here is that I don't 
believe only IESG members should be able to propose 
experiments that change things for the IESG.  I hope we can 
agree on that.

    Harald> RFC 3933 talks about the IESG and the community. 
The "stakeholder"
    Harald> concept is of your introduction, and I believe it 
serves no benefit.
    Harald> An RFC 3933 proposal author HAS to do three things:

    Harald> - Write up what he's proposing in enough detail 
that it can be evaluated
    Harald> - Convince the IESG that the experiment has 
enough merit to warrant a
    Harald> Last Call
    Harald> - Generate enough review and comment in the 
community that the IESG
    Harald> can confidently say that there's community 
consensus to run the
    Harald> experiment

I proposed the concept of stakeholder buy-in as an attempt to 
solve the first criteria.  I think that the IESG should 
almost always last call experiments when the proponets can 
demonstrate it is possible to do the work and when those 
effected are interested in doing the work.
I think that using this criteria will help resolve 
long-standing questions of where to focus process reform 
energy and of what people need to do in order to move their 
proposals forward.  I think that the IESG should focus on 
process reform where the stakeholders are interested in the 
work over process reform where the stakeholders are not interested.

I do agree that when the community really desires some reform 
and the stakeholders are not interested, we need some 
resolution that allows the reform to proceed possibly with 
resignations.  I think the bar in that case needs to be much higher.

But my concept of stakeholders is more about establishing a 
criteria for forward progress than about blocking things.

    Harald> Certainly, a community feedback that says "I'm 
one of the people
    Harald> required to do work under this experiment, and 
I'm not going to do it"
    Harald> is a strong statement that needs consideration 
when assessing whether
    Harald> there's IETF consensus to run the experiment. But 
sometimes that's
    Harald> just part of the "rough" in rough consensus, even 
when deciding to run
    Harald> the experiment will cause someone to resign his 
role. And sometimes
    Harald> it's not.
    >> The second reason that you want to allow 
meta-experiments is that we
    >> want to encourage RFC 3933 as the first step in process change.
    >> Process change often results in BCPs.  You want the 
3933 experiment to
    >> be reasonably similar to a BCP so that when 
appropriate you can easily
    >> convert a successful experiment into a BCP.  You would probably
    >> replace any evaluation criteria with results of the evaluation,
    >> replace the sunset clause with something else.  
However you want the
    >> operative language to remain the same.  A significant 
result of the
    >> mailing list discussion is the concern that our BCPs 
are too specific
    >> and encode operational details.  If you require that 
meta-experiments
    >> are not allowed you strongly push us in the direction of
    >> overly-specific BCPs.  I think that would be a very bad idea.
    >> 
    Harald> Here I agree, but disagree that this document 
does an appropriate job of it.
    Harald> I think the separation of "principle" and 
"process" is a good thing -
    Harald> the principle of this document being (I think) 
that the IESG sets
    Harald> mailing list management procedures, and the 
process being the process
    Harald> by which such procedures are set, and the third 
level being the
    Harald> procedures themselves.

    Harald> However, just stating the principle alone at an 
abstract level is not
    Harald> enough to either start or evaluate the experiment.
 I disagree.  If you'd provided more detail on why you don't 
think that this document is sufficient, I could respond to 
it.  However I think it might be useful to see what the 
consensus position is before we spend a lot of time on this issue.



--Sam

_______________________________________________
Ietf mailing list
Ietf(_at_)ietf(_dot_)org
https://www1.ietf.org/mailman/listinfo/ietf



_______________________________________________
Ietf mailing list
Ietf(_at_)ietf(_dot_)org
https://www1.ietf.org/mailman/listinfo/ietf