Sigh. I'd hoped we could avoid this discussion, but it seems like
people can't resist focusing on it to the detriment of attention to more
substantive issues. (Or maybe this really is the most substantive issue
in 2822, and we should regard this debate as a sign that we don't have
anything worse to worry about?)
The idea behind X-* fields (what 822 called user-defined fields) was
so that folks could use new field names without worrying that they
would interfere with the standard fields. That way, you didn't have
to go through the trouble of publishing an RFC just to use a field
that was only going to be internal to your particular system. The
downside was that someone else could come along and potentially use
the same field name in a completely incompatible way, but that was
the risk that you took.
I won't claim that I can make statements about the original idea behind
X- fields, since I wasn't in on the discussions at the time. However
there are other useful properties of X- fields that derive from the 822
specification, specifically that the use of X- serves as a notice that
the field is nonstandard (and thus it hasn't received the review and
scrutiny that a standard would have received), that it never will be a
standard, that IF it somehow becomes widely used there is a good chance
it will not be consistently used, and that you should therefore avoid
assuming that an X- field means what you think it means.
These properties *are* useful for flagging experimental or local
features. And it's useful to have such a flag precisely because such
fields will leak outside of their experimental or local environment.
The fact that these fields leak should not cause a problem as long as
(a) the extra fields don't change the meaning of other fields and (b)
the software that creates the extra fields doesn't assume that either
recipient's UAs or intermediares will understand those fields. (These
rules should generally be followed by all header fields, not just
extension fields that begin with X-.) But IF you follow the 822
specification, the fact that a field-name begins with X- is an
indication that you shouldn't expect it to work on a widespread basis,
and that's a useful indication.
But this does mean that X- fields shouldn't be used by widely-deployed
software, except perhaps for "write only" fields like X-User-Agent
that mostly serve as comments or indications to human recipients.
But since then, two things have made it clear that such user-defined
fields are unnecessary and problematic:
1. Unnecessary: We now have IANA registration services. We could very
simply create an IANA registry of e-mail field names. Then, if you
wanted to create a new field, you just fill out the registration form
and you're not only guaranteed that you won't interfere with
published standard fields, you're also guaranteed that nobody else
would use the field you chose in an incompatible way. And you
wouldn't have to have a name that started with "X-".
Even when we have header registration, this won't make X- fields
unnecessary. Sometimes it's just too much trouble to get a field name
registered, especially if it's just for some experiment or local
feature.
That, and it's too easy to just code new field names into software
without registering them. If you're writing code and you find that
you need a new field, are you really going to take the page fault
to get that field name registered, or are you just going to type
printf ("new-field-name: whatever\r\n"); and be done with it?
If we give implementors a convention that let's them say "this field
isn't intended for widespread use" then at least some of those
implementors will be responsible and use that convention, which is far
better than not having a convention at all and encouraging implementors
to allocate field names on a whim. (URI prefix space is a huge mess
because implementors have done precisely that.)
And if we really registered every field name for every
local or experimental extension, we'd soon find that the field names we
wanted to use for extensions that are worthy of standardization, were
already allocated by half-baked ideas not worthy of standardization.
Yet the half-baked ideas would have the appearance of legitimacy, since
they'd have the "better" or "more obvious" names. We'd also lose the
useful ability to mark extensions as "local" or "experimental".
(though this could be accomodated by a different naming convention)
2. Problematic: Some folks thought it was nice to have "X-*" fields
for completely private use. But history teaches us that inevitably
"X-*" fields *will* leak out onto the rest of the Internet. As
several people in this thread have pointed out, we now have some
"X-*" fields that are in widespread use. Sometimes, people use them
very interoperably and they have served a fabulous purpose.
Sometimes, they don't get used interoperably. It would be great if we
could publish some standards to make sure that they do get used
interoperably. But guess what: We can't! It is forbidden by RFC 822
to publish any standard for a field name that beings with "X-". No
matter how widespread the use, no matter how useful the field is, it
can not be standardized *by definition*. That decreases
interoperability on the Internet.
The cases where you really want to standardize a field whose name begins
with X- are rare, because it's rare that someone manages to define a new
field in such a way that it's suitable for widespread use without
benefit of a published spec and wide review before deployment of that
feature. But I don't read RFC 822 as preventing publication of
information about X- fields. Indeed RFC 2076 does mention some X-
fields, so clearly it's possible to publish information about X- fields
with Informational status.
For most cases where the X- field turns out to really be what we want to
standardize, what we should do is publish a new standard with a new
field name, document the previous usage in a non-normative section of
the standard, and describe how senders and receivers should cope with
the two fields.
But if there were a case where there was a compelling reason to
standardize the use of an existing X- field, then I wouldn't object to
that specific field being treated as an exceptional case. We should do
the right thing for interoperability and functionality even if it wasn't
what 822 anticipated. But we should make it clear that this is an
exceptional case and not the general rule. If we make X- fields the
same as any other fields then we are optimizing for the rare case, and
at the same time doing harm to our ability to extend the mail protocol.
Keith