[Top] [All Lists]

Re: X-* header fields (Was: Getting 2822 to Draft)

2004-01-04 00:55:35

Pete Resnick wrote:

X-* fields are perfectly legal in 2822 (they fall under "optional-field"); they simply are not given the special treatment of 822, that they won't ever be official published extensions to the standard. The only difference then between 822 and 2822 in this respect is that 822 gives publication guidelines for extensions where 2822 does not. This has absolutely *no* effect on implementations of the protocol.

There is a minor effect. A user-defined field (per 822 definition) can be recognized as such by examining the first two octets of the field name, which is quite efficient. While there exist efficient methods of identifying fields (e.g., they are not quite *as* efficient as a two-octet (case- insensitive) comparison. Moreover, "X-" serves to differentiate user-defined fields from non-defined fields (i.e. those field names for which there is no IETF published definition or which the implementation
does not recognize).

Now, let me use this opportunity to explain *why* I don't think X-* fields should be in the standard in the first place:

The idea behind X-* fields (what 822 called user-defined fields) was so that folks could use new field names without worrying that they would interfere with the standard fields. That way, you didn't have to go through the trouble of publishing an RFC just to use a field that was only going to be internal to your particular system. The downside was that someone else could come along and potentially use the same field name in a completely incompatible way, but that was the risk that you took.

But since then, two things have made it clear that such user-defined fields are unnecessary and problematic:

1. Unnecessary: We now have IANA registration services. We could very simply create an IANA registry of e-mail field names. Then, if you wanted to create a new field, you just fill out the registration form and you're not only guaranteed that you won't interfere with published standard fields, you're also guaranteed that nobody else would use the field you chose in an incompatible way. And you wouldn't have to have a name that started with "X-".

Yes, but we don't yet have such a field name registry (and obviously we didn't have one in 2001 when 2822 was published). In order to be at all useful to implementors, registration would have to be contingent upon the existence of a stable, formal, public definition of the proposed field's syntax (with ABNF) and semantics. That differs from publication via RFC regarding review and comment; and I'm not convinced
that foregoing such review and comment would be a good thing.

2. Problematic: Some folks thought it was nice to have "X-*" fields for completely private use. But history teaches us that inevitably "X-*" fields *will* leak out onto the rest of the Internet. As several people in this thread have pointed out, we now have some "X-*" fields that are in widespread use. Sometimes, people use them very interoperably and they have served a fabulous purpose. Sometimes, they don't get used interoperably. It would be great if we could publish some standards to make sure that they do get used interoperably. But guess what: We can't! It is forbidden by RFC 822 to publish any standard for a field name that beings with "X-". No matter how widespread the use, no matter how useful the field is, it can not be standardized *by definition*. That decreases interoperability on the Internet.

It's difficult to see how such problems are either unique to X- or are more of a hindrance to interoperability than for other fields. As an example, consider "Status" which was in "private" use (I believe) by BSD "mailx" decades ago, and which is currently in use by several other MUAs, and which *does* leak out; there is also a formal definition of a "Status" field -- incompatible with the private usage -- defined as one of the delivery status notification fields (RFC 3464). So neither leakage nor incompatibility seem to be unique to X- fields. Note that if BSD mailx' author(s) had used X-Status for private use, there would be no conflict with the formal DSN
Status field.

Now, personally, I would love the document to say, "There is no good reason to use fields that begin with 'X-' as defined by RFC 822; any name that does not interfere with a currently registered or published field name is fine, and using an 'X-' field is discouraged because they can't ever be standardized." But there was widespread non-consensus in the DRUMS Working Group when this was discussed. 2822 does say "Extension header fields no longer specifically called out." In 2822bis, I think it would be fine to clarify that and say "User-defined header fields (those starting with 'X-') no longer specifically discussed", and even have an informational reference to a document (or 2) on why it was pulled. But I see absolutely no reason that this should stop 2822bis from moving to full Standard and obsoleting 822.

There is one very good reason to use X- for private or experimental use, viz. interoperability. Use of X- as a field name prefix for private or experimental use guarantees that there will be no conflict with a formal field name. Use of other names can lead to conflicts, as in the case of "Status", the damage in that case fortunately being limited by the fact that in that case, DSN fields are significant only in one part of a specific type of MIME multipart message and the
private use occurs only in the top-level message header.

An issue not mentioned above is migration of X- fields to a formal definition following successful experimental use. That obviously entails a name change. However name changes are not uncommon, e.g. refer to the MIXER RFCs, which define a number of fields whose names have changed. That merely means that parsers need to recognize one name as a
synonym for another.

I don't see the X- issue as any reason to keep 2822 from progressing; from a practical point of view (as noted by others), parsers will need to be able to handle the X- fields already in use, reading "old" messages still requires parsers to be able to handle RFC 822 syntax -- and that will continue to be the case even after 822 is eventually obsoleted regardless of what 2822 et succ. have to say about the matter, just as it is now necessary to support at least some RFC 733 and 724 constructs to be able to read really "old" messages even though those RFCs
have been formally obsolete for decades.