Ned Freed <ned(_dot_)freed(_at_)mrochek(_dot_)com> writes:
...
I think leaving the encoded word intact in this case is slightly
better. But only slightly. The bottom line is that if there's a problem
decoding (either because you don't know the encoding or the encoded
material is gronked somehow) or converting (either because you don't
recognize the charset or the material doesn't actually match the
charset), getting totally consistent and reasonable match behavior is
next to impossible. As such, any preference we express should be just
that: A preference. There's no best practice here as far as I can
tell.
How about the following for the second paragraph of 2.7.2:
Comparisons are performed in Unicode. Implementations convert
text from header fields in all charsets [HEADER-CHARSET] to
Unicode as input to the comparator (see 2.7.3). Implementations
MUST be capable of converting US-ASCII, ISO-8859-1, the US-ASCII
subset of ISO-8859-* character sets, and UTF-8. Text that the
implementation cannot convert to Unicode for any reason, MAY be
omitted, treated as plain US-ASCII (including any [HEADER-CHARSET]
syntax), or processed according to local conventions,
Thought I sent a note about this, but cannot find it... Anyway, I think this
is fine, although I'd be tempted to out "treat as plain US-ASCII" first on the
list.
Ned