Andrew Gierth writes:
compatibility with the real world is important too.
In the real world, practically all messages that look like UTF-8 (i.e.,
that follow the UTF-8 byte-sequence restrictions) are, in fact, UTF-8.
In the real world, displaying a message as UTF-8 if it looks like UTF-8
does not lead to any user complaints, and produces immediately visible
benefits for some users.
In the real world, many users are acutely aware of the variability of
character encodings, and are quite happy to hear about software settling
on a universal character encoding.
---D. J. Bernstein, Associate Professor, Department of Mathematics,
Statistics, and Computer Science, University of Illinois at Chicago