For backward compatibility, application/xml and text/xml MAY,
but SHOULD NOT, also be used for "external parsed entities",
"external DTD subsets", and "external parameter entities".
I don't think "MAY but SHOULD NOT" is a valid state in RFC 2119
terminology, or called for. How about:
application/xml and text/xml MUST NOT be used for "external parsed
entities", "external DTD subsets", and "external parameter entities".
Note that RFC 2376 (obsoleted) allowed this usage, although
in practice it is likely to be rare.
The justification
... It is generally considered bad form to invalidate
implementations that made a good faith effort to conform to the previous
form of the standard.
isn't consistent with policy or practice; from RFC 2026
section 4.1.1:
A Proposed Standard specification is generally stable...
However, further experience
might result in a change or even retraction of the specification
before it advances.
More often than not, people just fix specs that are broken, and pay
attention to backward compatibility when it affects deployed software,
not theoretical compatibility.