perl-unicode

Re: removing accents

2004-01-03 18:30:05
Le 3 janv. 04, à 15:49, Jarkko Hietaniemi a écrit :

I'm afraid, the process of taking NFD followed by removing \pM characters (remove_accent() as below) would remove marks other than accents too much.

Say, it replaces '≠' (U+2260, <NOT EQUAL TO>) with '=' (<EQUALS SIGN>)
since a mathematic "negation slash" is encoded by U+0338
 <COMBINING LONG SOLIDUS OVERLAY> which is to be removed.

Also, although they are not accents, it's unclear (and quite language-dependent)
what should be done with ligatures.

Thanks to you both for your replies. I did some more research
and found that even removing accents is locale dependant.
I reverted back to my carefully crafted tr()s... Incidentally
much faster than the Unicode::Normalize / remove \pM approach.

--
Eric Cholet

<Prev in Thread] Current Thread [Next in Thread>