perl-unicode

clearing the utf8 flag

2004-11-09 10:30:05

I have a program that reads and writes (among others) strings that
should be utf8 encoded.  I say "should", because somewhere deep
inside the dark corners of that program, sometimes, the utf8 flag on
a string is lost. (I'm still investigating where, tips to attack
such a problem are welcome.)

When writing the string, the program clears the utf8 flag
and writes a simple string of octets using:

    $s = encode("utf8", $s) if $s =~ /[^\x00-\x7f]/;
    $n = length($s);   # yes, we need length in bytes
    ...
    print $s;

Why would someone test for pure 7-bit strings instead of:

    $s = encode("utf8", $s) if Encode::is_utf8($s);

which seems superior to avoid double utf8 encodings,
should the utf8-flag be lost.  And it's faster.

Or even simply:     Encode::_utf8_off($s)

The problem is that I'm usually wrong.  Am I this time?
Am I missing something?  Or do I need more coffee?


--
Paul Bijnens, Xplanation                            Tel  +32 16 397.511
Technologielaan 21 bus 2, B-3001 Leuven, BELGIUM    Fax  +32 16 397.512
http://www.xplanation.com/          email:  
Paul(_dot_)Bijnens(_at_)xplanation(_dot_)com
***********************************************************************
* I think I've got the hang of it now:  exit, ^D, ^C, ^\, ^Z, ^Q, F6, *
* quit,  ZZ, :q, :q!,  M-Z, ^X^C,  logoff, logout, close, bye,  /bye, *
* stop, end, F3, ~., ^]c, +++ ATH, disconnect, halt,  abort,  hangup, *
* PF4, F20, ^X^X, :D::D, KJOB, F14-f-e, F8-e,  kill -1 $$,  shutdown, *
* kill -9 1,  Alt-F4,  Ctrl-Alt-Del,  AltGr-NumLock,  Stop-A,  ...    *
* ...  "Are you sure?"  ...   YES   ...   Phew ...   I'm out          *
***********************************************************************



<Prev in Thread] Current Thread [Next in Thread>