perl-unicode

Re: Encode utf-16 problem

2002-12-02 19:30:07
On Tuesday, Dec 3, 2002, at 11:12 Asia/Tokyo, Jarkko Hietaniemi wrote:
Why the 'Partial character' warnings?  I would have though the input
files are just right.  Also, the warnings are given to stderr
unconditionally, I would have to redirect stderr to /dev/null to get
rid of the warnings.

$ perl -le 'print pack("v*", 0xFEFF, unpack("C*", "test"))' >! utf16
$ hex utf16
ff fe 74 00 65 00 73 00 74 00 0a                ..t.e.s.t..
$ ./perl -Ilib -e 'open(FH, "<:encoding(utf16)", "utf16");$a=<FH>;print $a'|hex
UTF-16:Partial character at -e line 1.
UTF-16:Partial character at -e line 1.
74 65 73 74                                     test
$ perl -le 'print pack("n*", 0xFEFF, unpack("C*", "test"))' >! utf16
$ hex utf16
fe ff 00 74 00 65 00 73 00 74 0a                ...t.e.s.t.
$ ./perl -Ilib -e 'open(FH, "<:encoding(utf16)", "utf16");$a=<FH>;print $a'|hex
UTF-16:Partial character at -e line 1.
UTF-16:Partial character at -e line 1.
74 65 73 74                                     test
$

Aw. You can't use 'utf16' for "use encoding" or PerlIO. You have to specify the endianness. Because of the BOM mark you can't use it for PerlIO stream.

I'll tweak Unicode.pm so that perlio_ok returns 0 for BOMless UTF's in the next version

Dan the Encode Maintainer

<Prev in Thread] Current Thread [Next in Thread>