perl-unicode

Re: Fix UTF Encoding issue

2007-12-04 05:18:11
On Tue, Dec 04, 2007 at 08:16:24AM +1030, Benjamin Close wrote:
Jakub Narebski wrote:
On Mon, 3 Dec 2007, Martin Koegler wrote:
On Mon, Dec 03, 2007 at 04:06:48AM -0800, Jakub Narebski wrote:
Ismail Dönmez <ismail(_at_)pardus(_dot_)org(_dot_)tr> writes:
Monday 03 December 2007 Tarihinde 12:14:43 yazm??t?:
Benjamin Close <Benjamin(_dot_)Close(_at_)clearchain(_dot_)com> writes:
-       eval { $res = decode_utf8($str, Encode::FB_CROAK); };
-       if (defined $res) {
-               return $res;
-       } else {
-               return decode($fallback_encoding, $str, 
Encode::FB_DEFAULT);
-       }
+       eval { return ($res = decode_utf8($str, Encode::FB_CROAK)); 
};
+       return decode($fallback_encoding, $str, Encode::FB_DEFAULT);
}
           
This version is broken on Debian sarge and etch. Feeding a UTF-8 and a 
latin1
encoding of the same character sequence yields to different results.

For the record, this was on a debian sid machine.

#perl --version
This is perl, v5.8.8 built for x86_64-linux-gnu-thread-multi

and the result of not using the original patch was:

<h1>Software error:</h1>
<pre>Cannot decode string with wide characters at 
/usr/lib/perl/5.8/Encode.pm line 166.
</pre>


I haven't tried the other solutions tested here.

Debian etch also has v5.8.8.

My main question is, why is the error not catched?

I'm not a perl programmer, but in your patch the first line is a
NOP. The return in eval seems to only returns from the eval block, so
any text is decoded as latin1 with the second statement.

In the original version, decode($fallback_encoding, $str,
Encode::FB_DEFAULT) can not emit an error, else it would in your
version too. 

In your version, eval is able to surpress the error of
decode_utf8($str, Encode::FB_CROAK);, but not in the original version.

Strange.

mfg Martin Kögler

<Prev in Thread] Current Thread [Next in Thread>