perl-unicode

Re: Fix UTF Encoding issue

2007-12-03 17:36:15
Jakub Narebski wrote:
On Mon, 3 Dec 2007, Martin Koegler wrote:
On Mon, Dec 03, 2007 at 04:06:48AM -0800, Jakub Narebski wrote:
Ismail Dönmez <ismail(_at_)pardus(_dot_)org(_dot_)tr> writes:
Monday 03 December 2007 Tarihinde 12:14:43 yazm??t?:
Benjamin Close <Benjamin(_dot_)Close(_at_)clearchain(_dot_)com> writes:
-       eval { $res = decode_utf8($str, Encode::FB_CROAK); };
-       if (defined $res) {
-               return $res;
-       } else {
-               return decode($fallback_encoding, $str, Encode::FB_DEFAULT);
-       }
+       eval { return ($res = decode_utf8($str, Encode::FB_CROAK)); };
+       return decode($fallback_encoding, $str, Encode::FB_DEFAULT);
 }
This version is broken on Debian sarge and etch. Feeding a UTF-8 and a latin1
encoding of the same character sequence yields to different results.

For the record, this was on a debian sid machine.

#perl --version
This is perl, v5.8.8 built for x86_64-linux-gnu-thread-multi

and the result of not using the original patch was:

<h1>Software error:</h1>
<pre>Cannot decode string with wide characters at /usr/lib/perl/5.8/Encode.pm 
line 166.
</pre>


I haven't tried the other solutions tested here.
eval { $res = decode_utf8(...); }
if ($@) return decode(...);
return $res

or

eval { $res = decode_utf8(...); }
if (defined $res)
      return $res;
else
    return decode(...);

show the same (wrong) behaviour on Debian sarge. They do not always
decode non UTF-8 characters correctly, eg.
#öäü does not work
#äöüä does work

On Debian etch, both versions are working.

I don't know enough Perl to decide if it is a bug in gitweb usage
of decode_utf8, if it is a bug in your version of Encode, or if it
is bug in Encode.

Send copy of this mail to maintainers of Encode perl module.
Ismail do you know if sid was also broken?

<Prev in Thread] Current Thread [Next in Thread>