perl-unicode

Re: [cpan #8089] Encode::utf8::decode_xs does not check partial chars

2004-10-25 01:30:22
Dan Kogai <dankogai(_at_)dan(_dot_)co(_dot_)jp> writes:
On Oct 25, 2004, at 03:01, Nick Ing-Simmons wrote:
But as Dan said at the start \xF6 on its own (say as 1023 octet
in a 0..1023 1024-octet buffer is not a fail.
Changing that will make :encoding() layer have problems as buffer
boundaries can occur in the middle of characters.

Right.  Encode-2.07 indeed had the problem, resulting bleedperl to fail 
on ext/PerlIO/t/encoding.t, test 14.  Encode-2.08 corrected the problem 
by checking if the caller is PerlIO and if so, sets 
Encode::RETURN_ON_ERR so it breaks out of the loop on partial character 
case.

I believe I have checked & tested enough but I would appreaciate if you 
guys take a look, especially Encode.xs and t/fallback.t.

What about other uses of Encode that may get partial characters?
ANY system which breaks strings into blocks can have this issue.



Dan the Encode Maintainer