perl-unicode

Re: Converting UTF-EBCDIC to UTF-8

2003-03-18 11:30:04
Thank you.

I want to know what UTF8-flag-on string (SvUTF8(sv) is true)
is equivalent with "\xef\xb9\xbf",
but I would misunderstand usage of pack('U*', LIST).

Are numbers < 256 in LIST
Unicode code points or native ASCII/EBCDIC code points?

Here is a draft of tests 25..28.
I guess this should succeses if pack(U) would take Unicode code points,
but I might be still in the wrong.

### TESTS START
$utf8_fe7f_upgraded = ord("A") != 0x41
     ? pack('U*', 213, 190, 215)  # EBCDIC "\xef\xb9\xbf"
     : pack('U*', 239, 185, 191); # ASCII  "\xef\xb9\xbf"

$utf8_fe7f_bytes = pack('C*', 239, 185, 191);

print "\x{fe7f}" eq utf8_to_unicode($utf8_fe7f_upgraded)
   ? "ok" : "not ok", " 25\n";

print "\x{fe7f}" eq utf8_to_unicode($utf8_fe7f_bytes)
   ? "ok" : "not ok", " 26\n";

print  $utf8_fe7f_upgraded eq unicode_to_utf8("\x{fe7f}")
   ? "ok" : "not ok", " 27\n";

print  $utf8_fe7f_bytes    eq unicode_to_utf8("\x{fe7f}")
   ? "ok" : "not ok", " 28\n";
### TESTS END

And many thanks for discovering STRLEN-UV type mismatchings, too.

SADAHIRO Tomoyuki



 >Thank you for your report.
 >
 >I was careless about the trap on a non-ASCII platform
 >like that ("a" eq "\x61") is not true.
 >
 >So the failed tests are fixed, and some tests are added.
 >Ver. 0.20 is available from there:
 >
 >[TAR-GZ, HTML-ized POD]
 >http://homepage1.nifty.com/nomenclator/perl/Unicode-Transform-0.20.tar.gz
 >http://homepage1.nifty.com/nomenclator/perl/Unicode-Transform.html
 >
 >SADAHIRO Tomoyuki

I have now run this latest version on z/OS with the following results:

/defects/brian/unicode/Unicode-Transform-0.20:>make test
PERL_DL_NONLAZY=1 /defects/brian/nonthreaded/perl-5.8.0/perl 
"-I/defects/brian/n
onthreaded/perl-5.8.0/lib" "-I/defects/brian/nonthreaded/perl-5.8.0/lib" 
"-MExtU
tils::Command::MM" "-e" "test_harness(0, 'blib/lib', 'blib/arch')" t/*.t
t/handler....ok
t/test.......FAILED tests 25, 27
        Failed 2/28 tests, 92.86% okay
Failed Test Stat Wstat Total Fail  Failed  List of Failed
-------------------------------------------------------------------------------
t/test.t                            28    2    7.14%  25 27
Failed 1/2 test scripts, 50.00% okay. 2/38 subtests failed, 94.74% okay.
make: *** [test_dynamic] Error 121

I also had the warnings similar to the following when compiling on z/OS:

WARNING CCN3196 ./Transform.c:21    Initialization between types 
"unsigned int(*)
(unsigned char*,unsigned int,unsigned int*)" and "unsigned 
long(*)(unsigned char*,unsigned int,unsigned int*)"
is not allowed.
FSUM3065 The COMPILE step ended with return code 4.

I got rid of these warnings by changing the return type all of the 
ord_in_* functions to STRLEN instead of UV, in the file unitrans.h. I 
choose this because the return type of the app_in_* functions were 
already STRLEN and they didn't generate warnings.

Brian