utf8, japanese, web-pages: beginning to see the light...
2004-05-13 14:30:15
A few days ago, I queried this list about my problems with a script
that finds the charset of Japanese web pages and translates their text
into utf-8.
The following solution, proposed by Nick Ing-Simmons, worked for my
case:
binmode STDOOUT,":utf8";
my $encoding = find_encoding($charset);
my $unicode = $encoding->decode($text);
print $unicode;
($charset is the charset as extracted from the html code of the page
and $text is all the text from the page itself, as returned by the LWP
agent.)
Thanks a lot to Nick and to all the others who responded to my plea for
help.
Now for a much less pressing issue: Does anybody know of something
similar to the HTML::FormatText module that can take utf-8 input, and
generate utf-8 output? In other words, of a module or command line tool
to which I could feed my Japanese html pages, or html documents in
other non-Latin alphabets, and get nicely formatted plain utf-8 text as
output?
(HTML::FormatText seems to break with utf-8 and with the Japanese
encodings.)
Thanks in advance.
Regards,
Marco
---
Marco Baroni
University of Bologna
http://sslmit.unibo.it/~baroni
<Prev in Thread] |
Current Thread |
[Next in Thread>
|
- utf8, japanese, web-pages, the horror, the horror..., Marco Baroni
- RE: utf8, japanese, web-pages, the horror, the horror..., Edward Batutis
- Re: utf8, japanese, web-pages, the horror, the horror..., Marco Baroni
- Re: utf8, japanese, web-pages, the horror, the horror..., Nick Ing-Simmons
- Re: utf8, japanese, web-pages, the horror, the horror..., Marco Baroni
- utf8, japanese, web-pages: beginning to see the light...,
Marco Baroni <=
- Re: utf8, japanese, web-pages: beginning to see the light..., Nick Ing-Simmons
|
Previous by Date: |
Re: BOM and principle of least surprise, Jarkko Hietaniemi |
Next by Date: |
Re: Printing Unicode from XS, Erland Sommarskog |
Previous by Thread: |
Re: utf8, japanese, web-pages, the horror, the horror..., Marco Baroni |
Next by Thread: |
Re: utf8, japanese, web-pages: beginning to see the light..., Nick Ing-Simmons |
Indexes: |
[Date]
[Thread]
[Top]
[All Lists] |
|
|