perl-i18n

Re: Using :encoding and :crlf together?

2006-02-08 08:11:23
Hi,

I haven't managed to test this on ActiveState yet as it's not in PPM. I've submitted a request for it to be on PPM. :D

I haven't had a response back yet, but in the meantime, I've seen that they weren't able to compile PerlIO::eol on Windows (see http://ppm.activestate.com/BuildStatus/5.8-windows/windows-5.8/PerlIO-eol-0.13.txt ), which is why it's not on PPM. I wasn't able to compile it either, which is frustrating as the project I'm working on needs to be platform-independent. In the end I ended up writing my own subs and using them:

sub crlfise {
  my ($text) = @_;
  $text=~s/\n/\x{0D}\x{0A}/g;
  return $text;
}

sub normalise {
  my ($text) = @_;
  $text=~s/(?:\x{0D}\x{0A}|\x{0A}\x{0D}|\x{0D}|\x{0A})/\n/g;
  return $text;
}

crlfise() will take your (already normalised) text and convert it to use CR/LF line endings, while normalise() will take un-normalised text using any of the different line endings (CR/LF, LF/CR, CR or LF) and normalise it. The test program then becomes:

---
#!/usr/bin/perl

open(FILE, ">:raw:encoding(UTF-16)", "test");
print FILE crlfise("Test \x{A3}45!\n");
print FILE crlfise("Test!\n");
close(FILE);
---

It's annoying, yes, but it's the only way I found that reliably does what I want in a platform-independent manner. Note the use of the ":raw" layer - this is necessary to get Windows to open binary files in binary mode, so Perl doesn't try to second-guess you on your newlines. The above program should give the same output on both Linux and Windows, and for that matter, it should give the same output on any platform.

Of course, you could also rewrite the above as:

---
#!/usr/bin/perl

my $output;
$output  = "Test \x{A3}45!\n";
$output .= "Test!\n";

open(FILE, ">:raw:encoding(UTF-16)", "test");
print FILE crlfise($output);
close(FILE);
---

if you only wanted to write to the file once, which is probably a more efficient way of doing things anyway.

This probably really will be my last post on the matter, given that this seems to work in all cases.

Hope this helps anybody out there struggling with this. :)

 - Ciaran.

<Prev in Thread] Current Thread [Next in Thread>