xsl-list
[Top] [All Lists]

RE: 10,000 document()'s

2003-04-09 12:38:42
I would suggest writing a SAX filter that invokes the XSLT
transformations (one transformation for each file) via JAXP, gets the
result back in a StringWriter, and adds an element containing the word
count to the output stream.

Michael Kay
Software AG
home: Michael(_dot_)H(_dot_)Kay(_at_)ntlworld(_dot_)com
work: Michael(_dot_)Kay(_at_)softwareag(_dot_)com 

-----Original Message-----
From: owner-xsl-list(_at_)lists(_dot_)mulberrytech(_dot_)com 
[mailto:owner-xsl-list(_at_)lists(_dot_)mulberrytech(_dot_)com] On Behalf Of 
Peter Binkley
Sent: 08 April 2003 17:06
To: 'xsl-list(_at_)lists(_dot_)mulberrytech(_dot_)com'
Subject: [xsl] 10,000 document()'s


I need advice on how to tackle this problem: I've got a file 
that contains a list of about 10,000 other files, and I want 
to process the list so as to add a wordcount for each of the 
external files. Something like this:

Input:

<files>
      <file>
              <filename>/path/to/file/2844942.xml</filename
      <file>
      <file> .... </file>
<files>

Output:

<files>
      <file>
              <filename>/path/to/file/2844942.xml</filename
              <wordcount>2938</wordcount>
      <file>
      <file> ....     </file>
<files>

The obvious approach is to use a for-each loop that includes 
a variable that opens the external file using a document() 
call. The problem is that the process inevitably runs out of 
memory, both with Saxon and Xalan. It seems that the 
variables are passing out of scope and being destroyed as 
they should, but I gather from a posting by Michael Kay
(http://www.biglist.com/lists/xsl-list/archives/200212/msg0050
7.html) that all of those document() source trees are remaining in
memory throughout the transformation, adding up to megabytes of data.

Can anyone suggest a strategy? The process doesn't have to be fast, it
just has to finish.

Peter Binkley
Digital Initiatives Technology Librarian
Information Technology Services
4-30 Cameron Library
University of Alberta Libraries
Edmonton, Alberta
Canada T6G 2J8
Phone: (780) 492-3743
Fax: (780) 492-9243
e-mail: peter(_dot_)binkley(_at_)ualberta(_dot_)ca




 XSL-List info and archive:  http://www.mulberrytech.com/xsl/xsl-list


 XSL-List info and archive:  http://www.mulberrytech.com/xsl/xsl-list



<Prev in Thread] Current Thread [Next in Thread>