xsl-list
[Top] [All Lists]

RE: 10,000 document()'s

2003-04-08 09:25:56
Peter

Forgive me if I've missed the point, but why not use old fashioned
procedural code? 

as in:

load the input into a dom object and then itterate through each file element
querying the specific document and  adding a wordcount element as you go.

That way you have no memory issues, and as you say speed is not a problem.

Cheers 

Rod

-----Original Message-----
From: Peter Binkley [mailto:Peter(_dot_)Binkley(_at_)ualberta(_dot_)ca]
Sent: 08 April 2003 17:06
To: 'xsl-list(_at_)lists(_dot_)mulberrytech(_dot_)com'
Subject: [xsl] 10,000 document()'s


I need advice on how to tackle this problem: I've got a file that contains a
list of about 10,000 other files, and I want to process the list so as to
add a wordcount for each of the external files. Something like this:

Input:

<files>
        <file>
                <filename>/path/to/file/2844942.xml</filename
        <file>
        <file> .... </file>
<files>

Output:

<files>
        <file>
                <filename>/path/to/file/2844942.xml</filename
                <wordcount>2938</wordcount>
        <file>
        <file> ....     </file>
<files>

The obvious approach is to use a for-each loop that includes a variable that
opens the external file using a document() call. The problem is that the
process inevitably runs out of memory, both with Saxon and Xalan. It seems
that the variables are passing out of scope and being destroyed as they
should, but I gather from a posting by Michael Kay
(http://www.biglist.com/lists/xsl-list/archives/200212/msg00507.html) that
all of those document() source trees are remaining in memory throughout the
transformation, adding up to megabytes of data.

Can anyone suggest a strategy? The process doesn't have to be fast, it just
has to finish.

Peter Binkley
Digital Initiatives Technology Librarian
Information Technology Services
4-30 Cameron Library
University of Alberta Libraries
Edmonton, Alberta
Canada T6G 2J8
Phone: (780) 492-3743
Fax: (780) 492-9243
e-mail: peter(_dot_)binkley(_at_)ualberta(_dot_)ca




 XSL-List info and archive:  http://www.mulberrytech.com/xsl/xsl-list


________________________________________________________________________
This email has been scanned for all viruses by the MessageLabs SkyScan
service. For more information on a proactive anti-virus service working
around the clock, around the globe, visit http://www.messagelabs.com
________________________________________________________________________


________________________________________________________________
Any opinions expressed in this email are those of the individual and not 
necessarily the Company. Unless expressly stated to the contrary, this email is 
not intended to give rise to a new, or affect an existing, contractual or other 
legal relationship.This email and any files transmitted with it, including 
replies and forwarded copies which may contain alterations) subsequently 
transmitted from the Company, are confidential and solely for the use of the 
intended recipient. The unauthorised use, disclosure or copying of this email, 
or any other information contained or attached,is prohibited and could, in 
certain circumstances, be a criminal offence.

If you have received this email in error please notify the sender as soon as 
possible.

This footnote also confirms that this email message has been swept for the 
presence of computer viruses.

www.focusdiy.co.uk
_________________________________________________________________

________________________________________________________________________
This email has been scanned for all viruses by the MessageLabs SkyScan
service. For more information on a proactive anti-virus service working
around the clock, around the globe, visit http://www.messagelabs.com
________________________________________________________________________

 XSL-List info and archive:  http://www.mulberrytech.com/xsl/xsl-list



<Prev in Thread] Current Thread [Next in Thread>