xsl-list
[Top] [All Lists]

RE: Out OF Memory Exception for Large XML data

2006-01-30 06:14:00
Yes, this looks like one that's amenable to serial processing. Something
like:

 <?xml version="1.0" encoding="UTF-8"?>
 <xsl:stylesheet version="2.0"
           xmlns:xsl="http://www.w3.org/1999/XSL/Transform";
           xmlns:f="http://localhost/functions";>
     <xsl:output encoding="UTF-8" indent="yes" method="xml" version="1.0"/>
     <xsl:param name="_userName"/>
     <xsl:param name="_password"/>
     <xsl:param name="_class"/>
     <xsl:param name="_identifier"/>
     <xsl:param name="_subject"/>
     <xsl:variable name="apos">'</xsl:variable>

     <xsl:template name="main">
       <DB>
         <xsl:apply-templates select="f:suppliers()"/>
       </db>
     </xsl:template>

     <xsl:function name="f:suppliers"             
         <xsl:copy-of
select="doc('suppliers.xml')/DB/dbo.VW_Ref_Supplier_2_TPL_Supplier"
                   saxon:read-once="yes"
xmlns:saxon="http://saxon.sf.net/"/>
     </xsl:function>

     <xsl:template match="dbo.VW_Ref_Supplier_2_TPL_Supplier[Ref_Status !=
'D']">
                    <dbo.TPL_Supplier>
                        <xsl:variable name="supplierRID" 
select="12345"/>
                        <xsl:attribute name="Action"/>
                        <xsl:attribute name="RollBackOnError">
                            <xsl:value-of select="'true'"/>
                        </xsl:attribute>
                        <RID>
                            <xsl:value-of select="$supplierRID"/>
                        </RID>
                        <xsl:if test="ST_SupplierPWID != ''">
                            <PWID>
                                <xsl:value-of 
select="ST_SupplierPWID"/>
                            </PWID>
                        </xsl:if>
                        <PDOPWID>
                            <xsl:value-of select="PDOPWID"/>
                        </PDOPWID>
                        <xsl:if test="Ref_SupplierCode != 
ST_CustomerKey">
                            <CustomerKey>
                                <xsl:value-of 
select="Ref_SupplierCode"/>
                            </CustomerKey>
                        </xsl:if>
                       </dbo.TPL_Supplier>
    </xsl:template>

    <xsl:template match="*"/>
</xsl:stylesheet>       
 

-----Original Message-----
From: Pankaj Bishnoi [mailto:pankaj(_dot_)bishnoi(_at_)adeptia(_dot_)com] 
Sent: 30 January 2006 12:00
To: xsl-list(_at_)lists(_dot_)mulberrytech(_dot_)com
Subject: Re: [xsl] Out OF Memory Exception for Large XML data

Hi Michael
                 Thanks for your help. I set the heap size to 
1024 and it
was not helpful. I had also seen the serial link. Now my 
quesstion is how
can i convert a xsl to serialized one. My XSL is::

<?xml version="1.0" encoding="UTF-8"?>
<xsl:stylesheet extension-element-prefixes="redirect" version="1.1"
xmlns:java="http://xml.apache.org/xslt/java";
xmlns:xsl="http://www.w3.org/1999/XSL/Transform";>
    <xsl:output encoding="UTF-8" indent="yes" method="xml" 
version="1.0"/>
    <xsl:param name="_userName"/>
    <xsl:param name="_password"/>
    <xsl:param name="_class"/>
    <xsl:param name="_identifier"/>
    <xsl:param name="_subject"/>
    <xsl:variable name="apos">'</xsl:variable>
    <xsl:template match="/">
        <DB>
            <xsl:for-each 
select="DB/dbo.VW_Ref_Supplier_2_TPL_Supplier">
                <xsl:if test="Ref_Status != 'D'">
                    <dbo.TPL_Supplier>
                        <xsl:variable name="supplierRID" 
select="12345"/>
                        <xsl:attribute name="Action"/>
                        <xsl:attribute name="RollBackOnError">
                            <xsl:value-of select="'true'"/>
                        </xsl:attribute>
                        <RID>
                            <xsl:value-of select="$supplierRID"/>
                        </RID>
                        <xsl:if test="ST_SupplierPWID != ''">
                            <PWID>
                                <xsl:value-of 
select="ST_SupplierPWID"/>
                            </PWID>
                        </xsl:if>
                        <PDOPWID>
                            <xsl:value-of select="PDOPWID"/>
                        </PDOPWID>
                        <xsl:if test="Ref_SupplierCode != 
ST_CustomerKey">
                            <CustomerKey>
                                <xsl:value-of 
select="Ref_SupplierCode"/>
                            </CustomerKey>
                        </xsl:if>
                       </dbo.TPL_Supplier>
                </xsl:if>
            </xsl:for-each>
        </DB>
    </xsl:template>
</xsl:stylesheet>


Now my quesstion is can i convert this XSL to serialized one?.

Thanks
Pankaj




----- Original Message ----- 
From: "Michael Kay" <mike(_at_)saxonica(_dot_)com>
To: <xsl-list(_at_)lists(_dot_)mulberrytech(_dot_)com>
Sent: Monday, January 30, 2006 4:43 PM
Subject: RE: [xsl] Out OF Memory Exception for Large XML data


         I am having a transformation in which the source 
file size is
large. The source xml size is around 200 MB. So my
transformer throws Out of
Memory exception after some time. I am using Xalan. This
issue also comes
when i use Saxon transformer. Is there a way by which i 
can set some
transformer parameter to overcome this problem.

You can set the amount of memory allocated to java using 
something like

java -Xms1024M -Xmx1024M net.sf.saxon.Transform ....

(or similarly for Xalan, of course).

I wouldn't attempt this with less than 1Gb of main memory.

If the transformation is essentially serial, take a look at

http://www.saxonica.com/documentation/sourcedocs/serial.html

to see if it provides a way forward.

Michael Kay
http://www.saxonica.com/





--~------------------------------------------------------------------
XSL-List info and archive:  http://www.mulberrytech.com/xsl/xsl-list
To unsubscribe, go to: http://lists.mulberrytech.com/xsl-list/
or e-mail: 
<mailto:xsl-list-unsubscribe(_at_)lists(_dot_)mulberrytech(_dot_)com>
--~--



--~------------------------------------------------------------------
XSL-List info and archive:  http://www.mulberrytech.com/xsl/xsl-list
To unsubscribe, go to: http://lists.mulberrytech.com/xsl-list/
or e-mail: 
<mailto:xsl-list-unsubscribe(_at_)lists(_dot_)mulberrytech(_dot_)com>
--~--





--~------------------------------------------------------------------
XSL-List info and archive:  http://www.mulberrytech.com/xsl/xsl-list
To unsubscribe, go to: http://lists.mulberrytech.com/xsl-list/
or e-mail: <mailto:xsl-list-unsubscribe(_at_)lists(_dot_)mulberrytech(_dot_)com>
--~--



<Prev in Thread] Current Thread [Next in Thread>