xsl-list
[Top] [All Lists]

Re: [xsl] Approach to transform 250GB xml data

2014-09-10 05:38:07
Out of curiosity: how do you intend to access/process the 250 GB once they are transformed?

If it is a huge DB dump, maybe you can dump it in slices or, if it is an XML database with XSLT 2 capabilities, transform it in place.

Gerrit

On 10.09.2014 11:48, Vishnu vishnu(_at_)innodata(_dot_)com wrote:
The transformation is just for renaming the element or attributes and to change 
the tree structure only(not for sorting).

Thanks!

Vishnu Singh

________________________________________
From: Michael Kay mike(_at_)saxonica(_dot_)com 
<xsl-list-service(_at_)lists(_dot_)mulberrytech(_dot_)com>
Sent: Wednesday, September 10, 2014 1:42 PM
To: xsl-list(_at_)lists(_dot_)mulberrytech(_dot_)com
Subject: Re: [xsl] Approach to transform 250GB xml data

It is not practical to transform this using XSLT except by use of a streaming 
XSLT processor such as Saxon-EE, and even then it depends on the detailed 
nature of the transformation to be performed. Some transformations are readily 
streamed (e.g. renaming all the elements), others are impossible (e.g. 
sorting). Tell us more about what the transformation is doing.

Michael Kay
Saxonica
mike(_at_)saxonica(_dot_)com
+44 (0) 118 946 5893




On 10 Sep 2014, at 08:36, Vishnu vishnu(_at_)innodata(_dot_)com 
<xsl-list-service(_at_)lists(_dot_)mulberrytech(_dot_)com> wrote:

Hi,

I have approx 250GB xml data and I want to transform it using XSLT 2.0. What 
should be the best approach to transform this database.

I tried it with ANT but it gave me JAVA heap space error message.

Please suggest.

Thanks!

Vishnu Singh
"This e-mail and any attachments transmitted with it are for the sole use of the 
intended recipient(s) and may contain confidential , proprietary or privileged 
information. If you are not the intended recipient, please contact the sender by reply 
e-mail and destroy all copies of the original message. Any unauthorized review, use, 
disclosure, dissemination, forwarding, printing or copying of this e-mail or any action 
taken in reliance on this e-mail is strictly prohibited and may be unlawful."


"This e-mail and any attachments transmitted with it are for the sole use of the 
intended recipient(s) and may contain confidential , proprietary or privileged 
information. If you are not the intended recipient, please contact the sender by reply 
e-mail and destroy all copies of the original message. Any unauthorized review, use, 
disclosure, dissemination, forwarding, printing or copying of this e-mail or any action 
taken in reliance on this e-mail is strictly prohibited and may be unlawful."



--
Gerrit Imsieke
Geschäftsführer / Managing Director
le-tex publishing services GmbH
Weissenfelser Str. 84, 04229 Leipzig, Germany
Phone +49 341 355356 110, Fax +49 341 355356 510
gerrit(_dot_)imsieke(_at_)le-tex(_dot_)de, http://www.le-tex.de

Registergericht / Commercial Register: Amtsgericht Leipzig
Registernummer / Registration Number: HRB 24930

Geschäftsführer: Gerrit Imsieke, Svea Jelonek,
Thomas Schmidt, Dr. Reinhard Vöckler
--~----------------------------------------------------------------
XSL-List info and archive: http://www.mulberrytech.com/xsl/xsl-list
EasyUnsubscribe: http://lists.mulberrytech.com/unsub/xsl-list/1167547
or by email: xsl-list-unsub(_at_)lists(_dot_)mulberrytech(_dot_)com
--~--

<Prev in Thread] Current Thread [Next in Thread>