xsl-list
[Top] [All Lists]

Floating point precision error

2006-01-25 18:16:26
This is something I don't understand. Given this XSL snippet:

<xsl:value-of select="$nodes[1]/unitamount" />_
<xsl:value-of select="$nodes[1]/quantity" />_
<xsl:value-of select="($nodes[1]/quantity * $nodes[1]/unitamount)" />_

I'm getting the printout as:
6.4_
3000_
19199.908447265625_ 

The literal values in XML is 6.4 and 3000. If I round the result to 2 
decimal places, I still get 19199.91, a far cry from what it should be 
(19200).

I'm using dom4j 1.6, so I guess that's XSLT 1.0? Also this only happens 
on SunOS and not on Windows. Still, the error is huge and obvious. 
Isn't there a way to force the calculation to be more exact?

Thanks,

Thang Le

-- 
NeoMail - http://neomail.sourceforge.net

--~------------------------------------------------------------------
XSL-List info and archive:  http://www.mulberrytech.com/xsl/xsl-list
To unsubscribe, go to: http://lists.mulberrytech.com/xsl-list/
or e-mail: <mailto:xsl-list-unsubscribe(_at_)lists(_dot_)mulberrytech(_dot_)com>
--~--



<Prev in Thread] Current Thread [Next in Thread>