Re: XSL SUM BUG?

xsl-list

Re: XSL SUM BUG?

2004-02-19 13:52:18



Its not a bug, it's a feature of binary computing and numerical
calculations.
You can not store 0.1 exactly as a base 2 floating point number, so you
can not get any exact answers from any calculation involving such a
number.

Use format-number() to round to a suitable number of decimal places and
the error terms will disapear.

David

-- 
http://www.dcarlisle.demon.co.uk/matthew

 XSL-List info and archive:  http://www.mulberrytech.com/xsl/xsl-list

[More with this subject...]

<Prev in Thread]	Current Thread	[Next in Thread>
extracting sequences, Saverio Perugini Re: extracting sequences, Dimitre Novatchev Re: extracting sequences, Saverio Perugini paths through DAGs, Saverio Perugini XSL SUM BUG?, Andrew XSL number BUG?, Andrew Re: XSL SUM BUG?, G. Ken Holman Re: XSL SUM BUG?, Andrew RE: XSL SUM BUG?, Andreas L. Delmelle Re: XSL SUM BUG?, J.Pietschmann Re: XSL SUM BUG?, David Carlisle <= The Solution (Was: Re: extracting sequences), Dimitre Novatchev Re: The Solution (Was: Re: extracting sequences), Dimitre Novatchev Re: The Solution (Was: Re: extracting sequences), Saverio Perugini Re: The Solution (Was: Re: extracting sequences), Dimitre Novatchev

Previous by Date:	Re: FW: Troubleshooting a sort, David Carlisle
Next by Date:	Re: Re: Counting nodes efficiently, Wendell Piez
Previous by Thread:	Re: XSL SUM BUG?, J.Pietschmann
Next by Thread:	The Solution (Was: Re: extracting sequences), Dimitre Novatchev
Indexes:	[Date] [Thread] [Top] [All Lists]