I believe its precision
float: stores a number between +/- 1.5 X 10-45 to +/- 3.4 X 1038
The "decimal" data type should be used when you require a high level of
precision, since it can accurately store data up to 28 digits after the
decimal points (128-bit precision).
----- Original Message -----
From: "Rowland Shaw" <Rowland(_dot_)Shaw(_at_)businessobjects(_dot_)com>
To: <xsl-list(_at_)lists(_dot_)mulberrytech(_dot_)com>
Sent: Monday, March 29, 2004 3:15 PM
Subject: RE: [xsl] Re: XSLT 2.0 Decimal number silliness
Maybe I'm being really dumb here, but what is the difference between a
decimal number and a floating point number?
-----Original Message-----
From: Jeff Kenton [mailto:jkenton(_at_)datapower(_dot_)com]
Sent: 29 March 2004 14:53
To: xsl-list(_at_)lists(_dot_)mulberrytech(_dot_)com
Subject: Re: [xsl] Re: XSLT 2.0 Decimal number silliness
Pete,
You missed the point. We're talking about decimal numbers here, not
floats.
--jeff
Pete Forman wrote:
Jeff Kenton <jkenton(_at_)datapower(_dot_)com> writes:
> I've been looking at XSLT 2.0 decimal numbers (again), and I still
> think the WC needs to do better.
> [...]
> <xsl:value-of select="2.0 div 3.0" />~
> <xsl:value-of select="0.444 div 0.666" />~
> [...]
> Both lines of output should be equal.
Why? The majority of implementations will use a binary format such as
IEC 60559 to hold floating point numbers. 2.0 and 3.0 can be
converted exactly but the representations of 0.444 and 0.666 will be
inexact (0.44400000000000001 and 0.66600000000000004 in my test). You
might get a better match with 0.5 div 0.75. If you want fixed point
number semantics try moving the decimal points to the right before
dividing.
Floating point arithmetic is inexact by definition. Your results
_are_ the same to within the precision specified by the language
and/or implementation.
--+------------------------------------------------------------------
XSL-List info and archive: http://www.mulberrytech.com/xsl/xsl-list
To unsubscribe, go to: http://lists.mulberrytech.com/xsl-list/
or e-mail:
<mailto:xsl-list-unsubscribe(_at_)lists(_dot_)mulberrytech(_dot_)com>
--+--
--+------------------------------------------------------------------
XSL-List info and archive: http://www.mulberrytech.com/xsl/xsl-list
To unsubscribe, go to: http://lists.mulberrytech.com/xsl-list/
or e-mail:
<mailto:xsl-list-unsubscribe(_at_)lists(_dot_)mulberrytech(_dot_)com>
--+--