xsl-list
[Top] [All Lists]

Re: [xsl] floor() and idiv

2008-11-06 14:44:08
A bug in the spec or in Saxon ?

   Xmlizer>   floor(1000000000000000000000000000000001 div
   Xmlizer> 100000000000000000000000000000001),
   Xmlizer> 1000000000000000000000000000000001 idiv
   Xmlizer> 100000000000000000000000000000001,
   Xmlizer> (1000000000000000000000000000000001 div
   Xmlizer> 100000000000000000000000000000001) cast as xs:integer

   Xmlizer> gives

   Xmlizer> 10, 9, 10

Saxon, I think.

Gestalt gives 10, 10, 10.

So in Gestalt how do you determine when to use the equivalent of
BigDecimal in Java, or do you not need to work about that sort of
thing?

(In Java the result using standard division is 9.999999999999998 which
I guess just gets truncated to 9, but using BigDecimal.divide() you
get the correct 10)

My particular issue is concerning pixel widths of mobile phones, so it
pretty unlikely to encounter these numbers :)



-- 
Andrew Welch
http://andrewjwelch.com
Kernow: http://kernowforsaxon.sf.net/

--~------------------------------------------------------------------
XSL-List info and archive:  http://www.mulberrytech.com/xsl/xsl-list
To unsubscribe, go to: http://lists.mulberrytech.com/xsl-list/
or e-mail: <mailto:xsl-list-unsubscribe(_at_)lists(_dot_)mulberrytech(_dot_)com>
--~--

<Prev in Thread] Current Thread [Next in Thread>