On 09/02/2012 09:24, Clint Redwood wrote:
Hi!
I've a slightly awkward comparison, where I have two numbers of
differing precisions, and I want to regard them as the same if the
less preicse is the more precise rounded down to the same precision.
Examples are: 17.166666666666668 ~= 17.1666666666667 and
8.333333333333334 ~= 8.33333333333333
Unfortunately, the precision is not always the same for all the
numbers I'm trying to compare.
I just wondered if anyone on this list had done anything similar, and
had a good way of doing it, while I try and figure out a suitable way
to do it.
Yours,
Clint Redwood.
How big are your precisions? in particular do you want xpath to treat
then as doubles (which will mean that anything after 16 or so places
will be lost anyway) and you can just use double precision numeric
functions such as round to deal with the rounding, or do you need
arbitrary precision decimals or (equivalently srings0 and expect to do
teh rounding "by hand".
David
________________________________________________________________________
The Numerical Algorithms Group Ltd is a company registered in England
and Wales with company number 1249803. The registered office is:
Wilkinson House, Jordan Hill Road, Oxford OX2 8DR, United Kingdom.
This e-mail has been scanned for all viruses by Star. The service is
powered by MessageLabs.
________________________________________________________________________
--~------------------------------------------------------------------
XSL-List info and archive: http://www.mulberrytech.com/xsl/xsl-list
To unsubscribe, go to: http://lists.mulberrytech.com/xsl-list/
or e-mail: <mailto:xsl-list-unsubscribe(_at_)lists(_dot_)mulberrytech(_dot_)com>
--~--