On 09/02/2012 09:21, Clint Redwood wrote:
Hi!
I've a slightly awkward comparison, where I have two numbers of differing
precisions, and I want to regard them as the same if the less preicse is the
more precise rounded down to the same precision.
Examples are:
17.166666666666668 ~= 17.1666666666667
and
8.333333333333334 ~= 8.33333333333333
Unfortunately, the precision is not always the same for all the numbers I'm
trying to compare.
I just wondered if anyone on this list had done anything similar, and had a
good way of doing it, while I try and figure out a suitable way to do it.
Yours,
Clint Redwood.
Presumably by "precision" you mean the number of decimal digits in the
lexical form of the number. That means that trailing zero digits change
the result, and since trailing zeroes are ignored for all the
system-supplied numeric data types, it means you are outside the range
of what the system data types can do.
Drafts of XSD 1.1 included a "precision decimal" data type that was
intended to meet this need - but it only defined the lexical and value
space of the type, not the arithmetic rules for comparing and
calculating with these values (which is why it was dropped from the
final spec).
I think in your position I would represent the values as elements of the
form <value precision="12">17.16666667</value>, and then write your own
function library with operations for comparing and calculating on these
values.
Michael Kay
Saxonica
--~------------------------------------------------------------------
XSL-List info and archive: http://www.mulberrytech.com/xsl/xsl-list
To unsubscribe, go to: http://lists.mulberrytech.com/xsl-list/
or e-mail: <mailto:xsl-list-unsubscribe(_at_)lists(_dot_)mulberrytech(_dot_)com>
--~--