xsl-list
[Top] [All Lists]

Re: RE: numerical inconsistencies doing dec to hex conv.

2003-03-10 08:05:16
Hi,

It is a linux version of Xalan that uses C++, not sure what 
version it is though (im at uni at the mo it is installed t 
home ). 

Thanks everyone for the sanity check!



Peter.




On Mon, 10 Mar 2003 12:59:30 -0000 Andrew Welch 
<AWelch(_at_)piper-group(_dot_)com> wrote:


Works fine for me with msxml3, 4, saxon 6.5.2, 7.2, and xalan 2.4.1

I know there are issues with rounding .5's, but I think floor() is constant.

Which processor are you using??

cheers
andrew

-----Original Message-----
From: Peter Doggett [mailto:ee99ppd(_at_)brunel(_dot_)ac(_dot_)uk]
Sent: 10 March 2003 12:05
To: XSL-List(_at_)lists(_dot_)mulberrytech(_dot_)com
Subject: [xsl] numerical inconsistencies doing dec to hex conv.


Hi,

I am using the below template to convert a decimal number 
into a hex number. It seems to work correctly apart from 
when I want to calculate the hex equivelent of the decimal 
number -  4294967295. This number is of particular 
significance as in Hex this should be FFFFFFFF (a 32 bit, 
bitmask; i am using xml to describe registers in h/w).
The template outputs FFFFFFF0 abnd I dont know why.

The template seems to output the incorrect value for 
1099511627775, which is the decimal representation of 
FFFFFFFFFF in hex.

The recursive template is as follows:


<xsl:variable name="hexDigits" select="'0123456789ABCDEF'" />
  <xsl:template name="toHex">
    <xsl:param name="decimalNumber" />
    <xsl:if test="$decimalNumber >= 16">
      <xsl:call-template name="toHex">
        <xsl:with-param name="decimalNumber" 
select="floor($decimalNumber div 16)" />
      </xsl:call-template>
    </xsl:if>
    <xsl:value-of select="substring($hexDigits, 
($decimalNumber mod 16) + 1, 1)" />
</xsl:template>

Can anybody figure out why this is happening or have an 
equivelent template that can calculate the conversion 
correctly?


Thankyou for your time.


----------------------
Peter Doggett
ee99ppd(_at_)brunel(_dot_)ac(_dot_)uk


 XSL-List info and archive:  http://www.mulberrytech.com/xsl/xsl-list





---
Incoming mail is certified Virus Free.
Checked by AVG anti-virus system (http://www.grisoft.com).
Version: 6.0.458 / Virus Database: 257 - Release Date: 24/02/2003
 


---
Outgoing mail is certified Virus Free.
Checked by AVG anti-virus system (http://www.grisoft.com).
Version: 6.0.458 / Virus Database: 257 - Release Date: 24/02/2003
 

 XSL-List info and archive:  http://www.mulberrytech.com/xsl/xsl-list


----------------------
Peter Doggett
ee99ppd(_at_)brunel(_dot_)ac(_dot_)uk


 XSL-List info and archive:  http://www.mulberrytech.com/xsl/xsl-list



<Prev in Thread] Current Thread [Next in Thread>