[xsl] Performance results of an XML-based neural network versus a map-ba

Hi Folks,

I implemented a neural network. My first implementation was XML-based. Running 
the neural network revolves around processing this XML document:

<?xml version="1.0" encoding="UTF-8"?>
<neuralNetwork>
    <!-- Set number of nodes in each input, hidden, output layer -->
    <inodes>784</inodes>
    <hnodes>100</hnodes>
    <onodes>10</onodes>  
    <!-- Matrix containing the weights for the input to hidden layer (wih) -->
    <Matrix id="wih">
        <row>
            <col>-0.015882097402764903</col>
            <col>0.04906187053472448</col>
            <col>-0.025639452565869168</col>
            ...
        </row>
        ...
    </Matrix> 
    <!-- Matrix containing the weights for the hidden to output layer (who) -->
    <Matrix id="who">
        <row>
            <col>-0.029846534548482826</col>
            <col>0.09713823372280408</col>
            <col>-0.07405568240941922</col>
            ...
        </row>
        ...
    </Matrix>
    <!-- Learning rate -->
    <lr>0.3</lr>
</neuralNetwork>

There are lots of matrix operations involved in running a neural network -- 
lots of matrix addition, transpose, dot product, etc., operations. The 
operations update the <Matrix> elements in <neuralNetwork>.

I used the SAXON profile tool to see the performance of my implementation. The 
performance of the matrix operations was very slow. Here's the performance of 
two of the matrix operations:

matrix:addition
average time (net/ms) = 273.170
total time (net/ms) = 27,043.813

matrix:dot-product
average time (net/ms) = 257.718
total time (net/ms) = 25,514.069

[Michael Kay: what does "net/ms" mean?]

In my second implementation I converted the first implementation to be 
map-based. I replaced the above XML with this map:

<xsl:map>
    <!-- Set number of nodes in each input, hidden, output layer -->
    <xsl:map-entry key="'inodes'" select="784"/>
    <xsl:map-entry key="'hnodes'" select="100"/>
    <xsl:map-entry key="'onodes'" select="10"/>
    <!-- Learning rate -->
    <xsl:map-entry key="'lr'" select="0.3"/>
    <!-- weights between the input layer and the hidden layer (wih) -->
    <xsl:map-entry key="'wih'" select="(-0.015882097402764903, 
0.04906187053472448, -0.025639452565869168, ...)"/>
    <!-- weights between the hidden layer and the output layer (who) -->
    <xsl:map-entry key="'who'" select="(-0.029846534548482826, 
0.09713823372280408, -0.07405568240941922, ...) "/>
</xsl:map>

I again used the SAXON profile tool to see the performance. The performance of 
the matrix operations for this implementation was astoundingly fast. Here's the 
performance of two of the matrix operations:

matrix:addition
average time (net/ms) = 0.003
total time (net/ms) = 0.254
 
matrix:dot-product
average time (net/ms) = 0.001
total time (net/ms) = 0.131
 
For all the matrix operations the map-based version was millions of times 
faster than the XML-based version.

Surprisingly, however, the overall time to train the XML-based neural network 
was faster than the time to train the map-based neural network:

neural-network:train

XML-based: 
average time (net/ms) = 1711.572
total time (net/ms) = 169,445.644

map-based: 
average time (net/ms) = 3633.811
total time (net/ms) = 359,747.295

I don't understand how this could possibly happen. 

In the map-based version of the train function, I have these statements to 
update the weights of the input layer to hidden layer (wih) and the weights of 
the hidden layer to output layer (who):

<xsl:variable name="neural-network-with-new-wih" 
        select="map:put($neural-network, 'wih', $updated-wih)" />
<xsl:variable name="neural-network-with-new-wih-and-new-who" 
        select="map:put($neural-network-with-new-wih, 'who', $updated-who)" />
<xsl:sequence select="$neural-network-with-new-wih-and-new-who" />

Notice that those statements create two new maps. Every time I call the train 
function, two new maps are created. The train function is called many times. 
Could that be the cause of why my map-based implementation is running slower 
than the XML-based implementation? Is there a way to _replace_ a value in a 
map, rather than _creating_ a whole new map?

/Roger
--~----------------------------------------------------------------
XSL-List info and archive: http://www.mulberrytech.com/xsl/xsl-list
EasyUnsubscribe: http://lists.mulberrytech.com/unsub/xsl-list/1167547
or by email: xsl-list-unsub(_at_)lists(_dot_)mulberrytech(_dot_)com
--~--