xsl-list
[Top] [All Lists]

Re: [xsl] Using XSLT to build an index

2011-10-30 18:07:47
On 30/10/2011 22:07, G. Ken Holman wrote:
At 2011-10-30 14:47 -0700, Mark wrote:
The list archives did not seem to contain an XSLT stylesheet that could index an XML file, but I may have missed it. Is it practical to write my own XSLT 2 indexing stylesheet? If so, I have a bilingual XML file that I want to index.

Where you simply want all words, except your stop words, collected to automate the index generation, I've never been successful with automated indexing myself. For my books I've authored the components of the index, and then pointed to those components from within the code.
Certainly, an automatically-generated index will never come close to one produced by hand. However, the index to the Saxon documentation at http://www.saxonica.com/documentation/index-entries/intro.xml is automatically-generated, and we thought it better than having nothing at all.

This index is constructed by taking all the title, subtitle, and index elements (the index tag allows phrases to be marked for manual indexing), and extracts tokens using

<xsl:for-each select="tokenize(replace(.,'\.(\s|$)','$1'),'[\W\p{S}-[\-\.'']]+')[not(.=$stop-words)]"> <token section="{$section}" page="{$page}" subpage="{$subpage}" title="{$title}" value="{.}"/>
</xsl:for-each>

It then sorts and groups the tokens to produce the index.

I wouldn't claim this regular expression is perfect, but it worked for the intended purpose. It could easily be improved if we had time to work on it.

The replace() preprocesses the input by removing any full stop that is followed by a space or by the end of the string.

The tokenize() then splits the string on any separator that is a non-word character or a Unicode separator character, other than hyphen or full-stop. This means that for the Saxon index, hyphens and embedded periods are considered word characters, resulting in index entries for terms like "xsl:strip-space" and "1.1".

--~------------------------------------------------------------------
XSL-List info and archive:  http://www.mulberrytech.com/xsl/xsl-list
To unsubscribe, go to: http://lists.mulberrytech.com/xsl-list/
or e-mail: <mailto:xsl-list-unsubscribe(_at_)lists(_dot_)mulberrytech(_dot_)com>
--~--