Dave Pawson wrote:
Hi,
Tony/Florent. I'm with you on the processing model. Keep the
test code as imported, then (reluctantly) separate the test
generation from the overall result generation.. [and output
formatting?].
Please note that XTC <http://fgeorges.org/xslt/xslt-unit/> is
still there only for historical reasons. I got my laptop stolen
with most of my code (call it a youthful error), and I use Jeni's
XSpec for two years now (which I am a maintainer of now).
Is it practical to further than testing fairly simple values?
Aren't we into a full XML diff engine then?
I tend to cut my XSLT code down to functions and template rules
with simple responsibilities (in addition to a few templates that
just create a bunch of the result tree, like the overall HTML
structure or a piece of Javascript in a <script> element for
instance). Those components are easily unit testable like any
other piece of code in other languages, by writing assertion
expressions (is the content of this subelement equal to this
string, or is the result of that function call equal to 42?)
But for a few components, mostly template rules, diffing
against an expected XML document is really what we need. And
clearly we lack here a general framework to perform XML
comparisons. For instance behind able to compare a result
document to something like:
<root id="...">
<title>...</title>
<para>This exact test.</para>
</root>
That is, a root element, with an id attribute we don't care the
value of, with a title with "something" as content, and a para
element with the exact same content (and do you see the indent
text nodes? do they count in the comparison?) For now, XSpec
provides this exact same semantics (using "..." to mean "every
content"). I feel we need something more rigorous, but that's
very powerful yet.
In XTC I got a way to define your own assertions by defining
something that looked like XSLT stylesheet functions. But that
was too much oriented toward XSLT (XSpec does provide XQuery
support too).
Another big area of investigation is "error expectations", as
Tony suggested. How to test a function is correctly throwing an
error or using xsl:message/@terminate? That is not possible with
the current architecture as the compiled stylesheet is a simple
stylesheet run once for every use cases and creating as result
the XML test report. If there is an error, there is no output at
all.
I think this go to the idea of test harness. For now, all
XSpec does is compiling the test suite to an XSLT stylesheet, and
providing a formating tool to present the report as a nice HTML
page. But we should create different test harnesses for Saxon,
eXist, MarkLogic, Zorba, etc. (taking care of errors within test
cases, pecularities of specific processors, etc.)
In conclusion, I'd say this problem can seem to be quite
simple: create an ad-hoc format for test suites, and compile it
(transform it using XSLT) to an XSLT stylesheet that will run the
test cases and generate a report. But then you run into details
of document comparisons (trust me, that's not an easy one), error
reporting, test harnesses for several processors, etc. So there
is really a benefit in using an existing, maintained framework,
and XSpec seems to be a good candidate among the existing ones).
We expect to release a new version any time soon now. Go to
http://groups.google.com/group/xspec-users to stay tuned and to
http://xspec.googlecode.com/ to learn more about XSpec.
Regards,
--
Florent Georges
http://www.fgeorges.org/
--~------------------------------------------------------------------
XSL-List info and archive: http://www.mulberrytech.com/xsl/xsl-list
To unsubscribe, go to: http://lists.mulberrytech.com/xsl-list/
or e-mail: <mailto:xsl-list-unsubscribe(_at_)lists(_dot_)mulberrytech(_dot_)com>
--~--