One of the practical challenges in testing the output of XSLTs is the sheer
cost of maintaining the test cases themselves: if you’re using some kind of
diff-based checker (this is something we used for the GAO HTML publishing
system using a home-grown XML diff mechanism) then you have to maintain the
exemplars you’re testing against, which can be a significant time cost.
In general I’ve found it more practical to depend on output inspection using
both artificial test case documents where the content says what the correct
expected result is as well as realistic content. This puts the time cost in the
inspection and depends on the knowledge of the inspectors but tends to be a
lower time cost overall. This is usually in the context of relatively
low-budget projects or projects where absolute correctness is not a top concern
(i.e., not safety critical or in a regulated industry).
Testing PDF output is even more of a challenge—Antenna House sells a PDF
comparator tool that appears to do what you need (I’ve never actually used the
product) but you face the same exemplar maintenance challenge as for XML or
HTML outputs.
Cheers,
E.
_____________________________________________
Eliot Kimber
Sr Staff Content Engineer
O: 512 554 9368
M: 512 554 9368
servicenow.com<https://www.servicenow.com>
LinkedIn<https://www.linkedin.com/company/servicenow> |
Twitter<https://twitter.com/servicenow> |
YouTube<https://www.youtube.com/user/servicenowinc> |
Facebook<https://www.facebook.com/servicenow>
From: Wendell Piez wapiez(_at_)wendellpiez(_dot_)com
<xsl-list-service(_at_)lists(_dot_)mulberrytech(_dot_)com>
Date: Friday, April 15, 2022 at 8:46 AM
To: xsl-list(_at_)lists(_dot_)mulberrytech(_dot_)com
<xsl-list(_at_)lists(_dot_)mulberrytech(_dot_)com>
Subject: Re: [xsl] How do you prove that your XSLT programs are correct?
[External Email]
Hi,
What's more, Mike doesn't even touch on the question of defining what is
correct, or for that matter how one of XSLT's great strengths is in its
capability to handle things that are incorrect by one definition while correct
by another.
But yes, unit testing. It is not only the only way to know, it is the only way
to know you really know.
Cheers, Wendell
On Thu, Apr 14, 2022 at 12:52 PM Michael Kay
mike(_at_)saxonica(_dot_)com<mailto:mike(_at_)saxonica(_dot_)com>
<xsl-list-service(_at_)lists(_dot_)mulberrytech(_dot_)com<mailto:xsl-list-service(_at_)lists(_dot_)mulberrytech(_dot_)com>>
wrote:
You're not talking about proving correctness here, you're talking about testing.
If you want to prove properties of the stylesheet statically, for example that
the output will conform to a particular schema, then consider writing
schema-aware XSLT code. (However, I'm very reluctant to talk about proviing
"correctness". You can prove a hypothesis about the behaviour, but you can't
prove that such behaviour will be considered "correct" in the real world.)
For testing, XSpec is a popular framework, which roughly does what you describe
in (2). This is also how the W3C XSLT test suite at
https://github.com/w3c/xslt30-test<https://urldefense.com/v3/__https:/github.com/w3c/xslt30-test__;!!N4vogdjhuJM!EGPurchrCkklwpQF6l9me3Je_xTvv1dU3Y_XP0DwhWgF2vKxLOw6cg3DXHTq7eRdXAW-pkoiBlgSSc1ghRE7PKIXjqC6SSQwc1R1GX88uKI$>
works. (That test suite, of course, is geared more to coverage of constructs
in the XSLT language, so the details will be rather different from tests of a
user stylesheet, but the principle is much the same.)
The biggest weakness I see in the way most users test their stylesheets is that
they use far too few different source documents to achieve good coverage of the
logic, especially the error cases.
Michael Kay
Saxonica
On 14 Apr 2022, at 17:08, Roger L Costello
costello(_at_)mitre(_dot_)org<mailto:costello(_at_)mitre(_dot_)org>
<xsl-list-service(_at_)lists(_dot_)mulberrytech(_dot_)com<mailto:xsl-list-service(_at_)lists(_dot_)mulberrytech(_dot_)com>>
wrote:
Hi Folks,
Scenario: You write an XSLT program that takes some input and outputs XML.
You want to be sure that your XSLT program is correct.
As I see it, there are two ways to help ensure the correctness of an XSLT
program :
1. Pepper the XSLT program with assert statements, using xsl:assert.
2. Create a second XSLT program that queries the XML that was generated by
the first XSLT program. This second XSLT program contains a series of XPath
expressions to check various parts of the XML.
Which of those do you use? Or do you use both? Or do you use something else?
How do you ensure the correctness of your XSLT programs?
/Roger
--
...Wendell Piez... ...wendell -at- nist -dot- gov...
...wendellpiez.com... ...pellucidliterature.org... ...pausepress.org...
...github.com/wendellpiez.<https://urldefense.com/v3/__http:/github.com/wendellpiez.__;!!N4vogdjhuJM!EGPurchrCkklwpQF6l9me3Je_xTvv1dU3Y_XP0DwhWgF2vKxLOw6cg3DXHTq7eRdXAW-pkoiBlgSSc1ghRE7PKIXjqC6SSQwc1R14Qb9XXQ$>..
...gitlab.coko.foundation/wendell...
XSL-List info and
archive<https://urldefense.com/v3/__http:/www.mulberrytech.com/xsl/xsl-list__;!!N4vogdjhuJM!EGPurchrCkklwpQF6l9me3Je_xTvv1dU3Y_XP0DwhWgF2vKxLOw6cg3DXHTq7eRdXAW-pkoiBlgSSc1ghRE7PKIXjqC6SSQwc1R12QKqii8$>
EasyUnsubscribe<https://urldefense.com/v3/__http:/lists.mulberrytech.com/unsub/xsl-list/3453418__;!!N4vogdjhuJM!EGPurchrCkklwpQF6l9me3Je_xTvv1dU3Y_XP0DwhWgF2vKxLOw6cg3DXHTq7eRdXAW-pkoiBlgSSc1ghRE7PKIXjqC6SSQwc1R1RIqInF0$>
(by email<>)
--~----------------------------------------------------------------
XSL-List info and archive: http://www.mulberrytech.com/xsl/xsl-list
EasyUnsubscribe: http://lists.mulberrytech.com/unsub/xsl-list/1167547
or by email: xsl-list-unsub(_at_)lists(_dot_)mulberrytech(_dot_)com
--~--