At 2022-04-15 14:39 +0000, Eliot Kimber
eliot(_dot_)kimber(_at_)servicenow(_dot_)com wrote:
One of the practical challenges in testing the
output of XSLTs is the sheer cost of maintaining
the test cases themselves: if you?re using some
kind of diff-based checker (this is something we
used for the GAO HTML publishing system using a
home-grown XML diff mechanism) then you have to
maintain the exemplars you?re testing against,
which can be a significant time cost.
In general I?ve found it more practical to
depend on output inspection using both
artificial test case documents where the content
says what the correct expected result is as well
as realistic content. This puts the time cost in
the inspection and depends on the knowledge of
the inspectors but tends to be a lower time cost
overall. This is usually in the context of
relatively low-budget projects or projects where
absolute correctness is not a top concern (i.e.,
not safety critical or in a regulated industry).
Testing PDF output is even more of a
challenge?Antenna House sells a PDF comparator
tool that appears to do what you need (I?ve
never actually used the product) but you face
the same exemplar maintenance challenge as for XML or HTML outputs.
I can attest to how valuable the Antenna House
PDF comparator tool AHRTS has been for the
https://RealtaOnline.com project that I'm on. I
could not live without it. Having it catch
something as subtle as adding an inherited
attribute and not realizing the impact that it
has by moving content over by just one pixel when
I didn't want it to. Never would I have noticed by eye.
And in our situation I have addressed many
customers by writing onion-skin shells around a
core set of stylesheets ... being able to test a
change in the core against all of the customers'
differing uses of the core means that AHRTS will
catch me when a change I make in the core for one
customer impacts the expected results of another customer.
I would not have the confidence in my changes
without this tool watching my back!
. . . . . . Ken
Cheers,
E.
_____________________________________________
Eliot Kimber
Sr Staff Content Engineer
O: 512 554 9368
M: 512 554 9368
<https://www.servicenow.com>servicenow.com
<https://www.linkedin.com/company/servicenow>LinkedIn
| <https://twitter.com/servicenow>Twitter |
<https://www.youtube.com/user/servicenowinc>YouTube
| <https://www.facebook.com/servicenow>Facebook
From: Wendell Piez wapiez(_at_)wendellpiez(_dot_)com
<xsl-list-service(_at_)lists(_dot_)mulberrytech(_dot_)com>
Date: Friday, April 15, 2022 at 8:46 AM
To: xsl-list(_at_)lists(_dot_)mulberrytech(_dot_)com
<xsl-list(_at_)lists(_dot_)mulberrytech(_dot_)com>
Subject: Re: [xsl] How do you prove that your XSLT programs are correct?
[External Email]
Hi,
What's more, Mike doesn't even touch on the
question of defining what is correct, or for
that matter how one of XSLT's great strengths is
in its capability to handle things that are
incorrect by one definition while correct by another.
But yes, unit testing. It is not only the only
way to know, it is the only way to know you really know.
Cheers, Wendell
On Thu, Apr 14, 2022 at 12:52 PM Michael Kay
<mailto:mike(_at_)saxonica(_dot_)com>mike(_at_)saxonica(_dot_)com
<<mailto:xsl-list-service(_at_)lists(_dot_)mulberrytech(_dot_)com>xsl-list-service(_at_)lists(_dot_)mulberrytech(_dot_)com>
wrote:
You're not talking about proving correctness
here, you're talking about testing.
If you want to prove properties of the
stylesheet statically, for example that the
output will conform to a particular schema, then
consider writing schema-aware XSLT code.
(However, I'm very reluctant to talk about
proviing "correctness". You can prove a
hypothesis about the behaviour, but you can't
prove that such behaviour will be considered "correct" in the real world.)
For testing, XSpec is a popular framework, which
roughly does what you describe in (2). This is
also how the W3C XSLT test suite at
<https://urldefense.com/v3/__https:/github.com/w3c/xslt30-test__;!!N4vogdjhuJM!EGPurchrCkklwpQF6l9me3Je_xTvv1dU3Y_XP0DwhWgF2vKxLOw6cg3DXHTq7eRdXAW-pkoiBlgSSc1ghRE7PKIXjqC6SSQwc1R1GX88uKI$>https://github.com/w3c/xslt30-test
works. (That test suite, of course, is geared
more to coverage of constructs in the XSLT
language, so the details will be rather
different from tests of a user stylesheet, but the principle is much the same.)
The biggest weakness I see in the way most users
test their stylesheets is that they use far too
few different source documents to achieve good
coverage of the logic, especially the error cases.
Michael Kay
Saxonica
> On 14 Apr 2022, at 17:08, Roger L Costello
<mailto:costello(_at_)mitre(_dot_)org>costello(_at_)mitre(_dot_)org
<<mailto:xsl-list-service(_at_)lists(_dot_)mulberrytech(_dot_)com>xsl-list-service(_at_)lists(_dot_)mulberrytech(_dot_)com>
wrote:
>
> Hi Folks,
>
> Scenario: You write an XSLT program that
takes some input and outputs XML. You want to
be sure that your XSLT program is correct.
>
> As I see it, there are two ways to help
ensure the correctness of an XSLT program :
>
> 1. Pepper the XSLT program with assert statements, using xsl:assert.
>
> 2. Create a second XSLT program that queries
the XML that was generated by the first XSLT
program. This second XSLT program contains a
series of XPath expressions to check various parts of the XML.
>
> Which of those do you use? Or do you use
both? Or do you use something else?
>
> How do you ensure the correctness of your XSLT programs?
>
> /Roger
>
>
--
...Wendell Piez... ...wendell -at- nist -dot- gov...
...wendellpiez.com... ...pellucidliterature.org... ...pausepress.org...
...<https://urldefense.com/v3/__http:/github.com/wendellpiez.__;!!N4vogdjhuJM!EGPurchrCkklwpQF6l9me3Je_xTvv1dU3Y_XP0DwhWgF2vKxLOw6cg3DXHTq7eRdXAW-pkoiBlgSSc1ghRE7PKIXjqC6SSQwc1R14Qb9XXQ$>github.com/wendellpiez...
...gitlab.coko.foundation/wendell...
<https://urldefense.com/v3/__http:/www.mulberrytech.com/xsl/xsl-list__;!!N4vogdjhuJM!EGPurchrCkklwpQF6l9me3Je_xTvv1dU3Y_XP0DwhWgF2vKxLOw6cg3DXHTq7eRdXAW-pkoiBlgSSc1ghRE7PKIXjqC6SSQwc1R12QKqii8$>XSL-List
info and archive
<https://urldefense.com/v3/__http:/lists.mulberrytech.com/unsub/xsl-list/3453418__;!!N4vogdjhuJM!EGPurchrCkklwpQF6l9me3Je_xTvv1dU3Y_XP0DwhWgF2vKxLOw6cg3DXHTq7eRdXAW-pkoiBlgSSc1ghRE7PKIXjqC6SSQwc1R1RIqInF0$>EasyUnsubscribe
(by email)
<http://www.mulberrytech.com/xsl/xsl-list>XSL-List info and archive
<http://lists.mulberrytech.com/unsub/xsl-list/96802>EasyUnsubscribe
(<>by email)
--
Contact info, blog, articles, etc. http://www.CraneSoftwrights.com/s/ |
Check our site for free XML, XSLT, XSL-FO and UBL developer resources |
Streaming hands-on XSLT/XPath 2 training class @US$125 (5 hours free) |
Essays (UBL, XML, etc.) http://www.linkedin.com/today/author/gkholman |
--~----------------------------------------------------------------
XSL-List info and archive: http://www.mulberrytech.com/xsl/xsl-list
EasyUnsubscribe: http://lists.mulberrytech.com/unsub/xsl-list/1167547
or by email: xsl-list-unsub(_at_)lists(_dot_)mulberrytech(_dot_)com
--~--