Re: [xsl] XSLT Unit Testing and Coverage

Yes that's what we do -- maintain a set of tests we run via JUnit that 
assert expected outputs (using XPath) given sample documents.  Having 
such tests is invaluable, as a tool to express the meaning of the code, 
to enforce that it continue to work as it evolves, and all the usual 
testing goodness.  I wouldn't call these unit tests, though; they are 
more akin to so-called "integration" tests.  Running tests via XSLT is 
of course possible, but we prefer JUnit since it is embedded in an 
ecosystem of helpful tools (like the test runners in eclipse and in 
jenkins).
Measuring test coverage we have also found useful, but less so. My team 
invested some effort in getting Cakupan working but ran into some 
roadblocks and ended up implementing our own (Saxon-specific) solution.  
We had a certain amount of discussion about whether line-oriented 
coverage metrics made sense for XSLT but ended up implementing that 
since it was easiest.  One time coverage tools are very useful is when 
unlimbering old code that may not be well maintained: a coverage 
measurement can give an idea of how complete the test suite is, and may 
point areas of dead code.
-Mike


On 5/29/2014 4:20 AM, Alan Painter alan(_dot_)painter(_at_)gmail(_dot_)com 
wrote:
> If you are already using a unit test framework from Java or Scala or 
> such with hamcrest matchers, you can use XML matchers to test the 
> results.
> From java, fire up a transformation and then test the results using 
> hamcrest XML matchers.
> You can decide the level of detail that you want to go to, from 
> overall,  end-to-end testing to individual functions and templates.  
> For functions and templates,  you'll probably need to write test XSLT 
> stylesheets that include the modules containing the details that 
> interest you so that they are exposed to a test transformation.
> Best of luck
>
> On May 29, 2014 9:11 AM, "BR Chrisman brchrisman(_at_)gmail(_dot_)com 
> <mailto:brchrisman(_at_)gmail(_dot_)com>" 
> <xsl-list-service(_at_)lists(_dot_)mulberrytech(_dot_)com 
> <mailto:xsl-list-service(_at_)lists(_dot_)mulberrytech(_dot_)com>> wrote:
>     On Wed, May 28, 2014 at 10:38 PM, Ihe Onwuka 
> ihe(_dot_)onwuka(_at_)gmail(_dot_)com
>     <mailto:ihe(_dot_)onwuka(_at_)gmail(_dot_)com>
>     <xsl-list-service(_at_)lists(_dot_)mulberrytech(_dot_)com
>     <mailto:xsl-list-service(_at_)lists(_dot_)mulberrytech(_dot_)com>> wrote:
>
>         On Wed, May 28, 2014 at 8:38 PM, Ihe Onwuka
>         <ihe(_dot_)onwuka(_at_)gmail(_dot_)com 
> <mailto:ihe(_dot_)onwuka(_at_)gmail(_dot_)com>> wrote:
>
>
>             On Wed, May 28, 2014 at 7:57 PM, Vasudev Kandhadai
>             vasu(_dot_)kandhadai(_at_)gmail(_dot_)com 
> <mailto:vasu(_dot_)kandhadai(_at_)gmail(_dot_)com>
>             <xsl-list-service(_at_)lists(_dot_)mulberrytech(_dot_)com
>             <mailto:xsl-list-service(_at_)lists(_dot_)mulberrytech(_dot_)com>> 
> wrote:
>
>                 Dear All,
>                 is there a good reason to deploy a XSLT unit testing
>                 framework?
>
>
>             No.
>
>                 I have never seen any serious XSLT dev env where the
>                 XSLT unit testing was either done religiously, or
>                 considered mandatory.  Other than a very religious
>                 Java development team with strict Junit set up with
>                 Maven etc, who have adopted XSLT into their dev env,
>                 who would now want to extend the same ideologies to
>                 the XSLT world?  I have personally never used or
>                 utilized practically any XSLT unit testing framework
>                 in any project and nor was there any requirement to do
>                 so...
>
>
>             Why is Java a valid reference point. It's a completely
>             different language.
>
>
>         Right. This merits amplification. The phrase "Unit Test
>         Frameworks"  has acquired in my view a specific connotation
>         related to ideas from Test Driven Development. They are a
>         creature that evolved from the procedural programming
>         community to solve problems that arise during the development
>         of procedural programs.
>
>         XSLT done right is declarative. The programmer does not have
>         the same level of control over what processing (and therefore
>         what tests) gets done when. So before adopting a methodology
>         founded on "Unit testing frameworks" the first question I
>         would ask is - in XSLT what should constitute a unit - or to
>         put it more finely what is the smallest component that should
>         be the subject of a discrete testing effort.
>
>         Is it a stylesheet. I don't think so, at least not if you are
>         coding declaratively. How would I think about it. Well what is
>         more useful on a bug report - that there is a bug on
>         stylesheet X, or that  executing tests targeted at template Y
>         in stylesheet X exposed a bug.  So I would say the focus on
>         testing an XSLT program should be at the template rule level
>         and if I were to adopt any sort of test driven methodology it
>         might evolve around the concept of the template rule as a unit
>         (with all that entails).  That however is  a big if.
>
>
>     If everything needed to test a particular template could be
>     assembled, and the template's name is passed in as initial
>     template... then there's mocking calls like the various
>     input/output doc stuff... and parameters to the template...  But
>     it sounds like a lot of the tools out there do this already, like:
>     "show me only the nodes resulting from this template".
>
>     I can see how that might be useful... all my xslt projects have
>     been implemented as series of small stylesheets applied in a
>     layering/pipeline, so I'd generally achieve mostly the same thing
>     by isolating a template (or a few templates) in a particular
>     layer/stage and using a 'tee' type utility... (works for
>     identity-based transforms).
>
>     I'm thinking for my next such project, having a stylesheet
>     transform the source stylesheet, adding a debug namespace and
>     instructions to add debug namespace elements to the result nodes
>     of specific templates, so I can track the output of those without
>     having to isolate.
>
>
>
>          If someone were to sit down and design from scratch a testing
>         methodology acclimated to XSLT in particular and declarative
>         programming in general it would not look like nUnit. The
>         efficacy of these testing methodologies is oversold. Similar
>         benefits would accrue to any effort that entailed the
>         automation of test execution. What nUnit has done is increase
>         the number of programmers that are willing to be involved in
>         testing by turning it into a programming activity and that has
>         a knock on beneficial effect especially in the paradigm from
>         which these methods evolved.
>         XSL-List info and archive
>         <http://www.mulberrytech.com/xsl/xsl-list>
>         EasyUnsubscribe <http://-list/965995> (by email)
>
>
>     XSL-List info and archive <http://www.mulberrytech.com/xsl/xsl-list>
>     EasyUnsubscribe <-list/552232> (by email)
>
> XSL-List info and archive <http://www.mulberrytech.com/xsl/xsl-list>
> EasyUnsubscribe <-list/648163> 
> (by email <>)
--~----------------------------------------------------------------
XSL-List info and archive: http://www.mulberrytech.com/xsl/xsl-list
EasyUnsubscribe: http://lists.mulberrytech.com/unsub/xsl-list/1167547
or by email: xsl-list-unsub(_at_)lists(_dot_)mulberrytech(_dot_)com
--~--