On the XML Calabash 3 Cutting Edge

8 min readJust now

–

Would you like to integrate custom XProc 3 steps into XSpec test suites that you run using XProc 3? An exciting feature of the XML Calabash processor enables you to do it, by making your XProc steps callable as functions. While revisiting the helper functionality situation from Ignoring Code Comments During XSpec Testing, this topic illustrates how to use an XProc step as a test helper function, as an alternative to using an XSLT stylesheet function or XQuery function.

In fact, the *same *XProc step can serve as a test helper function in both XSpec tests for XSLT and XSpec tests for XQuery.

Ingredients

Here’s what we need to make this example work:

• XML Calabash v3, because that’s the XProc processor that has the feature to make XProc steps callable as functions.
• XSpec v3.3, which contains new XProc 3 pipelines for running XSpec test suites for XSLT and XQuery. (For more, see Three Cheers for XProc 3 Running XSpec Tests.)
• An XProc library that declares the custom step we want to use as a test helper function.
• An XSpec test file (for XSLT or XQuery) that uses function syntax to call our custom XProc step, within an XPath expression.
• A small XProc pipeline whose purpose is to execute our XSpec test file while having access to the definition of our custom XProc step.
• Willingness to use a nonstandard feature, because calling steps as functions is not part of the XProc standard. As a mitigating factor, XML Calabash tests and documents the feature.

Here’s a preview of the file arrangement we’re aiming for:

Press enter or click to view image in full size

Software Setup

To get XML Calabash 3, download and unzip the latest xmlcalabash-3.*y*.*z*.zip file from Codeberg. XML Calabash 3 requires Java 11 or later.

To get XSpec v3.3, download and unzip one of the v3.3.2 Source code packages from GitHub.

Problem to Solve

In this example, we want XPath expressions within the XSpec test suite to be able to call a custom helper function that takes an XML document, removes comments starting with TEST NOTE:, and preserves all other markup and content in the document. Prior topics in this series implemented the custom helper function in XSLT, XQuery, and XQuery Update Facility. In this topic, we’ll implement the custom helper function using an XProc 3 step. The solution has four subtasks.

1. Declare the XProc Step

In XProc 3, the standard <p:delete> step is capable of removing comments that start with a particular prefix. Our own XProc step can use the following element:

<p:delete match="descendant::comment()    [starts-with(normalize-space(.), 'TEST NOTE:')]"/>

Our XProc step also has one input port representing the XML document from which to remove comments, and one output port for the result of removing comments.

At least in the current XML Calabash 3 version, there is one non-obvious aspect of declaring our own XProc 3 step for use in XSpec as a test helper function: we need to import the step from the XSpec distribution that executes an XSpec test suite. (I am not sure if this requirement will go away in a future version.)

To make our step more portable, we point to the XSpec execution step using a URI that can be resolved with a local XML catalog, regardless of where our XProc code is located. (The URI isn’t meant to represent a location for a web browser to visit. When invoking XML Calabash, we will specify the catalog using the --catalog argument, and the processor will find the files within our local XSpec installation.) Importing the XSpec execution steps for both XSLT tests and XQuery tests enables us to use this step in both kinds of tests.

Putting these pieces together, we get the following XProc file named step-as-test-helper.xpl.

<p:library xmlns:p="http://www.w3.org/ns/xproc" version="3.0"  xmlns:frc="urn:x-xspectacles:functions:helper:remove-comments:xproc">  <p:declare-step type="frc:remove-comments">    <p:import        href="http://www.jenitennison.com/xslt/xspec/xproc/steps/run-xslt"/>    <p:import        href="http://www.jenitennison.com/xslt/xspec/xproc/steps/run-xquery"/>    <p:input port="source"/>    <p:output port="result"/>        <p:delete match="descendant::comment()      [starts-with(normalize-space(.), 'TEST NOTE:')]"/>  </p:declare-step>    </p:library>

2. Call the XProc Step as a Function, in XSpec

Under certain conditions, XML Calabash 3 magically turns the XProc step with type="frc:remove-comments" into an XPath function named frc:remove-comments. The function takes one input argument because the step has one input port.

In XSpec, we want to call that function within XPath expressions in the select attributes of <x:param> and <x:expect> elements. The XSpec code for this example is almost identical to its counterpart in Ignoring Code Comments During XSpec Testing, except for these differences:

• This example does not use <x:helper> in XSpec to gain access to the test helper function. Imports in XProc accomplish the same thing (see subtask 3, below). Maybe a future XSpec version will enhance the <x:helper> element to accommodate XProc steps, but this topic is about what you can do right now in 2025.
• The frc:remove-comments function that arises from the XProc step doesn’t return an XML document or element. Instead, it returns a map. Within the map, the result entry holds the XML document that the XProc step returns on its output port named result. So, to retrieve an output XML document from the XProc step, append ?result to the function call expression, as in frc:remove-comments(dataset)?result. To retrieve the outermost element of the output XML document, further append /element() or /* to the expression, as in (frc:remove-comments(report)?result)/element().

Here is the scenario that uses the frc:remove-comments function within two select attribute values:

<x:scenario label="Use helper function">  <x:call function="rpt:report">    <x:param select="frc:remove-comments(dataset)?result">      <dataset>        <data>100</data>        <!-- TEST NOTE: Next data element exceeds 2^8. -->        <data>257</data>        <data>0</data>      </dataset>    </x:param>  </x:call>  <x:expect label="Report with standard comment"    select="(frc:remove-comments(report)?result)/element()">    <report>      <!--Report about data-->      <report-item>Sum: 357</report-item>      <!-- TEST NOTE: Mean happens to be an integer here. -->      <report-item>Mean: 119</report-item>    </report>  </x:expect></x:scenario>

3. Connect the Dots, in XProc

The XSpec v3.3 distribution has XProc steps that know how to run XSpec test for XSLT or XQuery, and our library has an XProc step that knows how to execute our helper functionality (i.e., deleting certain comments from a document). What we want is to combine those capabilities, to form an XProc step that knows how to run an XSpec test that calls our helper functionality.

How do XSpec and the XSLT/XQuery processor know about the frc:remove-comments step that XML Calabash magically turns into a function? How do they know how to execute the frc:remove-comments function when the test is running?

The answer to both questions is a small XProc pipeline that (a) imports the XProc library that defines the frc:remove-comments step (enabling the magic!) and (b) executes the XSpec test suite that arrives at the input port.

In this example, the small pipeline is named run-xslt-test-with-helper.xpl if we want to run a test for XSLT, and it is named run-xquery-test-with-helper.xpl if we want to run a test for XQuery.

Example 1. run-xslt-test-with-helper.xpl

<p:declare-step xmlns:p="http://www.w3.org/ns/xproc" version="3.0"  xmlns:x="http://www.jenitennison.com/xslt/xspec">  <p:import href="step-as-test-helper.xpl"/>  <p:import      href="http://www.jenitennison.com/xslt/xspec/xproc/steps/run-xslt"/>  <p:input port="source"><!-- XSpec file --></p:input>  <p:output port="result"><!-- HTML test result report --></p:output>  <x:run-xslt/></p:declare-step>

Example 2. run-xquery-test-with-helper.xpl

<p:declare-step xmlns:p="http://www.w3.org/ns/xproc" version="3.0"  xmlns:x="http://www.jenitennison.com/xslt/xspec">  <p:import href="step-as-test-helper.xpl"/>  <p:import      href="http://www.jenitennison.com/xslt/xspec/xproc/steps/run-xquery"/>  <p:input port="source"><!-- XSpec file --></p:input>  <p:output port="result"><!-- HTML test result report --></p:output>  <x:run-xquery/></p:declare-step>

4. Execute the Test Suite

When we want to run the XSpec test suite for this example, we tell XML Calabash to execute the run-xslt-test-with-helper.xpl or run-xquery-test-with-helper.xpl pipeline. The import of step-as-test-helper.xpl makes the operation different from directly running the XSpec test suite using the XProc pipelines in the XSpec distribution.

This section shows sample commands for Windows. Analogous commands should work on Linux or macOS.

First, setting some environment variables will help generalize the test execution commands. The following values assume we installed XSpec v3.3 in C:\xspec3.3 and installed XML Calabash v3.0.20 in C:\xmlcalabash3.

Example 3. Set Environment Variables

set XSPEC_HOME=C:/xspec3.3set XMLCALABASH3_VER=3.0.20set XMLCALABASH3_DIR=C:\xmlcalabash3\xmlcalabash-%XMLCALABASH3_VER%set XMLCALABASH3_JAR=%XMLCALABASH3_DIR%\xmlcalabash-app-%XMLCALABASH3_VER%.jar

Next, we navigate to the directory containing the example files.

Then, we run the test suite against the XSLT or XQuery test target as follows.

Example 4. Run Test for XSLT

java -jar "%XMLCALABASH3_JAR%" --catalog:%XSPEC_HOME%/catalog.xml --input:source=helper-remove-comments-xproc3.xspec --output:result=helper-remove-comments-xproc3-xslt-result.html run-xslt-test-with-helper.xpl

Example 5. Run Test for XQuery

java -jar "%XMLCALABASH3_JAR%" --catalog:%XSPEC_HOME%/catalog.xml --input:source=helper-remove-comments-xproc3.xspec --output:result=helper-remove-comments-xproc3-xquery-result.html run-xquery-test-with-helper.xpl

The command has these pieces:

• -jar "%XMLCALABASH3_JAR%" tells Java to run XML Calabash.
• --catalog:%XSPEC_HOME%/catalog.xml helps the processor find files within our XSpec installation.
• --input:source=helper-remove-comments-xproc3.xspec points to the XSpec test suite we want to run.
• --output:result=... points to the HTML test report file we want to create.
• run-xslt-test-with-helper.xpl or run-xquery-test-with-helper.xpl specifies the pipeline we want XML Calabash to execute, based on the language we are testing.

Output

The output looks like the following for XSLT and XQuery, respectively. Output is a little more verbose for XSLT.

Example 6. Output of Test for XSLT

XSpec v3.3.2Checking for deprecated Saxon versions: PassedTesting with SAXON HE 12.9Use helper functionReport with standard commentPENDING: (Fails due to TEST NOTE: comments) Without helper functionPENDING: (Fails due to TEST NOTE: comments) Report with standard commentpassed: 1 / pending: 1 / failed: 0 / total: 2

Example 7. Output of Test for XQuery

XSpec v3.3.2Checking for deprecated Saxon versions: Passedpassed: 1 / pending: 1 / failed: 0 / total: 2

Key Takeaways

• XML Calabash v3, one of the actively developed XProc 3 processors, makes custom XProc steps callable as XPath functions. While nonstandard, this feature is useful because XPath function calls can go in lots of places, such as in a select or test attribute of certain XSpec elements in a test suite. Many thanks to Norm Tovey-Walsh for thinking of this feature and implementing it!
• We used this feature to implement our desired test helper functionality as an XProc step. We called the same XProc step in an XSpec test for XSLT and also an XSpec test for XQuery.
• In this case, the XProc step is small and simple, but you might require complicated functionality that’s easier to implement in XProc than in XSLT or XQuery. Or, you might already have a custom XProc step that implements your desired functionality; no need to rewrite it in XSLT or XQuery if you can call the XProc step as a function.
• Importing our custom XProc step using the <p:import> element caused XML Calabash to do its magic of turning the step into a callable function.
• How about testing your custom XProc steps using XSpec? I’m imagining that a future XSpec version will make that possible, building on this same step-as-function feature and the existing XSpec capabilities for testing functions.