README
¶
go-xml
XML Schema 1.0/1.1 validation, XPath 2.0/3.0/3.1, XQuery 3.1 and XSLT 2.0/3.0 in pure Go. No cgo,
no JVM, no libxml2 — one go get, and it cross-compiles like any Go package.
go get github.com/knroy/go-xml
Validate a document against a schema:
schema, err := xsd.LoadFile("schemas/invoice.xsd", xsd.Options{})
tree, err := xdm.ParseString(src, xdm.ParseOptions{})
err = schema.Validate(tree.Root, xsd.ValidateOptions{})
Every error carries the spec's code and a path — cvc-datatype-valid.1 at
/invoice/total — not just "invalid".
For a document you did not write, use schema.ValidateContext(ctx, ...): it is
the same call with a deadline, and identity-constraint checking is where an
untrusted document can make validation expensive. See
docs/security.md.
The packages
xdm |
the parser and the XDM tree everything else reads |
xsd |
XML Schema 1.0 and 1.1: assembly, validation, PSVI annotation |
xpath |
XPath 2.0: lexer, parser, evaluator, fn: library |
xquery |
XQuery 3.1: constructors, FLWOR, the prolog |
xslt |
XSLT 2.0: templates, modes, keys, grouping, serialisation |
Safe by default
Aimed at input arriving over the wire, so the defaults are the settings you
would choose for untrusted documents. Options{} is already hardened:
- DOCTYPE is refused, which closes XXE and entity expansion at the door
- No network access — a
schemaLocationcannot make this process fetch xsi:schemaLocationis ignored, so a document cannot pick its own schema- Size, node-count and depth limits are on, sized for a general service
See docs/security.md for the threat model and docs/server.md for a validating HTTP endpoint measured against XXE, billion-laughs and resource exhaustion.
Building and checking
go build ./...
go test -race ./...
tests/check.sh runs everything that has to pass before a change is done —
build, vet, unit tests, race, every W3C suite and the production corpora.
docs/testing.md covers the layers, the environment
variables, the ratchet, and how to read a result:
tests/check.sh fast # no external suites
GOXSLT_UBL=<dir> GOXSLT_CII=<dir> tests/check.sh # everything
The real-world stylesheet corpora are found under testdata/ by default, or
pointed at a checkout of your own:
git clone --depth 1 https://github.com/docbook/xslt3ng testdata/xsltng
git clone --depth 1 https://github.com/xspec/xspec testdata/xspec
DocBook needs its localisation files generated first — the build compiles them
from src/main/locale/, and this engine can do it:
for f in testdata/xsltng/src/main/locale/*.xml; do
go run ./cmd/go-xml -xsl testdata/xsltng/src/main/xslt/modules/xform-locale.xsl \
-allow-dir testdata/xsltng -o testdata/xsltng/src/main/xslt/locale/"$(basename "$f")" "$f"
done
A missing suite is reported as skipped; a suite that is present but produces no result is a failure. A check that did not run must not look like one that succeeded.
One dependency: golang.org/x/text, for Unicode normalisation and
language-sensitive collation. Nothing else is outside the standard library.
Requires Go 1.25 or later. The floor is measured, not nominal: regexp
learned the Unicode category Cn in 1.25, and building on 1.24 costs four
conformance cases. See docs/testing.md.
Live test
go-xml fiddle runs a stylesheet against a document in the browser and shows you the result. No Go toolchain, no checkout — paste the XSLT or XQuery you already have and see what this engine makes of it.
It is the work of Martin Honnen, who built and maintains it as a project of his own.
Status
| XPath 2.0 | 100.00% of the W3C QT3 suite (15,217 of 15,217 in scope) |
| XPath 3.0 | 100.00% of the W3C QT3 suite (19,362 of 19,362 in scope) |
| XPath 3.1 | 100.00% of the W3C QT3 suite (21,898 of 21,898 in scope); maps, arrays, the lookup operator, the JSON family |
| XQuery 3.1 | 100.00% of the W3C QT3 suite (30,345 of 30,346 in scope); constructors, FLWOR, the prolog, try/catch, switch, typeswitch, windows, and both halves of import — module and schema. Schema import brought 416 cases into scope and 318 more passes; the 1 remaining failure is the tail catalogued in todo.md §1.5 |
| XSLT 2.0 | 99.87% of the W3C XSLT suite filtered to 2.0 (6,193 of 6,201 in scope); verified against Saxon-HE 12.4 on two production corpora |
| XSLT 3.0 | 99.77% of the W3C XSLT suite filtered to 3.0 (11,492 of 11,518 in scope). Streaming is now measured rather than excluded, which is why the denominator grew by 2,862 cases: 8 of the 28 failures want the XTSE3430 that more of the §19.8 posture-and-sweep analysis would emit — see Where it fails. Also measured against DocBook xslTNG and XSpec — see Real-world stylesheets |
| XSD 1.0 | 99.89% of the W3C xsdtests instance tests (24,973 of 25,000); 99.98% of its schema-validity tests (14,385 of 14,388) |
| XSD 1.1 | 99.98% instance (26,217 of 26,222); 99.97% schema-validity (15,350 of 15,354); opt-in via Version11 |
| RELAX NG | 100.00% of James Clark's spectest (965 of 965 assertions); XML and compact syntax |
| DTD | content models, attribute defaults, enumerations, ID/IDREF; external subset, parameter entities across both subsets, conditional sections — via dtd.Load with a caller-supplied resolver, nothing fetched by default |
| Tests | 2,416 func Test declarations, clean under -race (a few subtests skip without the corpora below) |
| Production schemas | UBL 2.1, UN/CEFACT CII, Factur-X/ZUGFeRD, Peppol BIS 3.0 — 88 schemas load, instances validate clean |
| API | 1.2; the exported surface is stable and additive over 1.1, and a breaking change means 2.0 with a new module path |
Read this before adopting it. Three things are commonly assumed and are not true here:
-
Both halves of the XSD numbers carry a residue of disagreements — 30 on 1.0 and 31 on 1.1 — and where they are schema-validity failures, a schema invalid in one of those ways is accepted rather than reported. They are listed in Where it fails, along with what the suite skips and why; most of them — 57 of 61 — are cases the W3C itself has queried or filed a bug against, including every one of the 44
MS-Regexdisagreements, which are a single open bug. A further 16 cases on 1.0 and 14 on 1.1 are excluded from both sides of the ratio because the suite marks themindeterminate— the working group left the area underspecified, so no result is prescribed and none is scored. -
A backreference to a variable-width group is refused, by design. RE2 has no backreferences, which is also why no pattern can hang this engine. Where the group has a fixed width the backreference is resolved exactly, in linear time — that covers
(a)\1,([md])[aeiou]\1and the like. Where it can vary,(a*)\1, there is more than one way to split the match and RE2 reports only one, so the answer isFORX0002rather than a guess. The XML Schema pattern facet has no backreference at all and rejects them outright, which is conformant: Appendix F's grammar has no form for one. -
XSLT 3.0 is the youngest of the measured numbers, at 99.77%, and still the one to check against your own stylesheets first. It no longer has a concentration: package composition was about a third of the failures and is now 4 of 13, all four documented as unreachable rather than outstanding. What is left is a long tail of one or two cases across thirty test sets, which is harder to summarise but easier to live with — no single feature is systematically weak. The corpus differential against Saxon remains stronger evidence for real stylesheets than the percentage.
Neither XSLT number is directly comparable to the XPath and XSD ones. There is no maintained XSLT 2.0 suite, so both are the XSLT 3.0 suite filtered by each test's declared version dependency — a different kind of measurement from running a suite written for the version under test.
Streaming is not implemented, and its 2,646 cases are out of scope rather than counted as failures. That is the single largest gap, and it is architectural rather than a matter of filling in instructions: streaming wants a pull parser and a streamability static analysis, not another feature. What is absent is streamed execution, not the vocabulary:
xsl:stream,xsl:fork,xsl:source-document,xsl:mergeandxsl:accumulatorall execute, by building the tree instead. §19.1 allows exactly that — a processor not claiming the streaming option "must still process a stylesheet and deliver the correct results, but is not required to use streaming algorithms". What a streamable stylesheet computes is another question — the spec requires it to be the same as a non-streaming evaluation, so thesi-*sets, which assert results rather than memory behaviour, are in scope and nearly all pass. Where it fails sets out the rest.
Every remote-reference mechanism — DOCTYPE, fn:doc, file reads — is off
unless enabled; see Security defaults.
What this is
Seven packages, each usable on its own:
| Package | What it holds |
|---|---|
xdm |
The XQuery/XPath data model: typed atomic values, the node tree, an XML parser |
xpath |
XPath 2.0, 3.0 and 3.1: lexer, parser, evaluator, and the fn:, map:, array: and math: function libraries |
xquery |
XQuery 3.1: constructors, FLWOR, the prolog, and the XQuery-only expression forms |
xslt |
XSLT 2.0 and 3.0: pattern matching, the stylesheet compiler, the transform runtime, serialisation |
xsd |
XML Schema 1.0 and 1.1: the component model, schema assembly, content models, facets, identity constraints |
dtd |
DTD validation: content models, attribute defaults, ID/IDREF, external subsets |
relaxng |
RELAX NG: the derivative algorithm, the section 7 restrictions, the XSD datatype library, the compact syntax |
cmd/go-xml |
A command-line transformer |
Documentation
- docs/xquery.md — XQuery 3.1: entry points, output, external variables, options
- docs/validation.md — XSD, DTD and RELAX NG validation, Schematron, and which kind of "valid" you actually need. Start here if you came to check a document against a schema.
- docs/xsd.md — the schema validator in detail: 1.0 versus 1.1, resolvers and what they will fetch, the PSVI, limits, concurrency, and what the conformance figures cover.
- docs/server.md — compile once, transform or validate per request: a complete XSLT service, a hardened XSD validation endpoint (measured against XXE, billion-laughs and resource exhaustion), timeouts, limits, hot-reloading rule sets.
- docs/recipes.md — batching, splitting, HTML rendering, parameters, custom resolvers, standalone XPath.
- docs/options.md — every option in
xdm,xpath,xsdandxslt: what each field does, what the zero value means, and the limits that bound a parse, a validation and a transform. - SECURITY.md — how to report a vulnerability, and what counts as one.
- docs/security.md — threat model and the results of three audits: XXE is closed, entity expansion is bounded per reference rather than per entity, and the third audit's findings are written up including the one left open.
- CHANGELOG.md — what each release contains, and what the 1.0 stability promise does and does not cover.
- RELEASE.md — how a release is cut: the manual steps in the order that matters, and what the tag-push workflow checks and refuses.
- docs/known-gaps.md — every measured failure and why it is still open, including the fix attempts that were reverted because they cost more than they gained.
Architecture
The layering is strict and one-directional. Both host languages use XPath, XPath uses XDM, and XDM knows nothing about any of them — which is what keeps the data model honest when a host language needs something awkward.
XSLT and XQuery are siblings rather than layers: neither uses the other, and
both compile their expressions with xpath and build their result trees with
xdmbuild. That sharing is the reason XQuery arrived at 99% in one push
rather than being a second engine — the expression language and the ~437
functions were already there and already at 100%.
cmd/go-xml command-line transformer
│
┌─────▼─────────────────────────┐ ┌──────────────────────────────┐
│ xslt stylesheet compiler │ │ xquery constructors · FLWOR │
│ patterns · templates │ │ the prolog │
│ instructions · output │ │ │
└─────┬─────────────────────────┘ └─────┬────────────────────────┘
│ │
│ ┌─────────────────────────┘
│ │ both build result trees with xdmbuild, and hand
│ │ every expression to xpath to compile and evaluate
┌─────▼────────▼─────────────────────────────────┐
│ xpath lexer → parser → optimiser → runtime │
│ functions · operators · type system │
└─────┬──────────────────────────────────────────┘
│ every value is an Item; every result a Sequence
┌─────▼──────────────────────────────────────────┐
│ xdm nodes · atomic values · sequences │
│ QNames · the XML parser │
└────────────────────────────────────────────────┘
XDM is the centre, not a wrapper over encoding/xml. The Go decoder is
used as a tokeniser only: it resolves prefixes into Name.Space and then
discards both the prefix and the xmlns declarations, and XSLT needs both —
namespace nodes are addressable on the namespace axis, and a literal result
element must serialise with the prefix its author wrote. So the tree is built
here. encoding/xml appears in exactly one file.
Three types carry the model. Item is a closed interface over *Node,
*Atomic, and *Opaque (engine-internal state threaded through the same
interface). Sequence is a flat []Item — XDM has no nested sequences, and
every constructor maintains that. Atomic is a tagged union rather than
interface{}, because XPath's type system is not Go's: xs:decimal is a
big.Rat so 0.1 + 0.2 is exactly 0.3, and xs:dateTime is not time.Time
because XML Schema timezones are optional and "absent" is a distinct value
from +00:00.
Static and dynamic context are separate types. Namespace bindings, the
default element namespace, and the function library are resolved at parse
time through NamespaceResolver. The context item, position, size, variables,
clock, and timezone live on Context at evaluation time. Merging them is a
trap: XSLT compiles one pattern once and evaluates it against every node, so
anything static must not be re-resolved per node.
Compilation is separate from execution. Compile is the expensive step and
produces an immutable Stylesheet; Transform does not mutate it. That is what
makes one compiled rule set safe to share across goroutines, and it is tested
under -race with a shared source tree as well as a shared stylesheet.
Patterns are matched right to left. An XSLT match pattern looks like a
path but means "does this node match", not "navigate from here". Evaluating the
path and testing membership is quadratic; matching upward from the candidate is
O(depth), which is the difference between finishing on a large document and not.
Use
sheetTree, err := xdm.ParseString(stylesheetSource, xdm.ParseOptions{})
sheet, err := xslt.Compile(sheetTree.Root, xslt.CompileOptions{})
docTree, err := xdm.ParseString(documentSource, xdm.ParseOptions{})
result, err := sheet.Transform(ctx, docTree.Root, xslt.TransformOptions{})
fmt.Println(result.String()) // serialised per xsl:output
result.Tree() // or keep navigating it
result.Messages // xsl:message output, collected not printed
result.Secondary // documents from xsl:result-document
Each SecondaryResult carries the Href the stylesheet asked for, its own
Nodes, and the OutputSettings that apply to it, and serialises itself:
for _, doc := range result.Secondary {
fmt.Println(doc.Href, doc.String())
}
Nothing is written to disk unless you write it — see Security defaults.
Compile is the expensive step and Transform does not mutate the compiled
stylesheet, so compile once and transform concurrently — the test suite
exercises that under -race.
XPath alone:
ctx := xpath.NewContext(docTree.Root, xpath.Builtins())
seq, err := xpath.Eval("sum(//invoice/@total)", ctx, nil)
XQuery, which compiles once and evaluates concurrently the same way a stylesheet does. A query returns a sequence; serialising it is a separate step:
q, err := xquery.Compile(`
<totals>{
for $i in //invoice
group by $y := year-from-date(xs:date($i/@date))
order by $y
return <year n="{$y}">{ sum($i/@total) }</year>
}</totals>`, xquery.Options{})
seq, err := q.Eval(xpath.NewContext(docTree.Root, xpath.Builtins()))
err = xslt.Serialize(os.Stdout, seq, xslt.OutputSettings{OmitXMLDecl: true}, nil)
// <totals><year n="2023">7</year><year n="2024">15</year></totals>
The xs:date cast is not decoration: an attribute in an unvalidated document
is xs:untypedAtomic, and year-from-date refuses it with XPTY0004 rather
than guessing.
See docs/xquery.md for options, external variables and what is not implemented.
CLI:
go-xml -xsl transform.xsl input.xml
go-xml -xsl rules.xsl -p year=2024 -allow-dir ./codelists invoice.xml
# Start at a named template. A stylesheet that generates its own content
# needs no source document, so none is given.
go-xml -xsl generate.xsl -initial-template main
# Validate a directory: report every failure rather than stopping at the first.
go-xml -xsl rules.xsl -keep-going invoices/*.xml
# Pin the clock and timezone so the run is reproducible.
go-xml -xsl report.xsl -now 2024-01-15T09:00:00Z -timezone 240 in.xml
# Report the source line each validation failure occurred on.
go-xml -xsl rules.xsl -track-positions invoice.xml
# Write the documents an xsl:result-document stylesheet produces.
go-xml -xsl split.xsl -result-dir ./out catalogue.xml
| Flag | Effect |
|---|---|
-xsl |
the stylesheet to apply (required) |
-o |
write to a file instead of stdout |
-p name=value |
supply a top-level xsl:param; repeatable |
-allow-dir |
open xsl:include/xsl:import/doc()/document() to further directories, each covering its subdirectories to any depth; the stylesheet's own directory is always readable. It says where, not what: raw text, external entities and XInclude each need their own flag as well |
-allow-doctype |
permit a DOCTYPE in the source |
-timeout |
bound the transform (default 60s) |
-initial-template |
start at a named template instead of matching the root; no input document is then needed |
-mode |
initial mode for apply-templates |
-messages |
print xsl:message output to stderr |
-now |
pin fn:current-dateTime (RFC 3339) |
-timezone |
implicit timezone in minutes |
-track-positions |
record source line/column; see below |
-result-dir |
where xsl:result-document outputs with an href are written; one with no href goes to the principal output instead |
-keep-going |
continue a batch past a failure, still exiting non-zero |
The exit status is 0 only if every input transformed.
Design notes
Compilation has an optimisation stage. Compile folds closed
sub-expressions to literals before evaluation, so (1 + 2) * 3 costs one
literal load per node rather than three operations — 13.77 ns for a whole
arithmetic expression. Two conditions guard every rewrite: the expression must
be closed (no variable, context item, or context-dependent function), and the
rewrite must preserve errors as well as values. 1 idiv 0 is deliberately not
folded, because refusing a stylesheet at compile time for an error in a branch
that is never taken would be wrong.
Foldable functions are an allowlist rather than a denylist. A user-defined function is never folded even if it happens to be pure, because the library it resolves through is supplied by the caller.
Errors carry their specification code as a field. A message is prose that
may be reworded; a code is what a caller branches on and what a conformance
suite compares. xdm.ErrorCode(err) returns it, unwrapping as it goes, and
falls back to recognising the code where an error still carries it as a message
prefix. The rendered text is unchanged, so nothing reading error strings had to
move.
This is not cosmetic: making codes inspectable let the QT3 harness stop accepting any error where a specific one was expected, which immediately found 961 wrong codes that the loose check had been hiding.
Aggregates over a range are arithmetic, not iteration. An integer range is fully determined by its bounds, so nothing has to be built to answer a question about it:
count(lo to hi) |
hi - lo + 1 |
sum(lo to hi) |
n(first + last) / 2 |
min / max |
the bounds themselves |
avg |
the midpoint |
sum(1 to 10000000) returns 50000005000000 in 6 MB rather than materialising
ten million values at 1.9 GB. The series is summed in big.Int, because
n(first+last) overflows int64 well before the bounds do —
sum(1 to 5000000000000) is about 1.25 × 10²⁵ and this engine returns it
exactly, where Saxon refuses the range outright (its sequences are capped at
int32).
The general form of the idea is a lazy sequence type carrying its own
cardinality — don't ask a sequence to enumerate itself if the operation can be
answered from its structure. That would mean making xdm.Sequence an
interface rather than a slice, which every operation across three packages
indexes and ranges over directly. The narrower version gets the same result for
the cases that arise, because lo to hi is the only XPath 2.0 construct that
can name a sequence too large to hold: a path expression is bounded by the
document, a literal sequence by the length of the stylesheet.
The recognition is deliberately narrow — only a bare to directly under the
aggregate. A predicate, a for, or a comma sequence changes which items
survive, so those evaluate normally and meet the item budget instead:
sum((1 to 10)[. mod 2 = 0]) is 30, not 55. That narrowness is what lets the
budget stay strict, and it is what the tests check hardest.
Sequences and types, not node-sets and strings. XPath 1.0 had four types
and coerced freely between them. 2.0 replaces that with sequences of typed
items and explicit promotion rules, and most of the difficulty of the language
is there rather than in the syntax. xs:decimal is a big.Rat, so 0.1 + 0.2
is exactly 0.3; xs:integer div xs:integer yields a decimal; the four
numeric types promote along integer → decimal → float → double before any
comparison. Implementing this with float64 throughout would pass a
surprising number of tests and then quietly misreport a monetary total.
Dates are not time.Time. XML Schema dates carry an optional timezone,
where "absent" is a distinct value from +00:00; they exceed the year range of
int64 nanoseconds; and their seconds have arbitrary fractional precision. The
xdm.DateTime type models all three, because a validator that treats an
unzoned date as UTC gets comparisons wrong at the day boundary.
Patterns are matched right to left. An XSLT match pattern looks like a
path expression but means "does this node match", not "navigate from here". The
obvious implementation — evaluate the path, test membership — is quadratic.
Matching from the candidate node upward makes it O(depth), which is the
difference between a transform that finishes on a large document and one that
does not.
Templates are pre-sorted. Selection scans the template list and stops at the first match, because the list is ordered by (import precedence, priority, declaration order) at compile time. Default priorities follow the spec's values, since getting them wrong silently selects the wrong rule — a much harder failure to debug than a crash.
The parser separates namespace nodes from attributes. Go's
encoding/xml resolves prefixes into Name.Space and then discards both the
prefix and the xmlns declarations. XSLT needs both: namespace nodes are
addressable on the namespace axis, and a literal result element must serialise
with the prefix its author wrote. So encoding/xml is used as a tokeniser only
and the tree is built here.
Instructions write to a builder, not to a string. A sequence constructor
produces a stream of nodes and atomic values: xsl:element opens a node that
later instructions add children to, an attribute added after children is an
error, and a variable with content becomes a navigable temporary tree.
Returning strings from each instruction would make all of that impossible.
That one decision is why xsl:result-document needed no architectural change:
every instruction already took a builder as a parameter, so a secondary
document is just a second builder rather than a rewrite of the output path.
Unset must not look like a real answer. Source positions are stored on the
node as the byte offset plus one, so the zero value reads as "unknown".
Nodes are built with plain struct literals in two dozen places across the
transform layer; had the raw offset been stored, every constructed node would
have claimed to start at line 1, and a construction site added later would
have inherited the bug invisibly. The same reasoning runs through the API:
gx:line-number() returns the empty sequence rather than 0, because a report
naming line 0 for every failure is worse than one naming none.
Concurrency and memory
A compiled Stylesheet is immutable and safe to share: compile once, transform
from many goroutines. The tests exercise that under -race rather than
asserting it — including a shared parsed source tree, which is the stronger
claim, since it means evaluation never writes back into the document.
That last claim was false until an audit found the counter-example, and it is
worth recording how it hid. xsl:sequence handed source nodes straight to the
output builder, which calls AppendChild on them — and AppendChild rewrites
the node's parent and tree pointers, while Finalize renumbers its document
order. So reading from the document mutated it. The visible symptom was not a
race at all:
<xsl:variable name="v"><xsl:sequence select="/r/b"/></xsl:variable>
Against <r><a/><b/><c/></r>, a later string-join(/r/*/name(), ',') returned
a,c,b. The variable is never used; evaluating it was enough to reorder the
input. xsl:copy-of was correct all along because it deep-copies. The guard now
lives in appendNode rather than at the two call sites, so a node that already
belongs to a tree is copied whatever reaches it.
The concurrency tests had covered key indexes, xsl:message and the regex cache
— everything that looked like shared state — and missed this because it did not
look like state at all.
A loaded xsd.Schema is the same kind of value: immutable once assembled, safe
to validate from many goroutines. Its one piece of lazily-built state, the
content-model cache, is a sync.Map for that reason. The tests exercise the
claim under -race rather than asserting it — validating cold schemas from
sixteen goroutines released together, so the cache is written under contention
rather than read from a warm one, and validating documents that carry the same
xs:ID values concurrently, since identity and ID/IDREF tables are the state
most likely to have been hung off the schema by mistake. Loading is covered
too, including a schema that declares over the built-in XML-namespace
attributes while others load beside it.
Per-transform state lives on a runtime struct that is copied on every focus
change. Anything that must survive those copies — xsl:message output,
xsl:result-document results — is held through a pointer, and there are tests
that fail if a transform ever sees another's.
The one piece of process-wide mutable state is the compiled-regex cache, and
it was an unbounded leak. The original reasoning was that patterns come
from stylesheets and so form a fixed set; that is wrong, because
matches($s, $node/@pattern) compiles a pattern taken from document data. A
long-running validator retained one compiled regexp per distinct pattern it had
ever seen — 17.6 MB after 20,000 patterns, still climbing. It is now bounded at
1024 entries and clears wholesale when full, which measures at 0.6 MB for the
same load and stays flat at five times it.
Clearing rather than evicting one entry at a time is deliberate: a true LRU
needs a lock on every read, which costs more than it saves when the working
set is the handful of patterns a stylesheet actually contains. Correctness does
not depend on a hit — every entry is reproducible from its key — and
TestRegexCacheStaysCorrectWhenCleared pins that.
The backtracking engine's single-character-atom cache had the same defect and
now has the same bound. It is reached only when SetBacktrackingRegex(true) is
set — off by default, and set only by cmd/go-xml's -backtracking-regex flag —
so it was an opt-in availability issue rather than a default-configuration one.
TestCharAtomCacheIsBounded counts the entries rather
than the heap, because one atom is small enough that the map size is the only
exact signal.
FileResolver's document cache exists because fn:doc is defined to return
the same node for the same URI within one execution, so doc('x') is doc('x')
requires it. It is mutex-guarded, and now capped at 256 documents. It had been
unbounded on the reasoning that the directories the caller opened with
-allow-dir bound it in practice — true for the size of the set, but the
process holds every document it has ever parsed for its whole lifetime, and a
stylesheet chooses which ones to fetch. Reaching the cap costs one reparse.
Results never pin their source. xsl:copy-of deep-copies, so holding a
Result from a 20,000-element document costs a few hundred KB rather than the
document.
Optimisation
Compilation has an optimisation stage between the parser and the runtime. Everything it does is measured below; nothing is included because it seemed likely to help.
Constant folding. A closed sub-expression is evaluated once at compile time
and replaced with a literal, so (1 + 2) * 3 - 4 idiv 2 costs one literal load
per node instead of five operations — 13.8 ns for the whole expression.
Two conditions guard every rewrite:
- The expression must be closed: no variable, no context item, no function
that reads the dynamic context.
1 + 2folds;position() + 1does not. - The rewrite must preserve errors as well as values.
1 idiv 0is deliberately not folded — refusing a stylesheet at compile time for an error in a branch that is never taken would be wrong.
Foldable functions are an allowlist, not a denylist. A user-defined function is never folded even if it happens to be pure, because the library it resolves through is supplied by the caller.
Aggregates over a range are arithmetic. An integer range is determined by
its bounds, so nothing is built to answer a question about it: count is
hi - lo + 1, sum is the arithmetic series, min/max are the bounds.
sum(1 to 10000000) returns 50000005000000 in 6 MB rather than
materialising ten million values at 1.9 GB. The series is summed in
big.Int because n(first+last) overflows int64 long before the bounds do —
sum(1 to 5000000000000) is exact here, and Saxon refuses the range outright
(its sequences cap at int32).
The recognition is deliberately narrow: only a bare to directly under the
aggregate. A predicate, a for, or a comma sequence changes which items
survive, so those evaluate normally — sum((1 to 10)[. mod 2 = 0]) is 30, not
55.
Compiled regexes are cached, bounded. Schematron applies the same handful
of patterns to every node, and regexp.Compile dominates otherwise. The cache
was originally unbounded on the reasoning that patterns come from stylesheets —
which is wrong, because matches($s, $node/@pattern) compiles a pattern from
document data. That leaked 17.6 MB per 20,000 distinct patterns and never
shrank. It is now capped at 1024 entries and clears wholesale when full, which
measures 0.6 MB for the same load.
Templates are pre-sorted, not indexed. Selection scans the template list
and stops at the first match, because the list is ordered by (import
precedence, priority, declaration order) at compile time. A per-element-name
index would make dispatch cheaper in principle — but profiling a 67-template
renderer put findTemplateFrom nowhere near the top, so it is not implemented.
See Where it is slow below.
What was tried and reverted
Context.WithFocus is the single largest allocation site in the engine —
around a quarter of everything a render allocates, because a path step runs
once per node. Reusing one focus context across a step loop looked like an
obvious win. Measured: 4,963,596 → 4,964,187 bytes per render. Nothing. Go's
escape analysis was already handling it, and the aliasing risk that reuse
introduces bought exactly zero, so it was reverted. The comment in
xpath/context.go records the numbers so nobody repeats the experiment.
Benchmarks
Apple M3 Pro, Go 1.26, -benchtime=200x, median of five runs. These are the
two production workloads, not microbenchmarks. Wall-clock figures vary about
±15% run to run on a laptop, so they are rounded; the allocation counts are
stable to three significant figures and are the more useful number.
| benchmark | time | allocated | allocs |
|---|---|---|---|
UBLRender — 100 KB stylesheet, 67 match templates → HTML |
~2.6 ms | 4.96 MB | 67,850 |
OmanValidate — 87-template Schematron → SVRL |
~1.05 ms | 1.66 MB | 21,703 |
CompileUBL — one-time stylesheet compilation |
~0.90 ms | 1.36 MB | 17,578 |
ParseInvoice — XML parse, 61 MB/s |
~0.15 ms | 0.16 MB | 2,404 |
CompileUBL is the cost Transform amortises: compile once, transform many.
Reproduce with go test ./xslt/ -bench=. -benchtime=200x (requires a
testdata/ corpus — see How this was tested).
UBLRender is a 100 KB, 67-template stylesheet rendering an invoice to HTML.
OmanValidate is an 87-template Schematron rule set producing an SVRL report.
CompileUBL is the one-time cost that Transform then amortises across every
document.
Against Saxon-HE 12.4
Validating the same document repeatedly, one process handling the whole batch:
| documents | go-xml | Saxon-HE 12.4 |
|---|---|---|
| 4 | 0.04 s | 0.81 s |
| 100 | 0.16 s | 0.81 s |
| 1000 | 1.38 s | 2.01 s |
Read this carefully rather than as a win. Saxon's cost is almost entirely fixed: 4 documents and 100 documents both take 0.81 s, because that is JVM startup plus stylesheet compilation. Its marginal cost is about 1.2 ms per document against go-xml's 1.3 ms — so on a hot loop the JIT is slightly ahead, and at a large enough batch Saxon would catch up and pass.
Where go-xml is unambiguously better is anything that pays startup often: a CLI invocation (0.26 s versus 0.67 s cold), a short-lived container, a per-request validator. Where it is not better is a long-running process transforming millions of documents. Neither engine is "fast"; they are fast at different shapes of work.
Security defaults
Every remote-reference mechanism is off unless you turn it on.
-
DOCTYPEis rejected by the parser unlessParseOptions.AllowDOCTYPEis set. It is the entry point for both XXE and entity-expansion blowup. -
fn:docandfn:documentfail closed as a library. With noDocumentResolverconfigured, every URI is refused. The CLI is not fail-closed in the same sense: it always configures a resolver rooted at the stylesheet's own directory, because a rule set that includes a sibling module is the normal case. That root is shared withdoc(), so a stylesheet can read any file beside it, not only include one — keep stylesheets in a directory of their own if that matters. Everything outside stays refused.xslt.FileResolverconfines reads to directories you name and refuses every non-filescheme; there is no network option. Confinement is enforced byos.Rootat the moment of opening rather than by resolving a path and checking it first, so a symlink swapped between the check and the open cannot escape — the older resolve-then-check design had that gap. Every rooted resolver works this way:xslt,xsd,dtdand therelaxngresolver incmd/go-xml, which the other three joined on 2026-09-10. Each read is bounded byFileResolver.MaxBytes, 64 MB by default, and a larger file is refused rather than truncated. -
xsl:includeandxsl:importfail closed the same way, viaCompileOptions.Resolver. -
XInclude is off unless a caller runs
xdm.ProcessXIncludeexplicitly. It is a document-level pass rather than a parsing option — XInclude 1.0 §4 defines it as a transformation from one infoset to another — and it reads only what anxdm.IncludeResolverhands it.xslt.FileResolverimplements that through the sameresolvePaththat gatesfn:doc,xsl:includeand external entities, so an inclusion is confined to the named roots on exactly the same terms: no non-filescheme, confinement enforced at the open byos.Root, the same per-file byte limit, nothing outside the roots. This matters more here than elsewhere, because with XInclude it is the source document — the party the threat model already treats as hostile — that names what to read. The CLI exposes it as-xinclude. -
xs:importandxs:includefail closed, viaxsd.Options.Resolver. The default reads a schema beside the one it was given and refuses a remote URL outright;HTTPResolveris how you opt in to the network.go-xml validateis confined on the same terms for-xsdand-rng: to-rootwhen given, and otherwise to the schema's own directory, as the transform is to its stylesheet's. Since schemas name their imports as absolute URLs — the XSLT 3.0 schema imports the XSD 1.1 schema for schemas fromw3.org— the usual answer is not to fetch them but to answer from a catalog:r := xsd.NewCatalogResolver() err := r.AddFromFS(os.DirFS("schemas"), xsd.W3CEntries())CatalogResolvermatches a document by namespace and by everyschemaLocationspelling it is referred to by, so one entry answers theTR/URL, the2001/URL, a bare relative path and a location-lessxs:importalike. A reference to something not in the table is an error rather than a request. The companion modulew3cschemasships the W3C documents themselves — separate because they are under W3C rather than MIT terms. -
Nesting and recursion are bounded — parse depth, XPath recursion and template recursion each have a limit that produces an error rather than a stack overflow.
-
Memory is bounded too. Depth bounds the stack and
-timeoutbounds the clock, but neither bounds allocation:sum(1 to 9999999)is one shallow, fast expression that materialised nine million values and peaked at 1.8 GB of resident memory. A per-evaluation item budget now refuses it. The budget resets for each expression, so a stylesheet evaluating a legitimate range once per node of a large document is unaffected — that distinction is the whole point, and it is tested. -
Cancellation works.
Transformtakes acontext.Contextand checks it at every loop boundary, so a pathological stylesheet is interruptible. -
xsl:result-documentnever writes to disk. Secondary documents are returned to the caller onResult.Secondary; a transform that can create files anywhere the process can write is a decision the caller should make. The CLI opts in with-result-dirand refuses anyhrefresolving outside it, symlinks included. Anxsl:result-documentwith nohrefnames no file at all: section 24.3 leaves the current output URI at the base output URI, so it is written to the principal output, and needs no flag. The CLI sets that base output URI from where the output actually goes — the-ofile, the-result-dirdirectory, or the working directory, a directory being spelled with a trailing slash — sofn:current-output-uri()reports the destination. The library still defaults to none, since it never writes files; section 19.1 makes the choice implementation-defined and permits it to be absent.
The CLI mirrors these: -allow-dir opens document access, -allow-doctype
opens the parser, and -timeout bounds the transform. All three are off or
conservative by default.
Treat a stylesheet as code, not as data. It can read any file inside the
permitted roots, write anywhere under -result-dir, and spend the whole
-timeout doing it. The boundaries are enforced and tested — traversal,
symlinks into and out of the roots, absolute paths, and non-file schemes are
all refused, and TestResolverContainmentAttacks probes them the way an
attacker would — but inside those boundaries a stylesheet is a program you are
choosing to run.
One consequence worth stating plainly: engine-internal state is bound to
variables in a private namespace, and a stylesheet that names that namespace
can reach those values. Doing so used to panic the process, which for an
embedding server is a denial of service written in stylesheet text; every such
path now answers or errors instead. TestInternalStateIsNotReachableAsAPanic
covers twenty-two of them.
What is implemented
Coverage was established by auditing against the spec's own inventories rather
than by recollection, across all three layers: the 7 XDM node kinds and the 22
instantiable XML Schema primitive types, all 58 XPath 2.0 grammar productions,
all 49 XSLT 2.0 elements with their behavioural attributes, and the 113
required fn: functions — checked at all 153 of their name/arity signatures,
since a call resolves against both and a function can be present at one arity
and missing at another.
XDM. All seven node kinds; the numeric tower with exact xs:decimal; the
date, time and duration types; and the five Gregorian types (gYear,
gYearMonth, gMonth, gMonthDay, gDay), which support equality but not
ordering — without a year, "is --01-15 before --02-01" has no answer that
holds for every year. xs:NOTATION is the one primitive not present: it cannot
be instantiated directly and exists only as a DTD attribute type.
XPath 2.0. All thirteen axes; every node and kind test; the full precedence
ladder; value, general and node comparisons; for, if, some/every;
instance of, cast, castable, treat; sequence operators; and the
complete required function library — strings, sequences, numerics, regular
expressions, node properties, QName accessors, URI handling, dates, durations,
format-dateTime, deep-equal, plus the xs: constructors.
Functions are registered by name and arity, since the two are what a call
resolves against. That distinction matters for the ten functions carrying an
optional trailing collation argument — compare, contains, starts-with,
ends-with, substring-before, substring-after, index-of,
distinct-values, min and max. All accept it; the codepoint collation is
honoured and any other is refused rather than silently applied as ASCII order.
xs:QName is resolved during parsing rather than at run time, because a QName
value carries the namespace URI and the prefix binding exists only in the
static context — which is also why the spec restricts its argument to a string
literal.
XSLT 2.0. Every element: apply-templates
(with modes and the built-in rules), apply-imports, next-match,
call-template, for-each, for-each-group (all four grouping modes), if,
choose, variable, param with tunnel parameters, element, attribute,
attribute-set, namespace, namespace-alias, comment,
processing-instruction, copy, copy-of, sequence, value-of, text,
sort, perform-sort, message, analyze-string, key, function,
include, import, strip-space, decimal-format, character-map,
number (all three levels — single, multiple and any — with count and
from), output (the xml, html and text methods, named as well as
unnamed), result-document, as type declarations, attribute value templates,
and the simplified literal-result-element stylesheet form.
XQuery 3.1. Everything the language adds on top of XPath, since the
expression half is xpath's and already at 100%: direct and computed
constructors for all seven node kinds; every FLWOR clause — for, let,
where, group by, order by, count, and both the tumbling and sliding
window clauses; the prolog, with namespace, variable, function, option and
decimal-format declarations, boundary-space, construction, ordering,
empty-order, copy-namespaces and the declared context item; try/catch;
switch; typeswitch; quantified expressions; ordered/unordered; the
extension expression; and the string constructor.
The version declaration is recorded rather than discarded. A module may open
with xquery version "1.0", "3.0" or "3.1"; anything else is XQST0031,
and a module that declares nothing is compiled as 3.1. The engine implements
3.1 throughout, but the three rules the versions are known to disagree on now
ask which version applies: an unprefixed declare option name is XPST0081
in 1.0 and legal from 3.0; a cast target naming a type not in scope is
XPST0051 in 1.0 and XQST0052 from 3.0; and a variable circularity running
through a function body is the static XQST0054 in 1.0 and the dynamic
XQDY0054 from 3.0. See docs/xquery.md for what stays at
3.1's reading whatever the module declares.
Two declarations parse and are then refused rather than mis-parsed, because
both need a module store this package does not have: import module raises
XQST0059, and import schema leaves the in-scope schema definitions empty
so validate { … } raises XQDY0084. See docs/xquery.md.
Collations. Two are implemented: codepoint, and the ASCII
case-insensitive collation the spec defines, which needs no locale data. Both
are applied rather than merely validated — an xsl:sort/@collation that was
accepted and then sorted by codepoint anyway would be the silent-wrong-answer
this engine exists to avoid. Relative URIs such as collation/codepoint
resolve, because stylesheets write them that way. Anything else is refused.
Sorting and Unicode. xsl:sort supports @case-order and @lang:
Swedish orders "ä" after "z" where German orders it next to "a", and codepoint
order gets both wrong. fn:normalize-unicode implements all four standard
forms. Both use golang.org/x/text. A @lang naming a language with no
collation data is refused rather than quietly falling back to codepoint order,
and @collation accepts only the codepoint URI — a language-sensitive
collation is spelled with @lang.
Real-world stylesheets
The W3C suites test the language a rule at a time. They do not test what a large stylesheet does with it, and two codebases that are widely used as the practical bar for an XSLT 3.0 processor found four defects the suites did not reach:
-
DocBook xslTNG — 97 stylesheet modules using
xsl:evaluate, accumulators, maps, higher-order functions and a multi-stagefn:transformpipeline. 577 of its 593 test documents render (549 before XInclude), and the HTML is byte-identical to the Saxon-produced reference output once the timestamp and generator metadata (both environment-dependent) are normalised.The 42 that once needed
ext:xincludewere the interesting case. DocBook's pipeline calls that Saxon-Java extension function on the source document, and itsfunction-available()fallback quietly passes the document through unchanged — which leaves thexi:includeelements in the tree, where the main transform has no template for them and raisesXTMM9000. Native XInclude at parse time is what those documents actually need, not the extension function: the pipeline stage is guarded bytest="exists(//xi:include)", so once the inclusions are already resolved the stage does not run at all and the extension is never asked for.-xincludetherefore fixes 28 of the 42 outright, andtests/check.shpasses it for that reason: measured without it the corpus scores 549, against inputs no reader of DocBook would use. The remaining 14 fail onxpointerschemes that are not part of XInclude: Saxon'sxpath()scheme, and RFC 5147 text fragments (line=,char=,search=) layered onparse="text". Those are DocBook conventions, and implementing them to move a number is not the same as conforming to a specification, so they are deliberately left out. Three more raiseXTDE0420: DocBook builds a temporary tree from a sequence containing an attribute, which §5.8.1 makes an error, and this engine is right to refuse it — the ordered rules there unwrap document nodes but never attributes, and the suite's ownerror-0420asays "the xsl:copy is copying a document node which can't have an attribute". Saxon accepts it anyway. Its 75 localisation files are generated with this engine too. -
XSpec — an XSLT compiler written in XSLT. All 225 of its
.xspectest descriptions that target a stylesheet compile; the other 59 declare no@stylesheetbecause they drive the Schematron and XQuery compilers instead.
What they found, none of which the suites covered:
xsl:copy over a non-node context item |
XTTE0945 is raised only when the context item is absent; one that is present but atomic returns the value. Conflating the two made xsl:copy inside xsl:for-each over atomics an error |
fn:key with a prefix bound per-module |
The key name is a lexical QName resolved at run time. Keeping one binding per prefix let the last module included decide what every such name expanded to — XSpec binds local to 19 different URIs |
xsl:evaluate calling the stylesheet's own functions |
§10.4.1 excludes private functions, and the default is private — but visibility is a property of a component of an xsl:package, and a plain xsl:stylesheet is not one. See below |
| A base URI that is a filesystem path | fn:resolve-uri and fn:static-base-uri are defined over RFC 3986 references, so a bare path made resolve-uri(rel, static-base-uri()) raise FORG0002. The CLI now spells it as a file: URI — file:///home/u/s.xsl on Unix and file:///C:/dir/s.xsl on Windows, the RFC 8089 empty-authority form. Two slashes would make the drive letter an authority, and the path would come back without it |
One deliberate divergence. Confining the private-function default to a real
xsl:package costs W3C evaluate-045, which asserts the strict reading — one
case the engine gives up on purpose, and the reason 11,348 is not 11,349. Saxon
does not enforce it either: its own XSLT 3.0 results report evaluate-045 as
wrongError. Inside an xsl:package, declared visibility is honoured exactly
as before. The alternative was that no stylesheet outside a package can call
its own functions from its own xsl:evaluate, which is not a boundary its
author drew.
Where it fails
Three separate things are worth distinguishing, because they fail for different reasons.
1. Constructs that are refused outright
Each errors rather than doing something plausible. An XSLT processor that
accepts an instruction and quietly ignores it is the worst failure mode,
because the output looks fine and is wrong. See What is not below for the
full list — fn:unparsed-text, a backreference to a variable-width group, and
fn:collection or fn:doc when the caller has configured no resolver.
Malformed stylesheets are refused too, which is the same principle applied
one level up. An unknown element in the xsl: namespace is XTSE0010 rather
than being skipped, so xsl:tempalte is a compile error instead of a silently
dropped template. So is an element in the wrong parent: xsl:when outside
xsl:choose, xsl:sort outside a sortable instruction, xsl:with-param
outside a call. Both used to be accepted and dropped, which meant a typo
produced an empty result and no diagnostic — found while writing
docs/server.md, by trying to demonstrate that a bad
stylesheet fails to compile and discovering it did not.
2. Where the QT3 suite still disagrees
It does not: 15,222 of 15,222 in-scope cases pass, and so do 19,307 on 3.0 and 21,863 on 3.1.
The last case to fall was fn-matches-51:
fn:matches("ab()cd()ef()gh", "^(ab)([()]*)(cd)([)(]*)ef\4gh$"). It names
([)(]*), a group whose width can vary, and places the backreference in the
middle of the pattern. Both are refused by RE2 — see The hard floor below —
so it passes only under xpath.SetBacktrackingRegex(true), which the
conformance harnesses enable and which stays off in production.
Two things about how that number was reached are worth more than the number itself.
The denominator grew by 461. Source paths in a test-set environment are
relative to the test-set file, not to the suite root; resolving them against
the root silently skipped every case whose environment named ../docs/…. They
were never counted as failures — they were never counted at all.
Five of the last seventeen failures were the harness, not the engine.
Assertions required a literal boolean where the spec asks for an effective
boolean value; the result serialiser dropped in-scope namespace declarations;
an assert-xml whose expected value lived in a file compared against the empty
string. A conformance number is only as honest as the harness producing it,
which is why those are written down rather than quietly absorbed.
2a. Where the XSD suite still disagrees
Two figures, and the second is the one that matters.
| schema-validity | instance | |
|---|---|---|
| XSD 1.0 | 14,385 / 14,388 (99.98%) | 24,973 / 25,000 (99.89%) |
| XSD 1.1 | 15,350 / 15,354 (99.97%) | 26,217 / 26,222 (99.98%) |
Earlier revisions of this file reported a single "99.56%" for XSD 1.0 and "XSD 1.1: 100%". Both were measured wrongly, and the correction is large enough to state outright. (That old figure is not the 99.56% in the table above, which was that revision's schema-validity number; the old one conflated the two scores and counted a whole category as skipped.)
The schema-validity tests were never scored. A test group whose schema the suite marks invalid by design was counted as a "skip" on the grounds that checking it is Schema Component Constraint territory rather than instance validation. But that is exactly what those groups test: the schema is meant to be rejected, and loading it without complaint is a failure. Scoring them turned roughly 2,200 silent passes in 1.0 into what they always were. Most have since been closed — the Part 2 facet constraints, the schema-for-schemas shape check, the XSD regex grammar, and Particle Valid (Restriction) were the four largest — but the figure is quoted here because the earlier one was not measuring them at all.
The 1.1 run scored about six per cent of its tests. Per common/xsts.xsd the
version attribute is a list of tokens, not a single string — OR-joined on
testSet/testGroup/schemaTest/instanceTest, AND-joined on expected,
and absent means the test applies to every processor. The driver compared
it for equality with "1.1", so the 1.1 run saw only the explicitly-marked 1.1
groups: 888 schema tests rather than the 15,365 that apply. The old "100%" was
a real result over an unrepresentative sixteenth of the suite.
The instance figures are now close to each other and to where 1.0 stood before, which is the expected shape: most groups carry no version attribute and so are scored identically by both runs.
The remaining gap is still dominated by schema false-accepts — invalid schemas loaded without complaint, i.e. Schema Component Constraints not yet checked — now concentrated in attribute declarations, wildcards, element declarations and identity constraints. False rejects, where a valid schema or instance is refused, number in the low tens and are tracked separately because they are the more serious kind: refusing valid input breaks a caller, while accepting invalid input only fails to catch their mistake.
One note on the denominator, and one on the numerator. The XmlVersions schemas carry
version="1.1" and this parser accepts and loads all of them, but reads them
under XML 1.0 rules, so what they test is not what is measured. See
todo.md.
And some of the suite is disputed. status="queried" on a test means the W3C
has challenged the expected result, usually with a bugzilla reference, so those
disagreements are a ceiling rather than work outstanding. Twenty-seven of the
1.0 disagreements are marked that way:
| bug | cases | what |
|---|---|---|
| 4113 | 19 | regex \p{...} general-category tests |
| 6901 | 2 | gMonth002_2061, gMonth004_2063 |
| 4952 | 1 | particlesW006 |
| 4680 | 1 | elemZ027_c |
| 4126 | 1 | anyURI_a004_1339 |
One test set is mislabelled rather than one test. ibmMeta/wildcard.testSet
carries version="1.0", which excludes it from the 1.1 run — but every one of
its seventeen groups cites the XSD 1.1 specification in its own
documentationReference, and four use notQName, a 1.1-only wildcard form,
while expecting the schema to be valid. Refusing notQName under 1.0 is
correct, and those four schemas do load under Version11, so the four
disagreements are the label's fault rather than the validator's.
Bug 4113 is the instructive one. The suite was written against Unicode 3.1,
and characters have moved between general categories since. reJ11 asserts that
\p{Lu}* rejects U+1D7A8, which is an uppercase letter in modern Unicode.
Passing that test would mean shipping a frozen Unicode 3.1 table and being wrong
about every text written in the twenty years since — so these are left
disagreeing on purpose.
Is 100% reachable?
For XPath, effectively yes: one case remains, and it is refused on purpose.
The route there is the useful part, because most of it was not what the
failures looked like. Of the seventeen that remained after the ordinary bugs,
five were the test harness rather than the engine, two needed DTD attribute
defaulting, two needed a document to be retrievable under the URI
fn:document-uri reports for it, one was a lexical form that disagreed with
its own value, and eleven were backreferences that turned out to be decidable.
Two of those are worth stating on their own.
for $var in "ABC" return $var castable as xs:QName is false while the same
expression with a literal is true, because a QName's namespace comes from the
static context and only a statically-known operand has one. That is a static
property of the operand, so it has to be decided where the expression is
visible — not in the cast, which sees a value and cannot tell a literal from a
variable holding one.
xs:decimal kept more precision than it printed. A literal with 360
fractional digits rendered as 0 while comparing unequal to zero, so
0.000…1 eq 0 was false and string(0.000…1) was "0". Both cannot be right;
the value is the one that must not move.
Three themes recur through everything that was fixed, because each is the same mistake made repeatedly:
A Go standard-library parser is not a schema validator. strconv.Atoi
accepts a leading sign, so every fixed-width date and time field did too and
"11:+1:11" parsed as 11:01:11. big.Rat.SetString reads Go's numeric
syntax, so xs:unsignedLong("0x0") was zero. url.Parse treats "b.html" as
a URI with an empty scheme, so it served as a base for fn:resolve-uri.
strings.Fields splits on every Unicode space, so xs:token() swallowed a
non-breaking space. strings.ToUpper applies simple case mapping, so
upper-case("ß") was not "SS". Each turned a malformed or misread input into
a plausible value — worse than an error, because nothing downstream can tell.
An out-of-range value is not a large value. "P768614336404564651Y" parsed
as a negative duration, because the year count multiplied by twelve wrapped
an int64. Three separate panics — a parser that skipped a fixed token count
past the end of its input, a NaN produced by adding opposite infinities that
survived every clamp because comparisons against NaN are false, and an infinite
divisor that looked like zero because an infinity has no rational form — were
all the same failure to bound an input. The converse bit just as often: a
400-digit xs:integer literal is an ordinary arbitrary-precision value, a
range from 10²¹ to 10²¹+3 is four items, and a repeat count of 2147483647 is a
pattern that matches nothing rather than a malformed one. Refusing any of them
because a machine word could not hold the number was equally wrong.
A later audit found five more of the same shape — a big.Int or big.Rat
narrowed with .Int64() or int(...) at a site whose sibling path already
range-checked. xs:yearMonthDuration("P768614336404564650Y") * 4 came back
negative; fn:remove((1,2,3), 2^64+2) deleted the second item because the
position truncated to 2; and a range spanning the whole of int64 wrapped its
count to exactly zero, so avg() divided by it and panicked. None of these is
reachable from the conformance suite, which is why they survived it.
A parameter declaration is a type, not a suggestion. fn:string-join(1 to 5, "") returned "12345" because the implementation called String() on each
item rather than applying the conversion rule for the xs:string* it declares.
The same slip appeared in fn:codepoints-to-string, fn:translate,
fn:normalize-unicode, fn:remove, fn:doc-available, fn:error and every
collation argument — an empty sequence passed to a parameter declared
xs:string is XPTY0004, not a default.
An earlier version of this table claimed ~210 of the failures needed a static typing pass. That was wrong, and worth recording: when the cases were actually read rather than inferred from their error codes, all but one turned out to be ordinary missing validation.
The one remaining case is regex, and the reasoning is worth setting out because the obvious fix is wrong.
RE2 has no backreferences by design — the trade that buys linear-time matching
and the reason a pathological pattern cannot hang this engine. The natural
workaround is to capture the groups and compare the text explicitly. That fails
silently: RE2 returns a single submatch assignment, the greedy one, so
(a*)\1 against "aa" reports the group as "aa", leaves nothing for the
backreference, and answers false where the truth is true — the split is
"a" + "a". The information needed was discarded before the comparison ran.
But when every group a backreference names has a fixed width, the greedy
assignment is the only assignment. There is nothing to enumerate, so
comparison is exact rather than approximate, and it runs in RE2's linear time:
measured on ([a-z])\1*, 4,000 characters in 53 µs and 64,000 in 567 µs.
So the split is by what can be decided, not by what a caller asked for.
(a)\1 and ([md])[aeiou]\1 are resolved; (a*)\1 raises FORX0002. There
is no option to switch this part off — an engine that answers correctly or
says it cannot is safe to have on always, where one that guesses is not safe at
any setting. The default keeps RE2's linear-time guarantee intact.
fn-matches-51 is the case that needs more: it names a variable-width group
and puts the backreference mid-pattern, which needs the comparison to feed
back into the automaton. A backtracking matcher that does exactly that is
available behind xpath.SetBacktrackingRegex(true) (-backtracking-regex on
the command line). It is off by default because it has no linear-time
guarantee and patterns can come from document data; even enabled, a step
budget bounds every match and exhausting it is an error rather than a silent
"no match".
Everything else about the XML Schema regex flavour is implemented, and most
of it had to be, because RE2 silently disagrees rather than refusing: XML
Schema's escape set is closed (RE2 reads \0 as a NUL byte), . excludes both
newline characters where RE2's excludes only \n, the i flag must not reach
inside \p{Lu}, the x flag strips whitespace before escapes are read, and
\p{IsBasicLatin} names a block RE2 has never heard of.
3. Where it is slow
Profiling the UBL renderer: the whole transform is ~12% of samples and GC is
~40%. The engine is allocation-bound, not algorithm-bound. The largest
single site is Context.WithFocus, at roughly a quarter of all allocation,
because a path step allocates a focus context per node.
The unimplemented items that would actually move this are structural, not incremental:
- Lazy sequence types.
Sequenceis a flat[]Itemthat three packages index and range over directly (~100 call sites). Making it an interface withArraySeq/IntegerRange/FilteredSeqimplementations is the textbook answer, and it is why1 to 10000000needs the arithmetic special-case above rather than being free. - Receiver-based output. Instructions write into an
outputBuilderthat materialises a tree (46 call sites). An event receiver —StartElement,Attribute,Text,EndElement— would let serialisation stream instead of building the whole result first. - Streaming. Depends on the receiver work. Some expressions genuinely
cannot stream (
last(),preceding::, backward navigation), so this is a per-expression capability, not a mode.
None of these is reachable by incremental change, and the profile says the payoff is smaller than intuition suggests — which is why they are documented here rather than half-done in the code.
What is not
Three things are refused until configured or refused outright, none of them an XSLT element. Each one errors rather than doing something plausible — an XSLT processor that accepts an instruction and quietly ignores it is the worst possible failure mode, because the output looks fine and is wrong.
fn:unparsed-textwithoutFileResolver.UnparsedText— reading files named by a stylesheet is off by default and enabled by a switch of its own, implied by nothing else. It is separate because it is the widest of these:fn:dochands back a parsed XML document, so a file that is not well-formed XML discloses nothing, whileunparsed-texthands back the raw bytes of any file insideRoots— a root holding one XML data file and one private key leaks the key.fn:collectionandfn:docwithout a resolver — both fail closed rather than returning nothing, so a misconfiguration is reported instead of looking like a document set that happened to be empty. Supply aCollectionResolverorDocumentResolverto enable them; they are separate switches, so enablingfn:docfor a known code list does not also let a stylesheet enumerate a directory.- A backreference to a variable-width group (
(a*)\1) — RE2 returns one submatch assignment, and for a group whose width can vary that assignment may be the wrong split, so the answer isFORX0002rather than a guess. A fixed-width backreference ((a)\1) is resolved, exactly and in linear time. Character-class subtraction ([a-z-[aeiou]]) is implemented, by expanding both sides into codepoint ranges and taking the difference; only subtraction from a shorthand class ([\i-[:]]) is refused, because that needs the Unicode tables defining the shorthand. A hyphen between two classes ([a-z]-[a-z], as in a UUID pattern) is not subtraction and is unaffected.
One approximation is documented rather than hidden: fn:id/fn:idref use
xml:id and conventional id attributes, because without a DTD or schema
nothing declares which attributes are of type ID. A stylesheet relying on
DTD-declared IDs gets an empty result rather than a wrong one.
How this was tested
Seven methods, each catching a class the others miss. That is the point: no single one of them was sufficient, and each was added because the previous set had let something through. How to run any of them, and how to read what comes back, is in docs/testing.md.
| method | what it catches | what it misses |
|---|---|---|
Unit tests (2,416 func Test declarations) |
places where a plausible implementation is quietly wrong | anything nobody thought to write a test for |
| Spec inventories | features absent entirely | features present but behaving wrongly |
| Saxon differential | subtle behavioural divergence on real stylesheets | constructs the corpora do not use |
| W3C QT3 suite | systematic conformance across 15,183 cases | XSLT (it is an XPath suite) |
| W3C xsdtests suite | systematic XSD conformance across 25,000 instance and 14,388 schema-validity tests (XSD 1.0; 1.1 adds 26,222 and 15,354) | schemas nobody writes by hand |
| Production schema sets | what large modular schemas do that suites do not | anything those industries happen not to use |
| Fuzzing (11 targets) | a crash, hang or wrong refusal on input no author would write | anything a coverage-guided search does not reach in the time it is given |
| Every suite feeds the parser well-formed input, which is the gap fuzzing | ||
| exists to close: the targets cover the XML parser, the schema assembler and its | ||
| content-model compiler, the stylesheet compiler, and a parse → serialise → | ||
| parse round trip, and they assert that a refusal arrives as an error rather | ||
| than as a panic. |
The production schema sets found the most per hour. Pointing the validator at UBL 2.1 turned up two bugs the entire W3C
suite had not: deduplication keyed on the schemaLocation as written rather
than the resolved path, so a diamond in the import graph read one file twice
and reported every global in it as a duplicate; and attribute inheritance
scheduled on a fixed number of passes, which no chain depth is guaranteed to
fit. Between them they meant all 65 UBL main-document schemas failed to load,
with 1,758 errors apiece. Neither is exotic — a diamond is the normal shape of
a modular schema set, and UBL's EndpointIDType is an empty
<xs:extension base="udt:IdentifierType"/>.
| schema set | schemas | result |
|---|---|---|
| UBL 2.1 (OASIS) | 65 | all load; 8 real invoice and credit-note instances validate clean |
| UN/CEFACT CII D16B (EN 16931) | 2 | both load; 15 instances validate clean |
| Factur-X / ZUGFeRD (all profiles) | 21 | all load; samples validate clean against their own profile |
| Peppol BIS Billing 3.0 | — | 24 UBL instances validate clean |
The three Mustang fixtures this rejects are genuinely invalid — two carry a
second URIID where the schema allows one, and the third is named
not_validating_… in its own corpus.
Unit tests concentrate on where being plausible is not enough rather than
on breadth: exact decimal arithmetic, the canonical form of doubles at the
exponent boundary, //title[1] versus (//title)[1], untypedAtomic comparing
as strings unless cast, reverse-axis position numbering, = not being the
negation of != over sequences, template priority values, whitespace stripping
under xml:space, and the security defaults.
Spec inventories are machine-checked, not recalled. Two of the 296 are
coverage guards rather than behaviour tests: one parses every XPath 2.0 grammar
production, the other looks up every required function by name and arity in
the library Builtins() actually returns.
That second guard exists because an earlier audit was done by grepping the
source for register("...") literals and reported 31 functions missing that
were all present — they are registered through helper wrappers whose names the
grep never saw. A grep-based inventory can be wrong in both directions;
asking the library is the only form of the check that cannot flatter the
result. Checking by arity as well as by name is what later exposed ten
collation overloads that did not resolve at all.
Verified against Saxon on two independent real-world corpora, with golden files generated by Saxon-HE 12.4, the reference XSLT 2.0 implementation.
A production rendering stylesheet. A 100KB UBL Invoice/CreditNote HTML renderer (67 match templates) applied to the OpenPEPPOL BIS 3.0 examples — credit notes and negative corrections included — producing output byte-identical to Saxon's.
A production validation rule set. The Oman PINT e-invoicing Schematron rules, including the 550KB jurisdiction-aligned set, applied to the official example documents. The SVRL reports exactly the assertions Saxon reports, at exactly the same locations. This is the closer test of the two: a validator that disagrees with the reference implementation rejects valid invoices.
These corpora are not in this repository. They are third-party production stylesheets and rule sets, so they are not redistributed;
testdata/is git-ignored. A fresh clone runs 293 tests and skips 4, cleanly and silently — the differential tests detect the absent directory rather than failing.This matters when reading the numbers below: the conformance figure is reproducible from a clone, the Saxon comparison is not. If you are putting this in front of a rule set that matters, run the differential yourself against your own corpus. docs/recipes.md shows the shape of the harness.
Reporting the line a failure is on
SVRL identifies a failing element by XPath, which is exact but not something a person navigates by — and it is ambiguous in the case that matters most, where two siblings share a path and only one of them failed.
Parsing with TrackPositions records where each element starts, and a
stylesheet reads it through two extension functions in the namespace
https://github.com/knroy/go-xml:
<xsl:if test="gx:line-number()">
<xsl:attribute name="line"><xsl:value-of select="gx:line-number()"/></xsl:attribute>
<xsl:attribute name="column"><xsl:value-of select="gx:column-number()"/></xsl:attribute>
</xsl:if>
$ go-xml -xsl rules.xsl -track-positions invoice.xml
<svrl:failed-assert id="PRICE-01" location="/order/group/item" line="4" column="5">
<svrl:failed-assert id="PRICE-01" location="/order/group/item" line="5" column="5">
Both failures carry the same location; the line is what separates them.
They return the empty sequence when the position is unknown — the document
was parsed without TrackPositions, or the node was built by the transform
rather than read from a file. That is why the example tests before emitting:
a report claiming line 0, or line 1, for every failure would be worse than one
carrying no line at all. Tracking costs about 10% more memory and no extra
parse time, and is opt-in because it buys nothing for a caller that never asks.
Between them these found seven real bugs, all since fixed:
fn:current()returned the predicate's context item rather than the node the enclosing instruction was processing. Schematron's location-path generator counts preceding siblings with[local-name() = local-name(current())], so the test was trivially true and every sibling counted — a document with twocac:TaxTotalelements reportedTaxTotal[18], and the mis-numbering produced a false failed-assert.xsl:number level="multiple"was unimplemented, so the rule set would not compile at all.- A path rooted at a variable (
$codelists/cl[@id=$x]) demanded a context item. It is legal insidexsl:function, where there deliberately is none. astype declarations were parsed and then ignored, so a parameter declaredas="xs:decimal?"received an untypedAtomic and its arithmetic silently became floating point.fn:format-numberwas missing entirely, along withxsl:decimal-format.U+00A0was stripped as whitespace. Go'sstrings.TrimSpaceusesunicode.IsSpace, which matches it; XML whitespace is only space, tab, CR and LF. A deliberate was silently disappearing.xsl:messageoutput was collected into a slice held by value in a struct that gets copied on every focus change, so messages emitted inside a template never reached the caller.
Successive audits against the spec's own inventories found what the corpora could not, because a corpus only exercises the constructs its stylesheets happen to use:
| Audit | Found |
|---|---|
| XSLT elements | 9 missing, including xsl:attribute-set and xsl:namespace-alias, which were being silently ignored |
| Function library | 24 missing |
| XDM types | the 5 Gregorian types absent |
| XSLT attributes | xsl:sort/@case-order stored and never applied; @lang never read |
| Function arities | 10 collation overloads unregistered, so compare($a,$b,$c) did not resolve; xs:QName absent |
| Deliberate refusals | xsl:number level="any", xsl:result-document, fn:normalize-unicode and @lang collation had been declared unsupportable rather than being hard |
Every one of those is now implemented. The last row is the one worth
dwelling on: four features had been written off in comments I had written
myself, and re-examining them found that three needed no architectural change
at all — xsl:result-document worked out to be a fresh output builder, because
every instruction already took one as a parameter.
The corpora are not in this repository. They are third-party material —
production stylesheets and jurisdiction-specific rule sets — so they are not
redistributed here. The tests that use them skip cleanly when testdata/ is
absent, which is why a fresh clone builds and runs its full suite with nothing
extra.
To run the differential tests against your own corpus, put the stylesheet, the
source documents, and Saxon's output for each in a top-level testdata/
directory named as <name>.xml and <name>.saxon.html. That layout is what
xslt/conformance_test.go expects; the Oman
Schematron comparison in xslt/oman_test.go reads
testdata/oman/ the same way.
Running the differential cases without the suite. Every expectation the QT3
run produced is also checked in as a plain Go table in
xpath/saxon_diff_test.go, with the reason each row
exists and Saxon's actual output as the expected value:
go test ./xpath/ -run TestSaxon -v
That needs no download and no environment variable. The file's header comment carries the four-line recipe for reproducing any single row by hand against both engines.
The W3C QT3 suite
The official W3C test suite for XPath — QT3, also called FOTS — runs against
this engine. It is not vendored: it belongs to the W3C, and testdata/ is
gitignored for that reason. Clone it and point GOXSLT_QT3 at the checkout;
without the variable those tests skip, so the ordinary go test ./... is
unaffected.
$ git clone --depth 1 https://github.com/w3c/qt3tests.git testdata/qt3tests
$ GOXSLT_QT3=$PWD/testdata/qt3tests go test ./tests/qt3/ -v -timeout 1800s
QT3: 31821 cases, 15222 in scope, 16599 skipped
in-scope: 15221 passed, 1 failed (99.99%)
The W3C XSLT suite
There is no maintained XSLT 2.0 suite. The original XSLTS froze at 1.1.0 in 2007, was distributed from w3.org behind a click-through licence rather than a repository, and has no GitHub home. What replaced it is the XSLT 3.0 suite, which carried most of those tests forward and records a version dependency on each — so an XSLT 2.0 run is a filtered run of the 3.0 suite.
$ git clone --depth 1 https://github.com/w3c/xslt30-test.git testdata/xslt30-test
$ GOXSLT_XSLTS=$PWD/testdata/xslt30-test go test ./tests/xslts/ -v -timeout 1800s
XSLT suite: 14601 cases, 6201 in scope, 8400 skipped
in-scope: 6193 passed, 8 failed (99.87%)
TestXSLT30Suite measures the same catalog at the 3.0 target; both run on every
change, because the question a change has to answer is not "how much 3.0 works"
but "how much 3.0 works without costing 2.0".
The filter decides what the number means, so each run prints its own exclusions: at the 2.0 target 6,027 cases need XSLT 3.0, 1,580 depend on a Unicode version and 421 need packages; at the 3.0 target 2,646 need streaming, 1,590 depend on a Unicode version and 1,098 are XSLT 2.0 only. A dependency the runner does not model excludes the test rather than being ignored — running a test under conditions it did not ask for reports the mismatch as a failure of the engine.
GOXSLT_XSLTS_VERBOSE=1 lists every failure rather than counting them.
Two environment variables make a failure workable:
GOXSLT_QT3_VERBOSE=1 # list every failure, with the expression it ran
GOXSLT_QT3_SET=fn-collection # run only test sets whose name contains this
GOXSLT_QT3_SET narrows a 31,821-case run to the handful you are working on;
the percentage is then labelled as filtered rather than quoted as the suite
result. Verbose output carries the expression, so a failure is reproducible
without opening the catalog:
FAIL fn-collection/collection-006: none of {all-of,error} held
expr: collection("collection1")
The figures above were measured against
w3c/qt3tests at commit 201a6e46
(2026-05-14), ~78 MB checked out. The suite is updated from time to time, so a
later checkout can move the denominator — see
docs/known-gaps.md for how a path-resolution fix moved it
by 461 in one step.
Skips are reported separately and are never counted as passes. The suite is FOTS 3.1 and covers XQuery as well as XPath 3.0/3.1. Each is measured on its own denominator, because a case that needs a language the target does not claim says nothing about that target's conformance: the XPath targets exclude the XQuery-only cases, and the XQuery target runs 29964 of the suite's 31,821 with 1,903 skipped. Counting an out-of-scope case as a pass is how a conformance number becomes meaningless.
It found four real bugs on the first run, none of which the two production corpora had exercised:
- Two crashes.
insert-before($seq, (), $x)andremove($seq, ())dereferenced the nil returned for an empty position argument and took the process down. Both now raiseXPTY0004, which is what Saxon raises. - A hang.
round-half-to-even(3.567812, 4294967296)used the precision argument directly as an exponent of ten, sobig.Intbegan computing a number with four billion digits. It consumed twenty minutes and all available memory before the run could report anything. Precision is now clamped, which is safe because rounding to more places than a value has is the identity. 24:00:00was not normalised. XML Schema admits it as midnight ending a day but defines it to mean00:00:00on the next one, so1999-12-31T24:00:00has to become2000-01-01T00:00:00. It was being kept as hour 24, which serialised differently from an equal value written the ordinary way.xs:hexBinaryandxs:base64Binarydid not inter-convert. Both were stored as plain strings, so the cast reinterpreted the lexical form instead of re-encoding the octets —xs:hexBinary(xs:base64Binary('D7c='))was an error rather than0FB7.
Working through the failure clusters found nine more, all now fixed:
| Bug | What was wrong |
|---|---|
| Casting table unenforced | The cast functions read the lexical form, so xs:base64Binary("10010101") castable as xs:float answered true. The source type is now checked against the spec's table. |
| Derived types had no facets | xs:byte and xs:token were aliases for their primitives, so 128 castable as xs:byte was true and xs:token("a	b") kept its tab. Both range and whitespace facets now apply. |
xs:QName not castable from a string |
CastAtomic had no QName case at all, so "ABC" castable as xs:QName was false. |
| Duration division unreachable | divideDurations existed and was correct, but the operator dispatch never called it — PT2H div PT1H raised "not defined on two durations". |
| Duration lexical forms too permissive | big.Rat.SetString accepts .5 and 30., so PT.5S parsed as half a second instead of being rejected. |
fn:codepoints-to-string unvalidated |
Any integer became a rune, so codepoint 0 silently produced U+FFFD instead of FOCH0001. |
Multi-line $ |
Go treats the position after a trailing newline as an empty final line; XML Schema does not, so ^$ matched "abcd\ndefg\n". |
to truncated its operands |
1.1 to 3 became 1 to 3 rather than raising XPTY0004 — a range the author never wrote. |
fn:QName unvalidated |
QName("http://x", "1person") built a QName whose lexical form cannot be written in any document. |
Together with the four crashes and hangs above, that took the in-scope pass rate from 92.42% to 98.16% — measured with the harness's original loose error check, which accepted any error where a specific code was expected. Tightening that check later dropped the honest figure to 96.22%, and grinding the tail down from there brought it to 100%; see below.
What the one remaining failure is is covered under Where it fails above: a backreference to a group whose width can vary, which RE2 cannot resolve and this refuses rather than guesses at.
Every other divergence between RE2 and the XML Schema flavour has been
implemented, and most of them had to be, because RE2 tends to disagree
silently rather than refuse. Character-class subtraction is implemented by
expanding both classes into codepoint ranges and taking the difference, but
only where both sides are literal characters and ranges — subtracting from
\d or \p{L} would need the Unicode tables that define them and is still
refused rather than approximated.
Known weaknesses of the harness itself, stated because a conformance number is only as honest as what it measures:
- Error codes are compared, not just error-ness. An expected error is
satisfied by any error only when the engine produced no code at all;
otherwise the code must match. This is the single assumption that most
inflates a conformance number — turning the check on dropped the score from
98.16% to 91.62% and exposed 961 wrong codes, most of them one systematic
confusion between "this value is wrong for the type" (
FORG0001) and "this conversion is not defined at all" (XPTY0004). serialization-matchesandassert-serialization-errorare unimplemented and count as failures, not skips. This understates the pass rate.assert-xml,assert-permutationand expression-valuedassertare implemented.- Every skip is now a spec or feature dependency, not a missing file. An earlier run reported 503 cases skipped for an unavailable source; those were the path-resolution bug above, not the checkout.
- Two assertion conventions are honoured that a naive harness gets wrong: the
catalog's
code="*"wildcard means "an error, unspecified" rather than a code spelled*, and anxsd-versiondependency selects between 1.0/1.1 pairs that assert opposite results for the same expression (xs:double("+INF")is an error under 1.0 andINFunder 1.1). This engine implements 1.1. Getting either wrong scores correct behaviour as failure.
What is still unverified. There is no XSLT 2.0 equivalent to this run: the W3C's XSLT suite (w3c/xslt30-test) targets XSLT 3.0, and its catalog format and result assertions differ enough that the QT3 harness does not carry over. The XSLT layer's evidence remains the Saxon differential corpora above. If you are putting this in front of a rule set that matters, diff its output against Saxon on your own corpus first — which is exactly how the earlier bugs were found.
The W3C xsdtests suite
The official W3C test suite for XML Schema. Like QT3 it is not vendored — it belongs to the W3C and is ~230 MB checked out:
git clone --depth 1 https://github.com/w3c/xsdtests.git testdata/xsdtests
Unlike QT3 there is no go test integration: the driver is not in the
repository, because it is a throwaway that walks suite.xml, loads each
schemaTest, validates each instanceTest, and compares against the
<expected> validity. The figures quoted in this file were produced by such a
driver built against the tree at HEAD.
If you are reproducing them, three details decide whether your numbers mean anything, and each one silently inflated an earlier measurement here:
versionis a list of tokens, not a string. OntestSet,testGroup,schemaTestandinstanceTestthe tokens are joined by OR — run the test if you support any of them. On<expected>the connector is AND. Comparing the attribute to"1.0"scores the multi-token spellings as neither version.statuslives on<current>, not on<expected>. A test markedqueriedis one the W3C's own suite disputes; 49 of the 1.0 disagreements and 48 of the 1.1 ones are in that category.- Invalid-by-design schemas are the point, not something to skip. Skipping them was a measurement bug here that hid roughly 14,000 real tests.
Measured against w3c/xsdtests at commit
7bc3365c (2026-04-01):
| schema-validity | instance | |
|---|---|---|
| XSD 1.0 | 14,385 / 14,388 (99.98%) | 24,973 / 25,000 (99.89%) |
| XSD 1.1 | 15,350 / 15,354 (99.97%) | 26,217 / 26,222 (99.98%) |
Every failure is catalogued in docs/conformance-gaps.md, with a verdict on whether it is fixable; docs/known-gaps.md is the reasoning behind the hard ones.
Production corpora
The highest-yield fixture, and the only guard against a schema-validity rule that is stricter than the spec — the conformance suite scores agreement with W3C labels, so a rule that is merely too strict shows up only if the suite happens to contain a valid schema exercising it. Real schemas catch it.
| corpus | source | result |
|---|---|---|
| UBL 2.1 | OASIS UBL 2.1 | 65 of 65 schemas load clean |
| UN/CEFACT CII, EN 16931 | ConnectingEurope/eInvoicing-EN16931 | 427 of 427 load clean |
| Factur-X / ZUGFeRD (all profiles) | factur-x | 21 load; samples validate clean |
| Peppol BIS Billing 3.0 | OpenPEPPOL | 24 UBL instances validate clean |
UBL needs ParseOptions{AllowDOCTYPE: true}: its dependency graph reaches the
W3C XML Signature schema, which carries a DOCTYPE, and without the flag all 65
fail with a cascade of unresolved ds: references from the one refused
include.
Where this is going
The conformance tail is no longer the interesting work. XPath is at 100% on all
three versions, XQuery has 1 failure left and XSLT 42 across both targets —
cases where the suite disagrees with the specification, where matching it would
cost XSD tests, or which want byte-identical reproduction of another
processor's indentation. That leaves none genuinely open: the last two were
validation-0201 on both targets, and the engine defect behind them — a
union's selected member type dropped on every tree copy, so that
xsl:strip-space silently untyped a validated document — is fixed. The case
still fails, on the indent width alone, which is implementation-defined.
The full catalogue is docs/known-gaps.md. Three larger
things are open, in rough order of how much they would change:
- Receiver-based output. The runtime builds a result tree and serialises it. Emitting events to a receiver instead is what makes streaming possible and would cut peak memory on large documents — it is the one change here that is architectural rather than additive.
- Schema-aware atomisation.
xsl:import-schemaloads a schema and makes its type names available, and a value's selected union member now survives a tree copy, so type assertions hold acrossxsl:strip-spaceandxsl:copy-of. What is still missing is the typed value itself: a validated<price>10.50</price>atomises as untyped, because the value would have to be carried on the node rather than its name. Stylesheets relying on type assertions work; ones relying on schema-aware arithmetic do not. - A differential harness as a standing target. The suites feed well-formed input and measure what happens after. The nested-occurrence defect — a schema admitting 5 children where only 10 were valid, and rejecting 10 — was found by generating models and comparing against an oracle, and no suite agreement reached it. That technique is still a one-off rather than something CI runs.
Contributions are welcome, particularly a differential against a corpus this has not seen — that is how most of the bugs above were found, and the failure modes it catches are the ones no suite covers.
Acknowledgements
Martin Honnen has found most of the user-facing XSLT bugs this project has fixed, and found them the hard way: a reduced test case, the same stylesheet run through Saxon for comparison, and often the diagnosis as well. More than one of them was a defect the W3C suites do not reach at all, which is the kind only a real user hits.
He also maintains the playground under Live test.
Licence
MIT. See LICENSE.
The third-party corpora used for differential testing are not covered by it and are not distributed here; see How this was tested.
Directories
¶
| Path | Synopsis |
|---|---|
|
cmd
|
|
|
genfunctions
command
Command genfunctions extracts the F&O 3.1 function proformas into the normalized manifest xpath/spec/function-signatures.json.
|
Command genfunctions extracts the F&O 3.1 function proformas into the normalized manifest xpath/spec/function-signatures.json. |
|
go-xml
command
Command go-xml applies an XSLT 2.0 stylesheet to XML documents.
|
Command go-xml applies an XSLT 2.0 stylesheet to XML documents. |
|
Package dtd validates an XML document against the DTD in its internal subset.
|
Package dtd validates an XML document against the DTD in its internal subset. |
|
internal
|
|
|
fileuri
Package fileuri spells a filesystem path as a file: URI.
|
Package fileuri spells a filesystem path as a file: URI. |
|
uripath
Package uripath holds the one decision every local-file resolver has to make before it parses a reference as a URI: whether the reference is a Windows drive-letter path rather than something naming a scheme.
|
Package uripath holds the one decision every local-file resolver has to make before it parses a reference as a URI: whether the reference is a Windows drive-letter path rather than something naming a scheme. |
|
version
Package version holds the go-xml module's release version.
|
Package version holds the go-xml module's release version. |
|
xmlfork
Package xmlfork implements a simple XML 1.0 parser that understands XML name spaces.
|
Package xmlfork implements a simple XML 1.0 parser that understands XML name spaces. |
|
xmlname
Package xmlname holds the XML Name productions.
|
Package xmlname holds the XML Name productions. |
|
Package relaxng validates XML documents against RELAX NG schemas.
|
Package relaxng validates XML documents against RELAX NG schemas. |
|
tests
|
|
|
conformance
Package conformance generates the conformance summary that docs/conformance-gaps.md publishes, from the measured counts recorded in tests/conformance/results.json.
|
Package conformance generates the conformance summary that docs/conformance-gaps.md publishes, from the measured counts recorded in tests/conformance/results.json. |
|
corpora
command
Command corpora loads production schemas and reports which ones fail.
|
Command corpora loads production schemas and reports which ones fail. |
|
qt3
Package qt3 runs the W3C QT3 (FOTS) test suite against this engine.
|
Package qt3 runs the W3C QT3 (FOTS) test suite against this engine. |
|
xsdsuite
command
|
|
|
xslts
Package xslts runs the W3C XSLT test suite against this engine, filtered to the tests an XSLT 2.0 processor is expected to pass.
|
Package xslts runs the W3C XSLT test suite against this engine, filtered to the tests an XSLT 2.0 processor is expected to pass. |
|
w3cschemas
module
|
|
|
Package xdm implements the XQuery/XPath Data Model (XDM) that XPath 2.0 and XSLT 2.0 are defined over.
|
Package xdm implements the XQuery/XPath Data Model (XDM) that XPath 2.0 and XSLT 2.0 are defined over. |
|
Package xdmbuild constructs XDM sequences and trees from the results of a sequence constructor.
|
Package xdmbuild constructs XDM sequences and trees from the results of a sequence constructor. |
|
Package xpath implements XPath 2.0: lexing, parsing to an AST, static analysis, and evaluation against an XDM tree.
|
Package xpath implements XPath 2.0: lexing, parsing to an AST, static analysis, and evaluation against an XDM tree. |
|
Package xquery implements XQuery 3.1.
|
Package xquery implements XQuery 3.1. |
|
Package xsd implements XML Schema 1.0 and 1.1 validation.
|
Package xsd implements XML Schema 1.0 and 1.1 validation. |
|
Package xslt implements XSLT 2.0 transformation over the xdm data model, using the xpath package for expression evaluation.
|
Package xslt implements XSLT 2.0 transformation over the xdm data model, using the xpath package for expression evaluation. |