Documentation
¶
Overview ¶
Package xmlx bounds the work an untrusted XML document can cost before and during an encoding/xml decode.
A byte cap on the response body is not a bound on decoding. encoding/xml materializes each token before any caller-side check runs, so a wire-capped body can still force allocations far past its own size:
- One token can be as large as the body. A single text node, attribute value, or start tag inside an 8 MB response is an 8 MB allocation before the caller's struct field sees it. The decoder's internal buffer never shrinks, so the largest token is a high-water mark for the whole decode.
- Element count amplifies. Millions of three-byte elements fit in a few megabytes of wire and expand into a decoded object graph many times larger.
- Nesting grows the decoder's element stack. The tokenizer pushes one heap-allocated entry per open element. Unmarshal does carry a fixed internal ceiling on its own recursion (10000 open elements, 5000 on wasm; the guard was introduced for CVE-2022-30633 and rebuilt for CVE-2026-56859, which closed a DecodeElement bypass), but the decoder samples it only where its schema recursion goes, and every child the caller's schema does not model is consumed by Decoder.Skip, which is iterative and has no depth bound at all. Measured on go1.27.0: a document 349,525 elements deep decodes clean under a schema that models only the root. A body of `<a><a><a>...` converts each 3 bytes of wire into a live stack entry.
- Concurrency multiplies all three. Each in-flight request holds its own copy of the worst case.
The package is two halves, matching where the cost is paid:
- Preflight is a lexical gate over the RAW bytes, run BEFORE the decoder sees them. It walks the document's surface structure (text runs, tags honoring quoted '>' bytes, comments, processing instructions, CDATA sections) and rejects a body already outside the caller's contract, in one allocation-free scan. Only a gate over the raw bytes can bound what the decoder must materialize.
- Budget is the decode-time text accounting a schema decoder charges each retained value against: a per-value cap and a cumulative document-wide cap, both applied before the value is stored. It bounds what the program BUILDS and KEEPS, which is the part Preflight cannot see, because raw bytes and decoded text are different quantities.
The two are complements. Preflight alone leaves the caller free to retain every byte it admits; Budget alone bounds no element count and no nesting, since Decoder.Skip still walks the parts the schema ignores.
Schema decoding stays the caller's. This package owns only the scaffold, the part that gets hand-rolled, subtly differently, in every program that parses XML it did not write. Per-name cardinality ("at most N <item> elements") needs the caller's vocabulary and belongs at the caller's decode site; the vocabulary-free total is Limits.MaxElements.
Typical use ¶
if err := xmlx.Preflight(body, xmlx.DefaultLimits()); err != nil {
return err
}
var doc feed
if err := xml.Unmarshal(body, &doc); err != nil { // custom UnmarshalXML
return err
}
with the document's UnmarshalXML methods charging a Budget as they decode.
Fail-closed, no silent defaults ¶
Every bound is explicit. A non-positive limit is a configuration mistake, reported as ErrInvalidLimits, never read as "unbounded": a bounds library whose zero value bounds nothing is the failure it exists to prevent. Use DefaultLimits and DefaultBudget as a starting point and size them to the document contract.
XML directives are rejected, deliberately ¶
Preflight refuses every `<!` form that is not a comment or a CDATA section: `<!DOCTYPE`, `<!ENTITY`, `<!ATTLIST`, `<!NOTATION`. This is a scope decision. encoding/xml tokenizes a directive by tracking nested '<'/'>' pairs, with quoting and nested comments, accumulating until a '>' at depth zero. A scan that merely stopped at the first unquoted '>' would report a short token where the decoder retains one the size of the whole body: a bound that reads as protection and is not. The objection is not that reproducing that tokenizer is laborious, it is that a silent divergence in a copied tokenizer is unfalsifiable at the call site while a refusal is falsifiable immediately. Rejecting the class by default also matches the ecosystem norm for untrusted XML.
A document that legitimately carries a directive cannot use Preflight; decode it under a byte cap and a Budget instead. Legacy RSS 0.91 did require a DOCTYPE, so the class is not extinct, only absent from the modern dialects.
Not in scope ¶
XXE and entity expansion are not this package's concern: encoding/xml does not resolve external entities and does not expand a DTD's internal entities, and the directive rejection closes that surface as a side effect.
Round-trip stability is a different hardening axis and is not covered. Go's XML parser uniquely accepts leading and trailing garbage around the document element, which was the mechanism behind a real authentication bypass (CVE-2020-16250); a caller whose security depends on the document's shape, rather than on its cost, wants a round-trip validator alongside Preflight.
Schema validation, namespace policy, character-encoding conversion, and well-formedness are all left to encoding/xml. Preflight never validates. The converse does not hold: malformed input can still trip a bound, so a rejection proves the document was outside the contract, not that it was oversized.
Decoder configuration ¶
The bounds model the DEFAULT decoder: Strict enabled, AutoClose empty, no caller-supplied Entity map, no CharsetReader. Each of those changes what the bytes mean, and two change it enough to matter: with Strict disabled a bare attribute name becomes an attribute the lexical count cannot see, and an Entity map can expand a short reference into an arbitrarily long value after the raw bound has passed. A caller that sets either keeps the token, depth and element bounds but should treat the attribute and text bounds as advisory.
Index ¶
Examples ¶
Constants ¶
This section is empty.
Variables ¶
var ( // ErrLimit reports that a document exceeded one of the caller's bounds. // Every rejection from Preflight and Budget wraps it, so a consumer can // classify "the document was too big to be worth decoding" with one // errors.Is and read *LimitError only when it wants the specific bound. ErrLimit = errors.New("xmlx: limit exceeded") // ErrInvalidLimits reports a non-positive bound: a configuration mistake in // the CALLER, not a property of the document. It is deliberately not // treated as an unbounded setting, see the package doc. ErrInvalidLimits = errors.New("xmlx: invalid limits") )
Sentinel errors, matched with errors.Is through the wrapped errors this package returns.
Functions ¶
func Preflight ¶
Preflight walks the raw bytes of an XML document and rejects one that is already outside lim, BEFORE any decoder allocates a token from it. It is a single allocation-free pass: one sequential scan, no buffering, no copies.
It rejects an overlong raw text or CDATA run, an overlong markup token, a start tag carrying more than lim.MaxTagAttrs XML attributes, more than lim.MaxElements elements, element nesting whose PEAK exceeds lim.MaxDepth (a self-closing element counts as one level), and any XML directive (see the package doc). Each rejection is a *LimitError naming the bound and the byte offset where it was detected, and every one wraps ErrLimit.
Preflight is a bounds check, not a validator. It reads the document's surface structure only, enough to know where one token ends and the next begins, so a malformed body that stays within the bounds passes through for the decoder to reject with its own parse error. The converse does not hold: malformed input can also trip a bound (a run of `=` bytes reads as attributes; a truncated `<!` opener is refused with the directive class), so a rejection does not prove the document was oversized, only that it was outside the contract.
Decoder configuration ¶
The bounds model the DEFAULT decoder: Strict enabled, AutoClose empty, no caller-supplied Entity map, no CharsetReader. Each of those knobs changes what the bytes mean. With Strict disabled a bare attribute name becomes an attribute this scan does not count; an Entity map can expand a short reference into an arbitrarily long value after the raw bound has passed; a CharsetReader transforms the very bytes that were scanned. A caller that sets any of them keeps the token and depth bounds but should treat the attribute and text bounds as advisory.
body is not modified or retained.
Example ¶
The gate runs over the raw bytes before the decoder sees them, so a document outside the caller's contract costs one scan instead of a decode.
package main
import (
"fmt"
"github.com/cplieger/xmlx"
)
func main() {
body := []byte(`<rss><channel><title>Example</title></channel></rss>`)
if err := xmlx.Preflight(body, xmlx.DefaultLimits()); err != nil {
fmt.Println("rejected:", err)
return
}
fmt.Println("accepted")
}
Output: accepted
Example (Directive) ¶
An XML directive is refused as a class: bounding one truthfully means reproducing encoding/xml's own directive tokenizer, so the package declines instead of reporting a bound it cannot honor.
package main
import (
"fmt"
"github.com/cplieger/xmlx"
)
func main() {
err := xmlx.Preflight([]byte(`<!DOCTYPE rss><rss/>`), xmlx.DefaultLimits())
fmt.Println(err)
}
Output: xmlx: XML directives are not allowed
Example (LimitError) ¶
A rejection names which bound fired, so a consumer can log the specific contract the upstream broke instead of "too big".
package main
import (
"errors"
"fmt"
"strings"
"github.com/cplieger/xmlx"
)
func main() {
// A body nested far past a feed's real shape: each 3 bytes of wire would
// cost the decoder one live open-element stack entry.
body := []byte(strings.Repeat("<a>", 5000))
err := xmlx.Preflight(body, xmlx.DefaultLimits())
if le, ok := errors.AsType[*xmlx.LimitError](err); ok {
fmt.Println(le.Kind, "limit:", le.Limit)
}
fmt.Println("is a limit error:", errors.Is(err, xmlx.ErrLimit))
}
Output: element nesting depth limit: 64 is a limit error: true
Types ¶
type Budget ¶
type Budget struct {
// contains filtered or unexported fields
}
Budget is the decode-time text accounting for ONE document: a per-field cap on any single decoded value, and a cumulative cap on everything the document retains. It is the half of this package that Preflight cannot cover, because raw bytes and decoded text are different quantities: entity references, CDATA seams, and repeated elements all break the correspondence, so a document can pass every lexical bound and still hand a schema decoder more text than the caller means to hold.
What it bounds is what the caller BUILDS and KEEPS. The per-token allocation happens inside encoding/xml before any check here can run, which is precisely why Preflight exists: only a gate over the raw bytes bounds what the decoder must materialize. The two are complements, not alternatives.
Use one Budget per document, threaded through the decoders that charge it. Create it with NewBudget or DefaultBudget: the caps are immutable afterwards, so a document cannot be part-way through its allowance when the allowance changes. Copying a Budget would fork its running total, leaving two copies each believing they hold the whole allowance, so a copy is a vet error. It is not safe for concurrent use; a document decodes on one goroutine.
A Budget does NOT bound element count or nesting. A caller using Budget alone still lets encoding/xml skip unknown children through Decoder.Skip, which is iterative and unbounded, so the decoder's element stack still grows to the document's true depth. Only Preflight bounds that.
Example ¶
A schema decoder charges each retained value against one Budget, so the document is bounded by what it makes the program HOLD, not only by its wire size.
const doc = `<item><title>Show & Tale</title><guid>x1</guid></item>`
budget := mustBudget(64, 4096)
var title, guid string
d := xml.NewDecoder(strings.NewReader(doc))
for {
tok, err := d.Token()
if err != nil {
break
}
start, ok := tok.(xml.StartElement)
if !ok {
continue
}
switch start.Name.Local {
case "title":
title, err = budget.DecodeText(d)
case "guid":
guid, err = budget.DecodeText(d)
}
if err != nil {
fmt.Println("rejected:", err)
return
}
}
fmt.Printf("%s / %s / charged %d bytes\n", title, guid, budget.Total())
Output: Show & Tale / x1 / charged 13 bytes
func DefaultBudget ¶
func DefaultBudget() *Budget
DefaultBudget returns the decode-time counterpart of DefaultLimits: 4 KiB per decoded value, 4 MiB of decoded text per document. The gap between this and DefaultLimits.MaxTextRunBytes is entity-expansion headroom (see that field).
func NewBudget ¶
NewBudget returns a Budget bounding one document to maxFieldBytes per decoded value and maxTotalBytes of decoded text overall. Both must be positive: a non-positive cap is a configuration mistake (ErrInvalidLimits), never read as unbounded.
Size maxTotalBytes from what the caller is willing to HOLD, which is usually far below the transport cap. A consumer parsing a catalogue-scale document sets it at that document's real ceiling; the DefaultBudget values are for a small structured document and will reject a multi-megabyte payload.
func (*Budget) Charge ¶
Charge accounts one already-decoded value against both caps and reports whether it may be retained. It is what a decoder calls for a value encoding/xml has handed it whole, such as an attribute value off a StartElement.Attr, where the allocation has happened but retention has not.
Call it BEFORE storing the value: a decoder that stores first and charges after has already kept the document it was about to reject.
Every occurrence is charged, including repeats of the same element name. That is deliberate: a document that sends <title> ten thousand times overwrites one destination field but costs ten thousand decodes, so charging only the value that survives would leave that repetition unaccounted. Note the bound is on BYTES, so repetition of EMPTY values costs nothing here; bounding the number of elements is Limits.MaxElements's job, at the lexical gate.
Charge each value exactly once. A value returned by DecodeText has already been charged.
func (*Budget) DecodeText ¶
DecodeText reads the text content of the element the decoder has just entered, bounded and charged, and returns it. It replaces d.DecodeElement(&s, &start) for a plain text field.
The difference that matters is WHEN the cap applies. DecodeElement concatenates every CharData token in the element and hands back the finished string, so a value split across CDATA seams, each chunk individually small enough to pass any lexical bound, is only measurable after it exists. DecodeText accumulates under the per-field cap AND under the document's remaining allowance, stopping at the token that would cross either, so the string it builds never exceeds what the caller would accept.
Nested markup is skipped whole; comments and processing instructions are ignored, as DecodeElement ignores them.
Where the two stop agreeing, measured ¶
The swap is byte-identical to DecodeElement for every value inside encoding/xml's own acceptance set, and the fuzz oracle pins that. It is NOT identical outside it, in ONE respect, and the difference is a real acceptance-set divergence rather than a wording caveat.
encoding/xml guards its unmarshal recursion by sampling the decoder's live open-element count on each entry into that recursion, against a fixed internal ceiling: 10000, and 5000 when GOARCH is wasm. DecodeElement enters that recursion, so it inherits the ceiling. Measured on go1.27.0 by entering an element at a known open depth: the two agree to open depth 9999, and from open depth 10000 DecodeElement refuses with an unexported, unwrapped errors.New("exceeded max depth") while DecodeText returns the value. So DecodeText is the LOOSER of the two above that depth.
That is deliberate rather than an omission. DecodeText is iterative (Token plus Skip), so it carries none of the recursion the ceiling exists to bound, and nesting is Preflight's bound in this package's split, not a Budget's: a Budget bounds bytes retained, never document shape. A caller that wants the decoder's ceiling enforced sets Limits.MaxDepth at or below it and runs Preflight, which rejects first and reports KindDepth wrapping ErrLimit -- a classifiable error, which the stdlib's is not.
On success the element's end tag is consumed, so the caller's token loop continues at the next sibling. On ANY error the document is over, and the caller must abandon it: a rejected value can leave the decoder part-way through the element, and a decoder error leaves it wherever encoding/xml stopped. There is no defined position to resume from.
Example (SplitValue) ¶
The per-field cap applies while the value accumulates, so a value split across CDATA seams, every chunk individually small, is refused at the token that would cross it rather than after the whole string exists.
doc := "<title>" + strings.Repeat("<![CDATA[xxxxxxxx]]>", 20) + "</title>"
budget := mustBudget(32, 4096)
d := xml.NewDecoder(strings.NewReader(doc))
if _, err := d.Token(); err != nil {
return
}
_, err := budget.DecodeText(d)
fmt.Println(err)
fmt.Println("charged:", budget.Total())
Output: xmlx: decoded field longer than 32 bytes charged: 0
func (*Budget) MaxFieldBytes ¶
MaxFieldBytes reports the per-value cap this Budget was built with.
func (*Budget) MaxTotalBytes ¶
MaxTotalBytes reports the document-wide cap this Budget was built with.
type ConfigError ¶
type ConfigError struct {
// Field is the struct field name of the offending bound.
Field string
// Value is what the caller set it to.
Value int
}
ConfigError reports a non-positive bound, naming the field that is wrong. It wraps ErrInvalidLimits.
type Kind ¶
type Kind uint8
Kind names which bound a *LimitError reports. It exists so a consumer can branch or log on the specific bound without parsing an error string: a depth rejection and an oversized-field rejection say different things about the upstream that sent the document.
const ( KindUnknown Kind = iota // KindTextRun: one contiguous run of raw character data exceeded // Limits.MaxTextRunBytes. KindTextRun // KindCDATA: one CDATA section's content exceeded Limits.MaxTextRunBytes. // It shares the text bound because a CDATA section IS character data, just // spelled so that markup bytes inside it are literal. KindCDATA // KindComment: one comment exceeded Limits.MaxTokenBytes. KindComment // KindProcInst: one processing instruction exceeded Limits.MaxTokenBytes. KindProcInst // KindToken: one tag exceeded Limits.MaxTokenBytes. KindToken // KindTagAttrs: one start tag carried more than Limits.MaxTagAttrs XML // attributes. KindTagAttrs // KindDepth: element nesting exceeded Limits.MaxDepth. KindDepth // KindElements: the document carried more elements than // Limits.MaxElements. KindElements // KindDirective: the document carried an XML directive (a DOCTYPE, ENTITY, // ATTLIST or NOTATION declaration), or a truncated `<!` opener that could // only become one. Preflight rejects the whole class; see the package doc // for why. Its LimitError carries no numeric bound. KindDirective // KindField: one decoded value exceeded Budget.MaxFieldBytes. KindField // KindTotalText: the cumulative decoded text charged to one Budget // exceeded Budget.MaxTotalBytes. KindTotalText )
The bounds this package enforces. KindUnknown is the zero value and names no bound; it never appears in a returned error.
type LimitError ¶
type LimitError struct {
// Kind names the bound that fired.
Kind Kind
// Limit is the configured bound, or 0 for a bound that is not numeric
// (KindDirective).
Limit int
// Offset is the byte offset in the document where the rejection was
// detected. Preflight sets it; Budget leaves it 0, because a decoded value
// has no single offset and the caller's decoder knows which field it was
// reading (wrap the error with that name).
Offset int
}
LimitError reports which bound a document exceeded. Match it with errors.AsType; match the class with errors.Is against ErrLimit.
It carries no excerpt of the document. That is deliberate: the offending bytes are untrusted input, and an error built from them would be an unbounded, unsanitized string on its way to a log line, which is the amplification this package exists to stop reintroduced through its own diagnostics. Offset is the safe half of that diagnostic need: one bounded integer, no attacker-chosen content, and enough to find the offending token in a saved payload.
func (*LimitError) Is ¶
func (e *LimitError) Is(target error) bool
Is reports ErrLimit for every LimitError, so a consumer that only cares about the class does not have to enumerate kinds.
type Limits ¶
type Limits struct {
// MaxTextRunBytes caps CHARACTER DATA: one contiguous run of raw text, or
// one CDATA section's content. Both are measured as content, which is what
// the decoder hands back as a CharData token.
//
// This bounds RAW bytes, while a decoded-value cap (Budget) bounds resolved
// text. The two are not interchangeable and the raw one must be the looser:
// a reference expands to fewer bytes than it occupies, and the widest
// predefined one is 6 to 1 (`"` and `'` are six raw bytes for one
// decoded), so a run that would decode within a 4 KiB value cap can
// legitimately arrive as tens of KiB of raw bytes. Setting this at or below
// the decoded cap would reject valid documents.
MaxTextRunBytes int
// MaxTokenBytes caps one MARKUP TOKEN whole, delimiters included: a tag
// (`<` through `>`), a comment (`<!--` through `-->`), or a processing
// instruction (`<?` through `?>`). A start tag carrying MaxTagAttrs
// attributes of the largest value the caller accepts should still fit with
// margin.
MaxTokenBytes int
// MaxTagAttrs caps the XML attributes on ONE start tag.
//
// This is a LEXICAL bound on attributes written inside a tag
// (`<enclosure url=".." length=".."/>` is two), which is a different
// question from how many child ELEMENTS the caller's schema permits. The
// latter is schema cardinality and belongs at the caller's decode site.
//
// Attributes are counted as `=` bytes outside a quoted value, which is
// exact for well-formed XML (the grammar is Name Eq AttValue) and for
// everything the DEFAULT decoder accepts. It is not exact for a decoder
// with Strict disabled, which accepts a bare name as an attribute whose
// value is its own name; such a tag carries attributes this bound does not
// see. See Preflight's decoder-configuration note.
MaxTagAttrs int
// MaxDepth caps element nesting.
//
// This is the bound with the widest gap between wire size and cost: the
// decoder pushes one heap-allocated entry per open element, and its
// Decoder.Skip path (which every child the caller's schema does not model
// goes through) has no depth bound at all, so a body of three-byte start
// tags converts each 3 bytes of wire into a live stack entry for the whole
// decode. Set it to the document's real shape (a syndication feed is about
// 4 deep), not to a round number.
//
// It bounds the PEAK number of simultaneously open ELEMENTS, which is what
// the decoder's stack holds. A self-closing element counts: encoding/xml
// pushes `<e/>` and pops it on the synthesized end tag, so `<a><b/></a>`
// reaches depth 2, not 1. The decoder additionally pushes one entry per
// namespace declaration it meets, so the stack LENGTH is bounded by
// MaxDepth * (1 + MaxTagAttrs) rather than by MaxDepth alone. Those
// namespace entries do NOT feed the decoder's own depth guard below, which
// counts start elements only.
//
// # The decoder's own ceiling, and where it overlaps this bound
//
// encoding/xml carries a fixed internal ceiling of its own on the same
// quantity: 10000 open elements, and 5000 when GOARCH is wasm. It is not an
// alternative to this bound, because the decoder SAMPLES it only on each
// entry into its unmarshal recursion rather than tracking the document's
// peak. Measured on go1.27.0: a document 349,525 elements deep under a
// schema that models only the root is accepted, and Decoder.Skip walks it
// unbounded, so a caller relying on the decoder's ceiling has no depth
// bound at all for the part its schema ignores. That is the case this
// bound exists for.
//
// Where the two do overlap is a schema whose own nesting follows the
// document's, which is what makes the region of MaxDepth above the
// decoder's ceiling unreachable for such a schema: measured at MaxDepth
// 10001, a 10001-deep document passes Preflight and is then refused by
// xml.Unmarshal with an unexported, unwrapped errors.New("exceeded max
// depth") that no errors.Is can classify. So keep MaxDepth at or below
// 10000 (5000 for a wasm build) whenever the schema is nested as deeply as
// the document, and this package reports the rejection instead, as
// KindDepth wrapping ErrLimit.
MaxDepth int
// MaxElements caps the total number of elements in the document.
//
// It is the bound for amplification by COUNT rather than by size: a body of
// millions of three-byte elements passes every per-token bound above (each
// token is tiny, the nesting is flat, there is no text at all) and still
// expands into a decoded object graph many times the wire size. Depth
// cannot catch it and neither can a text budget, because empty elements
// carry no text to charge.
//
// This is a lexical count of start tags, not schema cardinality: it needs
// no vocabulary from the caller, and unlike an "at most N <item>" rule it
// protects a plain xml.Unmarshal consumer that has no custom decode site to
// put such a rule in.
MaxElements int
}
Limits are Preflight's lexical bounds over the raw document bytes. Every field must be positive; a non-positive value is a configuration mistake (ErrInvalidLimits), never an unbounded setting.
Size these from the document CONTRACT, not from the transport cap. The question each bound answers is "what would a legitimate document from this endpoint never exceed?", and the useful answer is usually orders of magnitude below the byte cap: that gap is the amplification headroom the preflight removes.
func DefaultLimits ¶
func DefaultLimits() Limits
DefaultLimits returns bounds sized for a small structured document, such as a syndication feed or an XML API response, where fields are short, elements carry a handful of attributes, and nesting is shallow.
They are a starting point, not a recommendation. A caller that knows its document contract should tighten them, and one parsing genuinely large documents must raise MaxTextRunBytes, MaxDepth and MaxElements deliberately rather than discovering the rejection in production: a catalogue-scale dump of tens of thousands of records will exceed MaxElements, which is the point of the bound but not a surprise worth having at 3am.