goaxi

README

go-axi

Safe TOON output and input for Go command-line tools. A hardening and guard layer over the official codec, toon-go.

TOON is the token-efficient output format behind AXI principle 1, and toon-go is its official Go implementation. toon-go encodes and decodes. go-axi covers what an agent-facing CLI still has to get right before it prints, and when it reads TOON back in:

• Sanitize untrusted text. Strip ANSI escapes, U+2028/U+2029, every Unicode format character (bidi controls, zero-width and tag characters among them), lone C1 bytes and invalid UTF-8 out of anything the tool did not author, without dropping the surrounding visible text.
• Catch silent encoding failures. Some Go types encode to empty output instead of an error, so a command prints nothing and exits 0. Check turns that into a condition a test can assert on.
• Know when TOON is not the cheaper choice. Check measures the TOON payload against the JSON one and reports which is smaller.
• Read a tabular array back without losing what toon-go's public API can't return. DecodeTabular keeps the declared field order, delimiter and row count, and handles a paginated payload correctly; Decode reads any other document, forwarding straight to toon-go.
• Ship one exit-code set and one help[] writer as compiled constants instead of a convention in a style guide.

Install

go get github.com/samestrin/go-axi

Go 1.23 or newer. One dependency: toon-go.

Quick start

package main

import (
	"fmt"
	"os"

	goaxi "github.com/samestrin/go-axi"
)

type run struct {
	ID     string `toon:"id"`
	Status string `toon:"status"`
}

func main() {
	payload := map[string]any{
		"total": 2,
		"runs": []run{
			{ID: "4812", Status: "passed"},
			{ID: "4813", Status: "failed"},
		},
	}

	if err := goaxi.Encode(os.Stdout, payload); err != nil {
		fmt.Fprintln(os.Stderr, "ci:", err)
		os.Exit(int(goaxi.ExitError))
	}

	if err := goaxi.WriteHelp(os.Stdout, []string{"ci logs 4813", "ci rerun 4813"}); err != nil {
		fmt.Fprintln(os.Stderr, "ci:", err)
		os.Exit(int(goaxi.ExitError))
	}

	os.Exit(int(goaxi.ExitOK))
}

runs[2]{id,status}:
  "4812",passed
  "4813",failed
total: 2
help[2]: ci logs 4813,ci rerun 4813

Encode sanitized the payload on the way out, and WriteHelp appended a contextual-disclosure block that decodes as part of the same document.

API

import goaxi "github.com/samestrin/go-axi"

Full reference: pkg.go.dev/github.com/samestrin/go-axi.

Writing output

Function	Behavior
Encode(w io.Writer, v any) error	Sanitize, encode as TOON, terminate with exactly one newline
EncodeChecked(w io.Writer, v any) (Verdict, error)	Sanitize, verify TOON carries it losslessly, write it — one pass for both jobs
EncodeOrJSON(w io.Writer, v any) error	Same, but fall back to a JSON envelope when TOON would lose data
WriteHelp(w io.Writer, lines []string) error	Write help[N]: first,second; an empty list writes nothing

Encode sanitizes internally so a caller cannot forget, and writes nothing at all when encoding fails — a partial payload is worse than none, because it parses.

Choosing a format

When a command supports --toon and --json, the choice is a value rather than a branch at every call site:

Symbol	Behavior
Format	TOON or JSON; Default is TOON
ParseFormat(s string) (Format, error)	Maps a flag value to a Format; "" selects Default
f.Encode(w io.Writer, v any) error	Encode in that format, terminated by exactly one newline
f.EncodeProjected(w io.Writer, v any) error	Same, but TOON field names come from the json tags

The format is chosen by the caller and never detected from the data. Output shape is a property of the command, not of the payload — a format that varied per call would force every consumer to handle both, and would make an instruction like "the third column is SEVERITY" unsafe to write down. EncodeOrJSON is the opposite policy, for callers with no design-time rule.

EncodeProjected exists because toon-go does not read the json struct tag. A type carrying only json tags encodes to its Go identifiers — Count: 3, not count: 3. There are two ways out: add a duplicate toon tag to every field, or route the value through its JSON form first. Encode is the first, EncodeProjected is the second.

Projection is not free. Medians of six runs on a 500-row payload, Apple M5:

Path	Time	Allocations
Encode	257 µs	5,427
EncodeProjected	665 µs	15,055

Reproduce with go test -run '^$' -bench Format_Encode -benchmem. That is about 2.6× the time and 2.8× the allocations, so Encode stays the fast path and the default. Projection applies to TOON only — encoding/json already reads the json tag, so a JSON payload is passed straight through.

Reading TOON back

Function	Behavior
Decode(r io.Reader) (any, error)	Read a whole TOON document of any shape, preserving toon-go's types
DecodeFile(path string) (any, error)	Same, from a file, naming it on error
DecodeTabular(r io.Reader) (*Document, error)	Read ONE tabular array, keeping its declared field order, delimiter and row count
DecodeTabularFile(path string) (*Document, error)	Same, from a file
IsTabularHeader(line string) bool	Does this line open a tabular array? The dispatch point between the two

Decode forwards to toon-go, so there is one decoder rather than two. It refuses an empty document, because toon-go reports one as a non-nil empty container and a caller cannot tell that from real data.

DecodeTabular exists because a tabular array declares three things about itself that toon-go's public API cannot give back. It decodes in one strict pass and returns a map, and its parsedHeader type is unexported — so the declared field ORDER, the delimiter, and the declared row count are all unreachable. Three of its rules also make the strict pass unusable on real CLI output: an explicitly written comma delimiter is rejected outright, a trailing block declaring more rows than it carries fails the whole document, and a row count disagreeing with the physical rows is an error in either direction.

That last one is not a defect to route around — it is load-bearing. A paginated payload caps the rows it emits while keeping the true total in the header, and says so with a truncated sibling. So DecodeTabular reads the header itself, collects the rows belonging to that array, and hands toon-go a synthesized document carrying the physical count. Strict mode stays on, which keeps toon-go's row-width gate. Document.Declared and len(Document.Rows) are both returned and the caller decides:

doc, err := goaxi.DecodeTabularFile("findings.axi")
if err != nil {
    return err
}
for _, row := range doc.Rows {         // map[string]string, keyed by field name
    fmt.Println(row[doc.Fields[0]])    // Fields is order-bearing, Rows is not
}
if len(doc.Rows) < doc.Declared {
    fmt.Printf("showing %d of %d\n", len(doc.Rows), doc.Declared)
}

More rows than declared is still refused — that is a disagreement no contract allows.

Catching a missing toon tag

Function	Behavior
CheckTags(v any) TagVerdict	Reports fields that would be published under their Go identifier

A missing toon tag does not error and does not produce empty output. It silently keys your payload on names no consumer was told to expect, which is the same defect class Check exists for. Assert it in a test over every type a command prints:

if v := goaxi.CheckTags(SearchResult{}); !v.OK {
	t.Error(v.String())
}

A value that is already generic — a map, a slice, a scalar, or anything EncodeProjected produced — has no Go identifiers to leak and is always OK. Unexported fields and json:"-" fields are never encoded, so neither is flagged.

CheckTags walks types rather than values, so its cost tracks the shape of the type and not the size of the payload: 368 ns and zero allocations on the benchmark payload. It terminates on a self-referential type.

EncodeChecked is what you want when the guard matters and you are about to print. Calling Check and then Encode sanitizes and marshals the same value twice to serve one guard; EncodeChecked derives its verdict from the bytes it writes. Medians of six runs on a 2000-row payload, with a bare Encode as the floor:

Writer	Time	Against the floor
Encode	1.05 ms	—
EncodeChecked	1.16 ms	+11%
EncodeOrJSON	1.17 ms	+12%
Check then Encode	2.77 ms	+164%

Reproduce with go test -run '^$' -bench Output_Writers -benchmem. The guard is worth about 11%; the rest of that last row is duplicated work.

It writes nothing unless the verdict is OK, and its Tier describes the bytes actually emitted.

EncodeOrJSON falls back on data loss only, never on size, to a self-describing envelope:

{"axi_format":"json","axi_notice":"<why>","data":{...}}

Route on the {"axi_format prefix rather than on a leading { or [. Four TOON shapes begin with [: [0]:, [2]: a,b, [2]: 1,2 and [1]{id}:.

If your project fixes the output format per command at design time, EncodeOrJSON is runtime format detection and you should keep your build-time guard instead.

Checking a value before you print it

type Verdict struct {
	OK           bool   // encoding preserves the data
	Tier         Tier   // TierTabular | TierNested | TierLossy
	Efficient    bool   // the TOON payload is no larger than the JSON one
	SizeCompared bool   // whether Efficient was actually measured
	Reason       string // explains a false OK or Efficient, in actionable terms
}

func Check(v any) Verdict                     // sanitizes internally
func CheckSanitized(v any, clean any) Verdict // for a value already sanitized
func CanEncode(v any) bool                    // Check(v).OK

Tier	Meaning
TierTabular	Uniform rows encoded as a table with a declared field list — where the token savings come from
TierNested	Lossless but not columnar: ragged rows, list-valued fields, nested objects
TierLossy	Encoding would discard data. Do not emit this as TOON

Tiers describe the shape actually emitted, measured rather than assumed.

SizeCompared exists because the comparison costs a second marshal, so EncodeChecked skips it. Without the flag, a skipped comparison would be indistinguishable from a measured "TOON is larger". Check always measures.

OK and Efficient are separate on purpose. A payload can be perfectly lossless and still cost more as TOON, which is a reason to choose JSON for that command but never a reason to call the value broken.

A Check assertion in a table test is the cheapest place to catch the empty-output trap:

func TestListOutputSurvivesTOON(t *testing.T) {
	v := goaxi.Check(listRunsPayload())
	if !v.OK {
		t.Fatalf("payload is lossy: %s", v.Reason)
	}
	if v.Tier != goaxi.TierTabular {
		t.Errorf("expected a tabular payload, got %s", v.Tier)
	}
}

Use CheckSanitized when you already hold the cleaned copy, so the value is not sanitized twice.

Sanitizing

Function	Returns	Use when
Sanitize(v any) (any, error)	Cleaned copy, same concrete types	Default
MustSanitize(v any) any	Cleaned copy; panics on error	Keys are fixed identifiers and shapes are acyclic
SanitizeString(s string) string	Cleaned string	Building output one field at a time

Concrete types are preserved, so toon: struct tags survive. That matters: TOON field names are part of the contract a consuming tool parses, and flattening to map[string]any would rename every column to its Go identifier.

Sanitize returns two error types, and neither can be resolved without losing or inventing data, so it refuses rather than guessing. Match them with errors.As:

clean, err := goaxi.Sanitize(v)
if err != nil {
	var cyc *goaxi.CycleError
	var dup *goaxi.KeyCollisionError
	switch {
	case errors.As(err, &cyc): // a self-referential value
	case errors.As(err, &dup): // two keys that clean to the same string
	}
}

*KeyCollisionError fires when, for example, "na\x1bme" and "name" both clean to "name". Dropping one is silent data loss; renaming one invents data the caller never wrote.

*CycleError refuses a self-referential value before the encoder sees it. That includes the cases a pointer-only check misses: a cycle closing through a map key, a slice that contains itself, and a loop that closes at any depth. Encoding one exhausts the stack, and stack exhaustion is a fatal Go runtime error: recover() cannot catch it, so the process dies with a goroutine dump where encoding/json would return a clean error.

Exit codes

ExitOK         = 0
ExitError      = 1 // the tool itself failed
ExitUsage      = 2 // unknown subcommand or flag — AXI requires failing loud
ExitNotFound   = 3
ExitValidation = 4 // ran correctly; the thing it checked did not pass

ExitCode is a defined type with a String() method, so os.Exit(int(goaxi.ExitValidation)).

ExitValidation is deliberately distinct from ExitError. A checker reporting "this does not conform" is a successful run with a real result, not a broken tool — and an agent cannot decide whether a retry is worthwhile if the two collapse into one code.

Performance

The guard layer is not where your time goes. Medians of six runs, Apple M5, Go 1.26 — every figure below is produced by a committed benchmark rather than quoted from a commit message:

Operation	Cost	Allocations
SanitizeString, clean short string	2.4 ns	0
SanitizeString, clean sentence	18.8 ns	0
Sanitize, clean 500-row payload	35 µs	1
Check, clean 20-row payload	17 µs	486
EncodeOrJSON, 2000 rows	1.17 ms	33,974

Two of those are worth calling out. Sanitizing a payload that needs no cleaning costs one allocation regardless of size, because a clean value is returned rather than rebuilt — see the memory note under Guarantees. And a clean string is scanned without decoding a rune or allocating anything, so the common case of text that was already safe is close to free.

What remains is dominated by toon.Marshal itself, not by this layer.

These figures are enforced, not merely recorded. Every pull request compiles the base commit and the branch, measures them alternately on one runner, and fails if a benchmark regresses: wall time by more than 15%, allocations by any amount at all. The gate is .github/benchgate.sh.

It compares two commits rather than checking against a stored baseline, because the numbers above are Apple M5 figures that a shared CI fleet cannot reproduce — runner hardware rotates, so an absolute threshold recorded on one generation fails on the next for reasons that have nothing to do with the change under test. What CI can reproduce is the difference between two commits measured minutes apart on the same machine. Allocations carry the stricter bound because they are deterministic: every allocs/op measurement in this suite has a confidence interval of 0%, so a delta is always real. Wall time carries the looser one because it is not — spread reaches 21% on EncodeOrJSON even on an idle machine, and a tighter bound would buy flakes rather than sensitivity.

What this adds to toon-go

toon-go is a good codec. Use it directly for encoding, and Decode here forwards straight to it for reading. These are the gaps a production CLI hits:

Gap in toon-go	What happens without a guard	go-axi
Passes U+2028/U+2029, invalid UTF-8, lone C1 bytes (0x9B, 0x9D) and Unicode format characters (bidi controls, zero-width and tag characters) through	A payload carrying a raw escape sequence reaches whatever terminal renders it, and a bidi override can reorder what a reader sees (Trojan Source)	Sanitize
Supports neither defined string types nor encoding.TextMarshaler, and does not error on them	Empty output. The command prints nothing and exits 0	Check
Reference cycles reach the encoder	Stack exhaustion kills the process; recover() cannot catch it	Sanitize, Check
Size relative to JSON is not reported	A command pays more tokens as TOON than it would as JSON	Check.Efficient
Exit codes and help[] formatting are out of scope	Every tool invents its own, and they drift	Exported constants, WriteHelp
A tabular array's field order, delimiter and declared row count are unreachable — parsedHeader is unexported — and a paginated payload fails the strict pass outright	A reader hardcodes |, keys consumers on quoted field names, or cannot read paginated output at all	DecodeTabular

The help[] form is the inline TOON array, because it is the only one in circulation that survives its own codec: the AXI specification's indented example fails to decode with "list length mismatch", and the help[] line form fails with "missing colon after key".

AXI principle coverage

go-axi is a library, not an AXI tool itself. It implements the output-layer principles so each CLI does not reimplement them:

AXI principle	Provided by
1 — Token-efficient output	Encode, plus Check.Efficient and Check.Tier to prove it
5 — Definitive empty states	Check distinguishes a legitimately empty result from encoder failure
6 — Structured errors and exit codes	ExitCode constants; typed Sanitize errors
9 — Contextual disclosure	WriteHelp

Principles 2, 3, 4, 7, 8 and 10 are per-command design decisions and stay with the tool.

Guarantees

• Round-trip is a build failure. Decode(Encode(Sanitize(v))) must equal Sanitize(v). Codec pairs tested only against frozen fixtures go stale in silence.
• One write per payload. A body and its terminating newline land in a single Write call, so a closed pipe cannot deliver a payload missing its last byte.
• Nothing on failure. Encode writes zero bytes when encoding fails, and EncodeChecked writes zero bytes unless the verdict is OK.
• One pass per payload. EncodeChecked and EncodeOrJSON sanitize once and marshal once, deriving their verdict from the bytes they write rather than from a second encode.
• Input is never mutated. Sanitize never writes to the value you hand it. It does not promise a distinct copy: nothing is allocated unless a string actually needs cleaning, so for a clean value the result shares memory with the input. Nil containers, scalars, funcs, channels and unexported struct fields were always shared this way; a populated map, slice or struct that needed no cleaning now is too. Two rules follow. Do not mutate your own value afterwards and expect the returned one to stay unchanged — there is no signal that the two are the same value. And do not mutate it from another goroutine while Sanitize, Encode, EncodeChecked or EncodeOrJSON is running: a clean payload is marshalled from your map rather than from a private snapshot, and a concurrent write during that window is a fatal concurrent map read and map write that recover() cannot catch. Rebuilding unconditionally used to hide that race behind a copy; it never made the calling code safe, because encoding/json carries the identical hazard.
• No output means no output. An empty body and an empty help list write nothing, not a blank line an agent pays tokens to read.

Development

go test ./... -race -cover
go vet ./... && gofmt -s -l .

To run the benchmark gate locally the way CI runs it — compile both sides, then measure them alternately:

go install golang.org/x/perf/cmd/benchstat@v0.0.0-20260908200009-22c9c6c9d4da

git worktree add /tmp/go-axi-base main
(cd /tmp/go-axi-base && go test -c -o /tmp/base.test .)
go test -c -o /tmp/head.test .

: > /tmp/base.txt; : > /tmp/head.txt
for i in 1 2 3 4 5 6; do
  /tmp/base.test -test.run='^$' -test.bench=. -test.benchtime=100ms -test.count=1 >> /tmp/base.txt
  /tmp/head.test -test.run='^$' -test.bench=. -test.benchtime=100ms -test.count=1 >> /tmp/head.txt
done

bash .github/benchgate.sh /tmp/base.txt /tmp/head.txt
git worktree remove /tmp/go-axi-base

Alternating is the point, not a detail. Measuring all of one side and then all of the other puts each side in its own contiguous block, so any interference — a co-tenant VM, a throttle — skews every benchmark in whichever block it overlapped, and does so all at once. That is what a batch of benchmarks failing together means. Alternating denies interference the chance to land on one side only.

count=6 is the floor, not a preference: benchstat refuses a confidence interval below six samples and will not call a difference significant below four. Six rounds of -test.count=1 give the same six samples per side at the same total cost. Both thresholds are overridable for a one-off investigation — TIME_THRESHOLD=5 ALLOC_THRESHOLD=0 bash .github/benchgate.sh ... — but CI always uses the defaults.

License

MIT

Documentation

Overview

Package goaxi is a hardening and guard layer over the official TOON codec, github.com/toon-format/toon-go.

It is deliberately NOT a codec. toon-go encodes and decodes TOON; this package adds the five things toon-go does not cover, which an agent-facing CLI has to get right before it prints or reads:

• Sanitize: hostile input handling. toon-go rejects control bytes below 0x20 with an error but passes U+2028/U+2029, invalid UTF-8, lone C1 bytes and Unicode format characters (bidi controls, zero-width and tag characters) straight through. A payload carrying a raw ANSI escape reaches the terminal that renders it, and one carrying a bidi override can reorder what a reader sees.

• Check: toon-go supports neither defined string types nor encoding.TextMarshaler, and a violating type does not error — it emits EMPTY output. A command silently prints nothing. Check turns that into a detectable condition, and additionally reports when TOON is larger than JSON for a given shape. EncodeChecked applies that guard on the way out in a single pass, for callers that are about to print.

• Exit codes: one canonical set, as exported constants rather than a rule written in a style guide.

• Help blocks: one canonical writer for AXI contextual-disclosure lines.

• Tabular reading: toon-go decodes a whole document in one strict pass and returns a map, which drops the three things a tabular array declares about itself. Its parsedHeader type is unexported, so the field ORDER, the delimiter and the declared row count are all unreachable from the public API — and a paginated payload, which declares more rows than it carries on purpose, fails the whole document. DecodeTabular reads the header itself and returns those three. Decode is the ordinary whole- document read and forwards to toon-go, which keeps one decoder rather than two.

The constants-over-prose choice is deliberate. A documented convention drifts; a compiled constant cannot.

Index

• Constants
• func CanEncode(v any) bool
• func Decode(r io.Reader) (any, error)
• func DecodeFile(path string) (any, error)
• func Encode(w io.Writer, v any) error
• func EncodeOrJSON(w io.Writer, v any) error
• func IsTabularHeader(line string) bool
• func MustSanitize(v any) any
• func Sanitize(v any) (any, error)
• func SanitizeString(s string) string
• func WriteHelp(w io.Writer, lines []string) error
• type CycleError
  • func (e *CycleError) Error() string
• type Document
  • func DecodeTabular(r io.Reader) (*Document, error)
  • func DecodeTabularFile(path string) (*Document, error)
• type ExitCode
  • func (c ExitCode) String() string
• type Format
  • func ParseFormat(s string) (Format, error)
  • func (f Format) Encode(w io.Writer, v any) error
  • func (f Format) EncodeProjected(w io.Writer, v any) error
• type KeyCollisionError
  • func (e *KeyCollisionError) Error() string
• type TagVerdict
  • func CheckTags(v any) TagVerdict
  • func (v TagVerdict) String() string
• type Tier
  • func (t Tier) String() string
• type Verdict
  • func Check(v any) Verdict
  • func CheckSanitized(v any, clean any) Verdict
  • func EncodeChecked(w io.Writer, v any) (Verdict, error)

Constants

const Default = TOON

Default is the format used when the caller expresses no preference.

const MaxDocumentBytes = 64 * 1024 * 1024

MaxDocumentBytes bounds a whole document read by Decode.

toon-go's decoder takes a []byte, so the document has to be resident — but "resident" is not "unbounded". Decode accepts any io.Reader, including one whose size cannot be checked first, and an uncapped read there is a way to exhaust memory before a single byte is parsed. 64 MiB is far above any real findings payload (hundreds of rows of KB-scale text) and far below trouble.

Functions

func CanEncode

func CanEncode(v any) bool

CanEncode reports whether v survives TOON encoding without losing data. It is the boolean form of Check for callers that do not need the detail.

func Decode

func Decode(r io.Reader) (any, error)

Decode reads a TOON document of any shape, preserving toon-go's types.

DecodeTabular is tabular-only by contract and rejects anything else, which is correct for the findings payloads it was written for and useless for the rest. `llm-filesystem-axi get-file-info` emits a flat object of scalars with no array anywhere, and `list-directory` emits an array with a `total` sibling — the first would not decode at all and the second silently dropped the total.

Types are preserved rather than projected to strings: this is a new entry point with no existing consumers, and a size that stays a number is more useful than one that becomes text.

func DecodeFile

func DecodeFile(path string) (any, error)

DecodeFile reads a TOON document of any shape from a file.

func Encode

func Encode(w io.Writer, v any) error

Encode sanitizes v and writes it as TOON, terminated by exactly one newline.

Sanitizing here rather than leaving it to the caller is deliberate. The one time a caller forgets is the time a control byte reaches a terminal, and a control byte is exactly what arrives when a tool echoes back text it did not author.

Nothing is written when encoding fails. A partial payload is worse than no payload, because it parses.

func EncodeOrJSON

func EncodeOrJSON(w io.Writer, v any) error

EncodeOrJSON writes v as TOON when that is lossless, and otherwise falls back to a self-describing JSON envelope naming the reason.

The fallback is triggered only by data loss, never by size. Routing on cost would make a command's output format vary with payload length, which no consumer could predict and every consumer would have to handle both ways.

This is runtime format detection. A project that fixes the output format per command at design time should keep its build-time guard and not call this. It exists for callers that have no such rule.

func IsTabularHeader

func IsTabularHeader(line string) bool

IsTabularHeader reports whether line has the SHAPE of a TOON tabular array header: a name, a bracketed count, and a trailing colon.

It is the dispatch point between DecodeTabular and Decode, and it deliberately does NOT validate. Asking parseHeader instead — "does this header parse" — makes a BROKEN tabular header indistinguishable from a line that was never a tabular header, and the caller then reroutes it to the document reader. Observed, through the binary rather than the package:

$ llm-support toon parse f.axi          # f[1|]{a|b|a}: / 1|2|3
{"shape":"document","value":{"f":[{"a":3,"b":2}]}}          exit 0

A duplicate column is an error at the package level and there is a test proving it, but the payload never reached that error. toon-go kept one of the two `a` columns, last value wins, the 1 was gone, and the command reported success. Dropping a column in silence is the precise failure this reader exists to remove, so a line that LOOKS tabular stays on the tabular path and gets to report its real problem.

The shape test cannot swallow a document: an inline array (`tags[2]: x,y`) does not end in a colon, and a scalar or nested key has no bracket at all.

NOT the same test as the unexported isTabularHeader in check.go, and the two must not be merged. That one asks "does this WHOLE document contain a header with a braced field list", over []byte, to set Verdict.Tier on output that go-axi itself just encoded. This one asks "does this ONE line open a tabular array", over a string, to route a line the caller is about to read. The braced field list is the difference that matters: `findings[0]:` is a legitimate empty array here and is deliberately not a tabular header there, because an empty array is not what the tier is describing.

func MustSanitize

func MustSanitize(v any) any

MustSanitize is Sanitize for callers whose map keys are fixed identifiers and whose shapes are acyclic, where both failures are impossible by construction. It panics if one occurs, because continuing would mean emitting a payload with a field missing.

A panic is the lesser evil for a cycle specifically: it unwinds and can be recovered, whereas the stack overflow it replaces is fatal and cannot.

func Sanitize

func Sanitize(v any) (any, error)

Sanitize returns v with every string cleaned of characters that would either break the TOON encoder or reach stdout as a raw control byte.

Memory

The input is never MUTATED, and that is the guarantee callers rely on. What is NOT promised is that the result occupies different memory: nothing is allocated unless a string actually needed cleaning, so for a clean value the result IS v. This was already true before the copy became conditional — nil containers, scalars, funcs, channels and every unexported struct field were always passed through by reference.

Two rules follow, and a caller needs both:

• Do not mutate your own value after calling Sanitize and expect the returned value to stay as it was. There is no compile-time or runtime signal that the two are the same value.

• Do not mutate it CONCURRENTLY while this call runs, or while the result is being encoded. Encode, EncodeChecked and EncodeOrJSON pass the result straight to toon.Marshal, so a clean payload is marshalled from the caller's own map rather than from a private snapshot. A concurrent write during that window is a fatal "concurrent map read and map write" that recover() cannot catch. Rebuilding unconditionally used to hide that race behind a copy; it never made the caller's code safe, because encoding/json and every other reflective marshaller carry the identical hazard.

Why it exists

It exists because toon-go mishandles hostile input in two opposite directions, both confirmed against the pinned version rather than assumed:

• A raw ANSI escape (\x1b) makes toon.Marshal return an error, so a command handed reviewer text containing colour codes emits nothing at all.
• U+2028, U+2029, lone C1 bytes (0x9b, 0x9d) and invalid UTF-8 pass through untouched, reaching whatever terminal renders the output.
• Unicode format characters (category Cf) pass through too. Bidi controls and zero-width characters let text reorder or hide itself when shown in a terminal, editor or diff, and tag characters carry text a human cannot see.

Sanitizing before encoding fixes both: the encoder never sees a byte it would reject, and never sees one it would forward.

Characters are STRIPPED rather than replaced, so text on either side of a removed byte joins contiguously. Substituting a space would silently alter content; a reader cannot tell an inserted space from a real one.

Tab, carriage return and newline are deliberately preserved. toon-go already escapes those correctly, and removing them would corrupt multi-line content.

Concrete types are preserved, so struct tags survive. That matters more than it looks: TOON field names are part of the contract consuming tools parse, and flattening a struct to map[string]any would rename every column to its Go identifier. Strings inside unexported fields cannot be reached by reflection and are carried through as-is.

Two errors are returned: *CycleError for a self-referential value, and *KeyCollisionError for two map keys that clean to the same string. A caller whose keys are fixed identifiers and whose shapes are acyclic can rule both out and use MustSanitize.

func SanitizeString

func SanitizeString(s string) string

SanitizeString cleans a single string using the same rules as Sanitize. It is exported for callers that build output a field at a time rather than handing over a whole value. It cannot collide, so it returns no error.

func WriteHelp

func WriteHelp(w io.Writer, lines []string) error

WriteHelp emits an AXI contextual-disclosure block: next-step command templates with fixed flags carried forward and runtime values left as placeholders like <id>.

The form is the inline TOON array, `help[2]: first,second`, chosen because it is the only one of the three in circulation that survives its own codec. The AXI specification's own indented example fails to decode with "list length mismatch", and the `help[] line` form fails with "missing colon after key". Appending this after a body is safe: the combined payload decodes as one value carrying both.

An empty list writes nothing. A command with no meaningful next step should not emit a stray "help[0]:" that an agent pays tokens to read and learns nothing from.

Types

type CycleError

type CycleError struct {
	Type string // the type the cycle was detected in
}

CycleError reports a value that refers to itself, directly or through other nodes.

TOON has no way to express a back-reference, so there is no correct encoding to fall back to. The walk is refused before it is entered rather than bounded part way in: sanitizeValue recurses through pointers with no seen-set, and the resulting stack overflow is a fatal runtime error, not a panic. No recover() catches it, so the process dies with a goroutine dump instead of returning to the caller — the exact class of failure this module exists to convert into an error.

Check already refuses a cycle for this reason. Sanitize now agrees, so the guarantee does not depend on which entry point a caller happened to reach for.

func (*CycleError) Error

func (e *CycleError) Error() string

Error names the type the cycle was found in, and says why the value is refused rather than encoded.

type Document

type Document struct {
	Name      string              // the array name, e.g. "findings"
	Fields    []string            // declared field names, in order, unquoted
	Delimiter rune                // read from the header, never assumed
	Rows      []map[string]string // one map per row, keyed by field name

	// Declared is the row count the HEADER states. It is not always len(Rows):
	// atcr's paginated payload caps the emitted rows while keeping the true
	// total in the header, and says so with a `truncated` sibling.
	Declared int

	// Meta holds `key: value` sibling lines that follow the array at column 0 —
	// `truncated: false` is the one atcr emits today.
	Meta map[string]string
}

Document is one decoded tabular array.

func DecodeTabular

func DecodeTabular(r io.Reader) (*Document, error)

DecodeTabular reads one TOON tabular array from r.

func DecodeTabularFile

func DecodeTabularFile(path string) (*Document, error)

DecodeTabularFile reads one TOON tabular array from a file, naming it on error.

type ExitCode

type ExitCode int

ExitCode is the process exit status an agent-facing CLI returns.

These ship as constants rather than as a rule in a style guide because a documented convention drifts and a compiled constant cannot. The set extends the conventional 0/1/2 with two codes agent-facing tools need to distinguish.

const (
	// ExitOK is success.
	ExitOK ExitCode = 0

	// ExitError is a runtime failure inside the tool itself.
	ExitError ExitCode = 1

	// ExitUsage is a malformed invocation: an unknown subcommand or flag.
	//
	// AXI requires an unknown flag to fail loudly rather than be ignored. An
	// agent that invents a flag has to learn it did nothing, or it will trust
	// output that was never scoped the way it asked.
	ExitUsage ExitCode = 2

	// ExitNotFound is a path or item that does not exist.
	ExitNotFound ExitCode = 3

	// ExitValidation means the command ran correctly and the thing it checked
	// did not pass.
	//
	// This is deliberately distinct from ExitError, and it is the load-bearing
	// distinction in the set. A checker reporting "this does not conform" is a
	// successful run with a real result, not a broken tool. Collapsing the two
	// makes "the tool crashed" indistinguishable from "your input is wrong",
	// and an agent cannot tell whether retrying is worth anything.
	ExitValidation ExitCode = 4
)

func (ExitCode) String

func (c ExitCode) String() string

String renders the code for error messages and logs. A bare integer in a message tells an operator nothing.

type Format

type Format string

Format is an output encoding chosen by the caller.

The choice is explicit rather than detected. Output shape is a property of the command, not of the data: a format that varied per call would force every consumer to handle both, and would make an instruction like "the third column is SEVERITY" unsafe to write down. EncodeOrJSON is the opposite policy and exists for callers that have no design-time rule.

const (
	// TOON is the token-efficient default of AXI principle 1.
	TOON Format = "toon"

	// JSON is the escape hatch for genuinely nested or non-uniform output.
	JSON Format = "json"
)

func ParseFormat

func ParseFormat(s string) (Format, error)

ParseFormat maps a flag value to a Format.

An empty string selects Default, so an unset flag can be passed straight through without the caller special-casing it. Matching is case-insensitive and ignores surrounding spaces, because the value usually arrives from a shell.

func (Format) Encode

func (f Format) Encode(w io.Writer, v any) error

Encode writes v to w in the receiver's format, terminated by exactly one newline, in a single Write call.

TOON is sanitized first, which is load-bearing rather than cosmetic: toon-go errors on a raw ANSI escape instead of emitting it, so without sanitizing, a command handed coloured text would fail outright on data it should simply print. JSON is marshalled as-is; encoding/json escapes control bytes itself.

Field names come from struct tags — `toon` for TOON and `json` for JSON. toon-go does NOT fall back to the `json` tag, so a type carrying only `json` tags publishes its Go identifiers here. Use CheckTags to catch that in a test, or EncodeProjected to encode from the `json` tags instead.

Nothing is written when encoding fails, including on an unknown format. A partial payload is worse than none, because it parses.

func (Format) EncodeProjected

func (f Format) EncodeProjected(w io.Writer, v any) error

EncodeProjected writes v the way Encode does, but derives TOON field names from the `json` struct tags rather than requiring a second `toon` tag.

WHY THIS EXISTS. toon-go does not read the `json` tag. A codebase whose result types carry only `json` tags therefore has two options: add a duplicate `toon` tag to every field, or route the value through its JSON form first. The first is what cadence-axi does — 102 tags, maintained by hand, where a single omission silently publishes a Go identifier. The second is this function.

It is not free. Medians of six runs on a 500-row payload: 257us and 5,427 allocations direct, against 665us and 15,055 projected — about 2.6x the time and 2.8x the allocations, because the value is marshalled and unmarshalled once more. Encode remains the fast path and the default; this is the convenience. Reproduce with:

go test -run '^$' -bench Format_Encode -benchmem

Projection applies to TOON only. encoding/json already reads the `json` tag, so a JSON payload has nothing to correct and is passed to Encode unchanged — routing it through a generic map would also reorder its keys, since struct fields marshal in declaration order and maps in sorted order.

Going through JSON additionally normalises values to the shapes toon-go handles, so a type that marshals specially — time.Time, a json.Marshaler — reaches the encoder looking the way JSON output would have shown it.

type KeyCollisionError

type KeyCollisionError struct {
	Cleaned any // the key both inputs produced
	First   any // the key already present
	Second  any // the key that would have overwritten it
}

KeyCollisionError reports two map keys that became identical after cleaning.

This is the one case where sanitizing cannot proceed without losing data. "na\x1bme" and "name" are different keys on input and the same key after the control byte is removed, so one would overwrite the other. Go randomizes map iteration order, which makes the survivor vary between runs — the worst kind of loss, because a test can pass today and fail tomorrow with no code change.

Sanitize refuses rather than picking. Renaming one key would invent data the caller never wrote, and dropping one is the silent loss this module exists to prevent.

func (*KeyCollisionError) Error

func (e *KeyCollisionError) Error() string

Error quotes both original keys with %#v, so the invisible byte that made them collide is readable in a terminal that would not render it.

type TagVerdict

type TagVerdict struct {
	// OK is true when no field would be published under its Go identifier.
	OK bool

	// Fields names each field that would be, as "Type.Field".
	Fields []string
}

TagVerdict reports whether encoding a value as TOON would publish Go identifiers as its field names.

OK is true when every encodable field carries a `toon` tag. Fields names the ones that do not, as "Type.Field", in the order they were found.

func CheckTags

func CheckTags(v any) TagVerdict

CheckTags reports whether v's type is safe to hand to Format.Encode as TOON.

WHY THIS EXISTS. toon-go does not fall back to the `json` struct tag. Probed against the pinned version: a struct carrying only `json` tags encodes to its Go identifiers, so a field whose json tag reads "count" still emits "Count: 3" rather than "count: 3". Nothing errors and nothing is empty, so neither Check nor a smoke test catches it. The output is simply keyed on names no consumer was told to expect, and a skill instructed that "the third column is SEVERITY" reads the wrong column.

That is the same defect class Check was built for — a codec failing quietly rather than loudly — which is why the guard lives here rather than in each tool. Call it from a test over every type a command prints:

if v := goaxi.CheckTags(SearchResult{}); !v.OK {
    t.Error(v.String())
}

A value that is already generic — a map, a slice, a scalar, or anything Format.EncodeProjected produced — carries no Go identifiers and is always OK. Unexported fields are never encoded, and a field marked `json:"-"` is never reached, so neither is flagged.

func (TagVerdict) String

func (v TagVerdict) String() string

String renders the verdict as a message suitable for a failing test or a build-time guard. It returns the empty string when the verdict is OK.

Deliberately not named Error. A method of that name would make TagVerdict satisfy the error interface, so `var err error = verdict` would compile and never be nil — including for a passing verdict.

type Tier

type Tier int

Tier describes the payload shape TOON encoding actually produces.

The tiers are defined by measured behavior, not by what a format "should" handle. toon-go encodes ragged rows, list-valued fields and nested objects losslessly using a list form, so "can it be encoded" separates nothing useful. What differs is the shape emitted, and whether anything is lost.

const (
	// TierTabular is TOON's best case: uniform rows that encode as a tabular
	// array with a declared field list. This is where the token savings come
	// from.
	TierTabular Tier = iota

	// TierNested is lossless but not tabular. Ragged rows, list-valued fields
	// and nested objects land here. The payload is correct; it just does not
	// get the columnar win.
	TierNested

	// TierLossy means encoding silently discards data. Nothing in this tier
	// should be emitted as TOON.
	TierLossy
)

func (Tier) String

func (t Tier) String() string

String names the tier for logs and test failures. It is not part of any output format: consumers switch on the Tier constant, never on this text.

type Verdict

type Verdict struct {
	// OK reports that encoding this value as TOON preserves its data.
	OK bool

	// Tier is the shape TOON encoding produces.
	Tier Tier

	// Efficient reports that the TOON payload is no larger than the JSON one.
	// Only meaningful when SizeCompared is true.
	Efficient bool

	// SizeCompared reports whether Efficient was actually measured. The
	// comparison needs a second marshal, so a path that writes the payload and
	// never reads the cost signal skips it. Without this flag a skipped
	// comparison is indistinguishable from a measured "TOON is larger".
	SizeCompared bool

	// Reason explains a false OK or Efficient in terms an operator can act on.
	Reason string
}

Verdict is the result of inspecting a value before encoding it.

OK and Efficient are deliberately separate. Correctness and cost are orthogonal: a payload can be perfectly lossless and still cost more as TOON than as JSON, which is a reason to choose JSON for that command but never a reason to call the value broken.

func Check

func Check(v any) Verdict

Check reports whether v can be encoded as TOON without losing data, what shape the payload takes, and whether TOON actually costs less than JSON.

It exists because toon-go fails in ways neither an error check nor a length check catches, all confirmed against the pinned version:

• A defined string type (type Kind string) returns a real error. Loud.
• A TextMarshaler at top level returns a nil error with EMPTY output, so the command prints nothing and exits zero.
• A TextMarshaler nested in a struct or map encodes to "m:" — key present, value gone, nil error, non-zero length. A guard testing only err and len(b) passes this while the data is lost.

Loss is found by reflection rather than by round-tripping and comparing against the JSON projection. That comparison looks correct and is not: toon-go does not fall back to the json tag, so a field tagged only `json:"x"` encodes under its Go field name and the two projections disagree for reasons that have nothing to do with data loss.

func CheckSanitized

func CheckSanitized(v any, clean any) Verdict

CheckSanitized is Check for a value that has already been sanitized, so a caller holding the cleaned copy does not pay to sanitize it twice.

v supplies the declared types for the loss walks; clean is what gets measured. Passing both is deliberate — sanitizing preserves concrete types, so the walks give the same answer either way, but the measurements must be taken on what will actually be emitted.

func EncodeChecked

func EncodeChecked(w io.Writer, v any) (Verdict, error)

EncodeChecked sanitizes v, verifies TOON can carry it losslessly, and writes it — one sanitize walk and one TOON marshal serving both jobs.

Use this instead of Check followed by Encode. That pair sanitizes and marshals the same value twice to serve a single guard. Measured medians on the same payload, against a bare Encode as the floor:

rows   Encode    EncodeChecked   Check+Encode
100    134us     156us (+16%)    338us (+152%)
2000   2.91ms    3.32ms (+14%)   7.18ms (+147%)

Allocations at 2000 rows: 98,073 / 110,099 / 222,204. The guard costs about 14%; the rest was duplicated work.

Nothing is written unless the verdict is OK, because a partial payload is worse than none — it parses. The returned Verdict describes the bytes actually written, so Tier is measured rather than predicted.

Efficient is deliberately NOT measured; SizeCompared reports that. The comparison needs a second marshal that a caller writing the payload does not read. Call Check when you want the cost signal.