whittle

README

whittle

Carves your agent's tool outputs down to what matters. Never cuts what doesn't come back.

Whittle is a content-aware compressor for the text AI agents read: tool outputs, file reads, logs, JSON, terminal streams. Long agent sessions drown in tokens — but most compressors buy their ratio by silently destroying things agents need: array rows vanish, file reads get gutted, identifiers come back mangled.

Whittle holds one hard line: structural compression is lossless or clearly marked, code never reaches a lossy model, and every anomaly fails open to the original bytes. The reduction number it reports is calibrated to real tokenizer counts — not byte counts that overstate savings by up to 4×.

Install

go install github.com/firstops-dev/whittle/cmd/whittle@latest
whittle setup

That's the whole thing. setup:

• installs the Claude Code PostToolUse hook — every tool output your agent reads is whittled from now on (Claude Code is the supported agent today; Cursor, Codex and OpenCode adapters are on the roadmap);
• materializes the ML prose sidecar (embedded in the binary) into ~/.whittle, builds its venv, and uses your GPU automatically (CUDA > Apple MPS > CPU) — if python3 is missing, whittle simply runs deterministic-only;
• registers a launchd agent (macOS) so the service starts at login and is kept alive.

Manage it with whittle status, whittle stop, and whittle cleanup (stops the service and removes the hook). Everything fails open: if whittle is down, your agent sees original outputs, never an error.

Use

whittle compress output.json                 # compressed to stdout
cat build.log | whittle compress -stats      # stats to stderr
whittle serve -addr :45871                    # HTTP: POST /v1/compress

As a library:

eng := whittle.New(whittle.Options{})
res := eng.Compress(ctx, toolOutput)
// res.Output, res.Action ("compressed"|"skipped"), res.SkipReason,
// res.Strategy, res.Detected

What it does per content type

detected	strategy	guarantee
JSON	minify + columnar reshape (union schema, typed CSV, nested flattening, constant factoring)	lossless — reconstructs byte-exact; rows are never dropped
logs / build output	keep errors, warnings, stack traces, summaries	lossy but marked — ... [N lines omitted], exact accounting
terminal	ANSI strip + CR-overwrite collapse (progress bars → final frame)	what the terminal actually displayed; rune-safe
markdown file reads	structure-aware: prose compressed by the model, code fences / tables / lists / headings passed through byte-exact	code never reaches the model
source code	untouched	routed away from every lossy path
prose	extractive model (optional) with fidelity guards: entity protection, whole-token deletion, negation preservation	fails open on any guard trip

Every path is wrapped in fail-open guardrails: empty-output, expansion (both byte- and token-honest), panic recovery. The worst case is always "not compressed", never "corrupted".

Optional: the ML prose path

Deterministic strategies need nothing. To also compress natural-language prose, run the model sidecar (LLMLingua-2 + whittle's fidelity guards — see model/):

cd model && python -m venv .venv && .venv/bin/pip install -r requirements.txt
.venv/bin/uvicorn app:app --port 45872
export WHITTLE_MODEL_URL=http://127.0.0.1:45872

Configuration

env	default	meaning
WHITTLE_MODEL_URL	(unset — prose off)	model sidecar URL
WHITTLE_MAX_CHARS	262144	global size ceiling (skip before classify)
WHITTLE_PROSE_MAX_CHARS	4500	prose-path latency ceiling

Why whittle — compress at write-time, not read-time

Context compressors typically integrate with coding agents as request-path proxies: your agent's base URL is redirected through a local server that rewrites conversation history at read time, on every LLM call. That position forces hard problems — prompt-cache stabilization (mutating history invalidates cached prefixes), per-call re-compression, terminating your API traffic (keys, system prompts and all), and a resident runtime that must stay up or your agent goes down with it. It also makes lossy compression the default, backed by a retrieval loop: the runtime is guaranteed present, so dropped content can be served back on demand.

Whittle takes the other position: it is a PostToolUse hook. Each tool output is compressed once, at the moment it is born, before it ever enters conversation history. Everything else follows from that choice:

• Savings compound. A tool output lives in context for every subsequent turn. Tokens removed at write-time are removed from every later call — no per-call rework, no cache surgery, because history is never mutated.
• No trust expansion. A hook sees one tool output at a time, locally, with zero credentials. Nothing terminates your API traffic.
• Failure is free. The hook fails open; if whittle is down or declines, the agent proceeds with the original output. A gateway outage is an agent outage.
• The loss budget is honest. A read-time proxy can afford recoverable lossy compression — its resident runtime serves dropped content back when the model asks. Whittle keeps no runtime in your request path; reduced outputs carry a tiny retrieval pointer served by the local daemon (whittle_get), and lossless transforms carry nothing at all — lossless-or-marked stays the construction, recovery is the safety net, never the license.

The hook is whittle's default surface, not its only one: library (whittle.New) → HTTP service (whittle serve) → hook adapters (Claude Code PostToolUse today; Cursor, Codex, OpenCode adapters on the roadmap) — and the same library embeds in gateways or pipelines if that is where you need it. The position is the point: compression happens where output is born, whatever surface delivers it there.

Performance

Deterministic strategies are pure CPU, single static binary, zero allocatable model state (Apple M-series, go test -bench):

input	size	latency
JSON array, 200 rows, pretty-printed	~21 KB	~1.0 ms
terminal progress stream	~12 KB	~3.9 ms
build log, 800 lines	~66 KB	~21 ms

The hook runs after the tool call completes, so this cost is off the LLM request path entirely — model-call latency is unchanged. The optional ML prose path is capped by a fail-open budget (default 1.5 s) and never blocks beyond it.

Design principles

1. Fail open. A compressor that breaks your agent is worse than no compressor.
2. Never silent loss. Lossy paths mark what they removed and account for it exactly.
3. Code is sacred. File reads, fences, identifiers: byte-exact or untouched.
4. Token-honest. Accept gates and reported savings use calibrated token estimates (MAE ~8% vs o200k_base), not bytes.
5. Adversarially tested. The invariants above are pinned by reconstruction fuzzing, per-language routing suites, and fail-open contract tests.

Acknowledgments

Whittle's log-selection strategy, several content-detection heuristics, and the tabular parser were adapted from Headroom (Apache-2.0) — adapted portions are marked in source comments, and we think their compaction work is excellent. Whittle exists because we wanted the other position: a write-time PostToolUse hook instead of a read-time request-path proxy, with the stricter fidelity contract that position requires. See NOTICE.

License

Apache-2.0.

Documentation

Overview

Package whittle carves agent tool outputs down to what matters — and never cuts what doesn't come back.

Whittle is a content-aware compressor for the text an AI agent reads: tool outputs, file reads, logs, JSON, terminal streams. It routes each input to a type-specific strategy and holds one hard line: structural compression is LOSSLESS or CLEARLY MARKED, code never reaches a lossy model, and any anomaly fails open to the original bytes.

The deterministic strategies (JSON columnar reshaping, log selection with omission markers, terminal CR-overwrite collapse, ANSI stripping) work with no dependencies. The optional ML prose path (LLMLingua-2, see model/) activates only when Options.ModelURL (or WHITTLE_MODEL_URL) points at a running model sidecar.

eng := whittle.New(whittle.Options{})
res := eng.Compress(ctx, toolOutput)
if res.Action == "compressed" {
    use(res.Output) // res.SkipReason explains any skip
}

Index

• Variables
• func EstimateTokens(s string) int
• type Engine
  • func New(opts Options) *Engine
  • func (e *Engine) Compress(ctx context.Context, content string) Result
  • func (e *Engine) CompressInput(ctx context.Context, in compress.Input) Result
• type Options
• type Result

Variables

var SidecarFS embed.FS

SidecarFS carries the Python model sidecar inside the binary so that `go install .../cmd/whittle@latest` is a complete installation: `whittle setup` materializes these files to ~/.whittle/model and builds a venv there.

Functions

func EstimateTokens

func EstimateTokens(s string) int

EstimateTokens is whittle's calibrated token estimator (MAE ~8% vs tiktoken o200k_base) — the same accounting the engine's accept gates use.

Types

type Engine

type Engine struct {
	// contains filtered or unexported fields
}

Engine is a reusable, concurrency-safe compressor.

func New

func New(opts Options) *Engine

New builds an Engine from opts.

func (*Engine) Compress

func (e *Engine) Compress(ctx context.Context, content string) Result

Compress routes content to its type-specific strategy. It never returns an error and never corrupts: every failure or guard trip returns the original content with Action "skipped" and a reason.

func (*Engine) CompressInput

func (e *Engine) CompressInput(ctx context.Context, in compress.Input) Result

CompressInput is the full-control variant (routing override, tool-name / MIME gating hints, prose keep-rate).

type Options

type Options struct {
	// ModelURL enables the ML prose path (an LLMLingua sidecar, see model/).
	// Empty = prose and doc-read content pass through untouched.
	ModelURL string
	// MinTokens skips inputs shorter than this (default 64; compression isn't
	// worth it below that). Use -1 to keep the default; 0 disables the floor.
	MinTokens int
	// MaxChars is the global safety ceiling; larger inputs skip before
	// classification (default 256 KiB).
	MaxChars int
	// ProseMaxChars caps only the ML prose path (default 4500 — a latency
	// budget, not a correctness bound).
	ProseMaxChars int
	// Logger receives pipeline diagnostics (nil = silent).
	Logger compress.Logger
}

Options configures an Engine. The zero value is production-safe: deterministic compressors only, default gates.

type Result

type Result = compress.Outcome

Result is the outcome of one compression: Output (compressed or original), Action ("compressed"|"skipped"), SkipReason, Strategy, Detected type, sizes.