whittle

README

🪓 whittle

Cut your AI agent's token bill. Lose nothing that matters.

Install · How it works · Compression · Model routing · Benchmarks · Why whittle?

Long agent sessions drown in tokens, and most compressors buy their ratio by silently destroying what agents need: array rows vanish, file reads get gutted, identifiers come back mangled. Whittle holds one line: lossless or clearly marked, code never touched, every anomaly fails open to the original bytes.

Built for engineers running long Claude Code sessions who refuse to trade fidelity for token savings.

Highlights

• 🪓 Write-time compression: a Claude Code PostToolUse hook whittles each tool output before it enters context, so the savings repeat on every later turn. History is never mutated, so prompt caches stay intact.
• 🔒 Lossless or marked: JSON reshapes byte-exact (rows are never dropped), logs keep every error plus an honest [N lines omitted], source code passes through untouched.
• 🎯 Token-honest numbers: savings measured against calibrated tokenizer counts, not byte counts that overstate by up to 4×.
• 🧭 Opt-in model router: hard reasoning stays on your strongest model, trivia drops to the cheapest, per one auditable policy file driven by trained-classifier signals (how it works).
• 🛟 Fail-open, local, yours: if whittle is down your agent runs on originals, a rejected rewrite retries your original request, zero credentials leave your box, and the Go binary has zero external dependencies.

Install

npm install -g @firstops/whittle   # or: go install github.com/firstops-dev/whittle/cmd/whittle@latest
whittle setup                      # hook + local daemon + optional ML sidecar, one command

Tool outputs are whittled from now on; whittle stats shows what you're saving. Try it with zero commitment: npx -y @firstops/whittle compress build.log. (Homebrew: brew install firstops-dev/tap/whittle. Linux runs the daemon under systemd, see notes.)

Optional: turn on model routing.

whittle policy init                              # calibrated policy, your model ids auto-detected
whittle route -install                           # background service (or `whittle route` in a terminal)
export ANTHROPIC_BASE_URL=http://127.0.0.1:45873

See it

$ whittle compress -stats demo/build.log
... [59 lines omitted]
2026-07-04T10:00:58Z INFO  compiling module pkg/058 ok (406ms)
2026-07-04T10:00:41Z ERROR migrate failed: relation "users" does not exist
... [79 lines omitted]
2026-07-04T10:02:19Z ERROR shutdown: connection reset by peer
2026-07-04T10:02:20Z INFO  144 ok, 2 errors

whittle: action=compressed detected=log strategy=ansi_strip+log_compressor tokens=4728->163

Both errors, their context, and the summary survive; 138 lines of noise collapse into two counted markers. Across 5,000 real agent sessions: 22% tool-output reduction at zero measured information loss: mechanically lossless on 15,846/15,846 items, and a blinded 4-judge panel found 0/120 material loss on the lossy prose path. Full receipts, including an honest side-by-side against headroom: bench/.

Compression: what happens to each content type

JSON is reshaped losslessly (byte-exact reconstruction). Logs and terminal streams are cut lossily but marked and exactly accounted. Markdown file reads keep code fences, tables, and headings byte-exact while prose is compressed by an optional local model with fidelity guards. Source code is never touched. Every path is guarded: the worst case is always not compressed, never corrupted.

The full per-type contract, ML prose path, architecture, and performance tables: docs/compression.md · what each guarantee is pinned by: GUARANTEES.md.

Model routing (opt-in)

Whittle's second surface: a local proxy on ANTHROPIC_BASE_URL that sends each request to the cheapest model tier that can still handle it, per a policy you can read in one screen.

• Calibrated out of the box: whittle policy init writes a conservative default (hard reasoning → strongest tier, confident chit-chat → cheapest, everything else untouched) with your account's real model ids auto-detected. What the default does · how to write your own policy.
• Multi-signal, not keyword-matching: a trained 14-subject classifier (probability-mass thresholded, so an uncertain classification never escalates), a contrastive difficulty score, and your own keywords. Every log line shows each signal's value against its gate.
• Rewrites the model, never your history: prompt-cache prefixes survive; capabilities the cheaper model rejects are stripped automatically; credentials pass through untouched.
• Never blocks you: bad policy, dead classifier, or a rejected rewrite all fall back to your original request. Unset the env var and you're direct again.
• Savings you can measure: every request logs requested model, served model, and real token usage. whittle watch streams both feeds live (routing verdicts + compression carves) in one view.

The router itself (engine, policy design, signal composition) is whittle's own; two pretrained models power its ML signals (the 14-subject domain classifier and the text embedder, both from vLLM Semantic Router). Architecture, signal math, and precise credits: docs/ROUTER.md.

Why write-time?

Most context compressors are read-time proxies: they rewrite your conversation history on every LLM call, which breaks prompt caches, terminates your API traffic, and makes lossy compression the default. Whittle compresses each output once, at the moment it's born, before it enters history: savings compound across every later turn, and nothing sits in your request path.

	whittle	read-time compressors
compresses	once, at write time	re-runs on every LLM call
prompt cache	intact, history never rewritten	invalidated by history rewrites
your request path	nothing resident in it	a proxy that must stay up
loss model	lossless or marked	lossy by default, recover on demand
cost of failure	original bytes	a broken or blocked call

The full argument, and why routing is a different kind of proxy: docs/why-whittle.md.

FAQ

Will it break my agent? No, and that is the core design constraint. Every path fails open: if whittle is down, declines, or errors, your agent sees original bytes. The router likewise: worst case is your request untouched.

Does it need Python? No. The deterministic compressors (JSON, logs, terminal, markdown structure) are pure Go. Python powers the optional prose model and router classifiers; whittle setup installs it if python3 exists, and everything else works without it.

Are token savings dollar savings? Not 1:1. Under prompt caching, cheap cache-reads dominate the bill, so a 22% token cut is roughly 3–5% of session cost. We publish both numbers rather than pretending otherwise: bench/.

How does it compare to headroom? On identical bytes, headroom's defaults compress ~5 points more, by dropping rows whittle refuses to drop. On conversation-shaped content whittle leads while staying lossless. Which trade you want is the whole point: bench/SIDEBYSIDE.md.

Where does my data go? Nowhere. Hook, daemon, models, and router all run on your machine. The router forwards your own credentials to Anthropic and logs token counts, never prompt text.

Which agents? Claude Code today (hook + router). Cursor, Codex, and OpenCode adapters are on the roadmap; the compression engine is also a plain Go library and HTTP service.

Verify it yourself

Every claim here is checkable from a clone; that is the point.

make test                          # guarantees as executable tests (GUARANTEES.md)
go run ./bench                     # corpus reductions + fidelity, SHA-pinned, CI-gated
python bench/calibrate_tokens.py   # reproduces the token-estimator MAE

Contributing

The bar: guarantees are executable, see CONTRIBUTING.md. Fidelity bugs (whittle changing an output's meaning) are treated as urgent.

Near-term roadmap: a tagged release that ships the router, agent adapters (Cursor, Codex, OpenCode), Linux packaging. Good first issues: adapters, packaging, detection corpus cases.

Acknowledgments

Whittle's log-selection strategy, several detection heuristics, and the tabular parser were adapted from Headroom (Apache-2.0); their compaction work is excellent; we wanted the write-time position and the stricter fidelity contract it demands. The router's two pretrained models (the domain classifier and the text embedder behind the similarity signals) come from vLLM Semantic Router (whitepaper); the routing engine and policy design are whittle's own. See NOTICE.

If whittle's fidelity contract is the trade you want, a ⭐ helps other agent users find it.

License

Apache-2.0.

Documentation

Overview

Package whittle carves agent tool outputs down to what matters — and never cuts what doesn't come back.

Whittle is a content-aware compressor for the text an AI agent reads: tool outputs, file reads, logs, JSON, terminal streams. It routes each input to a type-specific strategy and holds one hard line: structural compression is LOSSLESS or CLEARLY MARKED, code never reaches a lossy model, and any anomaly fails open to the original bytes.

The deterministic strategies (JSON columnar reshaping, log selection with omission markers, terminal CR-overwrite collapse, ANSI stripping) work with no dependencies. The optional ML prose path (LLMLingua-2, see model/) activates only when Options.ModelURL (or WHITTLE_MODEL_URL) points at a running model sidecar.

eng := whittle.New(whittle.Options{})
res := eng.Compress(ctx, toolOutput)
if res.Action == "compressed" {
    use(res.Output) // res.SkipReason explains any skip
}

Index

• Variables
• func EstimateTokens(s string) int
• type Engine
  • func New(opts Options) *Engine
  • func (e *Engine) Compress(ctx context.Context, content string) Result
  • func (e *Engine) CompressInput(ctx context.Context, in compress.Input) Result
• type Options
• type Result

Examples

• New

Variables

var SidecarFS embed.FS

SidecarFS carries the Python model sidecar inside the binary so that `go install .../cmd/whittle@latest` is a complete installation: `whittle setup` materializes these files to ~/.whittle/model and builds a venv there.

Functions

func EstimateTokens

func EstimateTokens(s string) int

EstimateTokens is whittle's calibrated token estimator (MAE ~8% vs tiktoken o200k_base) — the same accounting the engine's accept gates use.

Types

type Engine

type Engine struct {
	// contains filtered or unexported fields
}

Engine is a reusable, concurrency-safe compressor.

func New

func New(opts Options) *Engine

New builds an Engine from opts.

Example

package main

import (
	"context"
	"fmt"
	"strings"

	"github.com/firstops-dev/whittle"
)

func main() {
	eng := whittle.New(whittle.Options{MinTokens: 0})
	toolOutput := "ERROR boot failed\n" + strings.Repeat("2026 INFO tick handler ok\n", 80)
	res := eng.Compress(context.Background(), toolOutput)
	fmt.Println(res.Action, res.Detected, strings.Contains(res.Output, "ERROR boot failed"))
}

Output:
compressed log true

func (*Engine) Compress

func (e *Engine) Compress(ctx context.Context, content string) Result

Compress routes content to its type-specific strategy. It never returns an error and never corrupts: every failure or guard trip returns the original content with Action "skipped" and a reason.

func (*Engine) CompressInput

func (e *Engine) CompressInput(ctx context.Context, in compress.Input) Result

CompressInput is the full-control variant (routing override, tool-name / MIME gating hints, prose keep-rate).

type Options

type Options struct {
	// ModelURL enables the ML prose path (an LLMLingua sidecar, see model/).
	// Empty = prose and doc-read content pass through untouched.
	ModelURL string
	// MinTokens skips inputs shorter than this (default 64; compression isn't
	// worth it below that). Use -1 to keep the default; 0 disables the floor.
	MinTokens int
	// MaxChars is the global safety ceiling; larger inputs skip before
	// classification (default 256 KiB).
	MaxChars int
	// ProseMaxChars caps only the ML prose path (default 100000 — a latency
	// budget sized for GPU/MPS inference, not a correctness bound; CPU-bound
	// machines should lower it, see compress/gate.go).
	ProseMaxChars int
	// Logger receives pipeline diagnostics (nil = silent).
	Logger compress.Logger
}

Options configures an Engine. The zero value is production-safe: deterministic compressors only, default gates.

type Result

type Result = compress.Outcome

Result is the outcome of one compression: Output (compressed or original), Action ("compressed"|"skipped"), SkipReason, Strategy, Detected type, sizes.