squoze

README

squoze

Your context, squoze.

Universal, deterministic LLM context optimizer. One static binary that sits between your AI agent and any OpenAI/Anthropic-compatible provider and shrinks request bodies — without losing information the model needs, and without breaking provider prompt caches.

squoze detects the wire format, classifies every content block and squeezes verbose machine output (test runs, logs, CLI progress spam) with head/tail elision that never drops error lines. Prose, source code and JSON take different paths. Compression presets adapt per model family, and every elision is locally reversible by ref.

Status

v0.2.0 is the current tag. main additionally carries the contract repairs below, which are not in any tag yet.

An external audit of v0.2.0 found that three of the contracts this README states were not in fact held: table column order was nondeterministic, source files could be elided as machine output, and already-sent history was rewritten on a later turn. All three are fixed on main and now pinned by tests, with savings unchanged — median 97.06% → 97.02% across the cases both versions compress. The cross-turn dedup marker text changed; if you match on it, read CHANGELOG.md before upgrading. Full detail, including how to re-run each number: .kiro/specs/distill-contracts/.

Why

Agents waste most of their token budget on noise: repeated file reads, verbose CLI output, stale tool results. squoze removes the noise and keeps the signal — deterministically, under contracts that are tested rather than asserted:

• Fail-open — any error means the request goes through untouched.
• Never-elide — errors, failures and stack traces are never compressed away. Where a cap does force a drop, the marker says how many lines it dropped: a silent loss is worse than a disclosed one.
• Cache-safe — identical bytes always produce identical output, and bytes already sent to the provider are never rewritten on a later turn. Both matter for the same reason: a conversation reaches the provider as a growing prefix, so one changed byte discards the cache from that position on (a 90% discount is worth more than a 20% squeeze).
• Source-safe — prose, source code and JSON are classified and routed, not mutated. A file you are about to compile or diff comes out byte-identical.
• Disclosed — every lossy step reports itself in the marker it leaves: dropped failure lines, truncated table cells, hoisted constant columns.
• Reversible — originals stay local; the model can pull them back on demand.

Measured

Two levels, defined in docs/eval-protocol.md.

Level 1 — deterministic fixtures, every commit. go test ./internal/eval -v:

Fixture	Bytes	Saved	Contract
go_test_600	39 156 → 2 983	92.4%	every FAIL line kept
pytest_verbose	24 385 → 3 174	87.0%	every FAILED / E line kept
server_logs_errors	30 111 → 1 756	94.2%	every ERROR line kept

A violated contract is a red test, not a footnote.

A 15-case corpus lives in the consumer repo and grades needle recall, idempotency, determinism and cross-turn prefix stability as well as savings. Head of main, three runs of the corpus:

15 cases · 14 pass · 0 fail · 1 known-limit
compression fired on 9/15 · median savings when it fired: 97.02%
worst-case p95 engine latency: 5.6-6.5 ms across the three runs
needle recall 100% (except the disclosed MaxKept known-limit)
prefix-stable on every turn pair

The same corpus on v0.2.0 scores 11 pass · 3 fail: the three failures are the contract bugs listed above. One repeat spiked to 14.9 ms on a single case under host contention from parallel container runs; it is measurement noise, not a second code path, and it is left in the reports rather than dropped.

git clone https://github.com/Rethinger/2papi && cd 2papi
go run ./test/squozebench          # writes test/results/squoze_quality_report.json
cat test/squozebench/repro/README.md   # A/B against a released squoze

Level 1b — the work guard, v0.4.0. Deciding "this body has nothing to compress" used to cost a full json.Unmarshal; it is now answered from the raw bytes. prescan=false is literally the v0.3.0 path, so the two arms are a release comparison:

Shape	v0.3.0	v0.4.0	Δ	Allocations
140 KB object of objects	5.077 ms	0.847 ms	−83%	23 835 → 0
560 KB object of objects	26.371 ms	3.115 ms	−88%	95 274 → 0
140 KB array of scalars	6.197 ms	0.372 ms	−94%	35 957 → 0
140 KB liftable rows (work happens)	8.599 ms	9.645 ms	+12%	unchanged

The last row is the honest cost: on a body that does compress, the structural scan runs once in the engine and once inside the JSON pass, which needs its result anyway. 0.4-0.7 ms bought 4.2-23 ms, so the trade is kept and written down rather than tuned away. Output is unchanged in both arms by construction — every gate condition is a necessary condition of its pass succeeding — and that is a test over the whole shape corpus, not a claim.

go test ./internal/distill/ -run '^$' -bench Gate -benchtime 50x
go test ./internal/engine/  -run '^$' -bench 'Shapes|Turns' -benchtime 20x

Not measured: answer quality against a live model. Level 2 of the protocol (LoCoMo, RULER, BFCL v3 through a real provider, gate Δaccuracy ≤ 2 pp) is specified but has not been run. Until it is, squoze makes no claim about task success rate, Pass@1 or benchmark scores — earlier revisions of this README quoted such figures from fixtures that could not support them, and that was wrong. Savings, latency and the contracts above are what the numbers here cover.

Install

go install github.com/Rethinger/squoze/cmd/squoze@latest

No prebuilt binaries yet — go install or go build from source. On Windows a freshly built binary may trip a Defender false positive (ThreatID 251873); add an exclusion or sign it locally.

Usage

# Drop-in proxy for any OpenAI/Anthropic-compatible client
squoze proxy --port 8787 --upstream https://api.anthropic.com
ANTHROPIC_BASE_URL=http://localhost:8787 claude

# Or zero-config: wrap the agent, env injection does the rest
squoze wrap --upstream https://api.anthropic.com claude

# Resolve an elision marker ref back to the full original text
squoze retrieve a3f9c2e1b4d0

# Check savings against a live provider, or patch opencode's provider list
squoze livecheck
squoze oc

Every response carries X-Squoze-Original-Bytes, X-Squoze-Sent-Bytes, X-Squoze-Format and X-Squoze-Upstream.

Use as a library

The root package is a thin facade over the engine — one instance per process keeps the decision memo warm, which is what makes repeat turns byte-identical:

import "github.com/Rethinger/squoze"

eng := squoze.NewEngine(squoze.DefaultMemoCapacity)
out, res := eng.Apply(requestBody)

Bounds are optional and measured in bytes:

eng := squoze.NewEngineWithLimits(squoze.DefaultMemoCapacity, squoze.Limits{
	MaxBodyBytes:  8 << 20, // skip a body this large without sniffing its format
	MaxBlockBytes: 2 << 20, // leave one oversized content block alone
})
out, res := eng.Apply(requestBody)
if res.Skipped {
	log.Printf("squoze skipped: %s", res.SkipReason) // "body_too_large"
}

A zero Limits is v0.3.0 behaviour byte for byte. There is deliberately no deadline option: a time budget makes the output depend on how loaded the host was, and a body that compresses on an idle box but passes through on a busy one breaks the provider prompt cache for every turn after it. Bounds you can reason about ahead of the request are sizes.

This is how 2papi embeds it as an optimization mode.

Build

go build ./...
go test ./...                # unit tests + quality contracts
go test ./internal/eval -v   # the was → is savings table above

Apache-2.0. Pending work is tracked honestly in docs/UNFINISHED.md; released changes in CHANGELOG.md.

Documentation

Overview

Package squoze is the library facade for the universal, deterministic LLM context optimizer ("your context, squoze.").

Typical gateway embedding:

eng := squoze.NewEngine(squoze.DefaultMemoCapacity)
out, res := eng.Apply(requestBody)

One Engine instance per process keeps the decision memo warm: identical original bytes always produce byte-identical output, which keeps provider prompt caches stable across turns. See README.md for the quality contracts (fail-open, never-elide, cache-safe, reversible).

Index

• Constants
• type Engine
  • func NewEngine(memoCapacity int) *Engine
  • func NewEngineWithLimits(memoCapacity int, lim Limits) *Engine
• type Limits
• type Result
  • func Process(body []byte) ([]byte, Result)

Constants

const DefaultMemoCapacity = engine.DefaultMemoCapacity

DefaultMemoCapacity bounds the per-process decision memo (~4k blobs).

const SkipBodyTooLarge = engine.SkipBodyTooLarge

SkipBodyTooLarge is the Result.SkipReason set when MaxBodyBytes rejected a body before its format was even sniffed.

const Version = engine.Version

Version mirrors the CLI version.

Types

type Engine

type Engine = engine.Engine

Engine runs the optimization pipeline with its cache-guard state.

func NewEngine

func NewEngine(memoCapacity int) *Engine

NewEngine returns an isolated pipeline instance.

func NewEngineWithLimits

func NewEngineWithLimits(memoCapacity int, lim Limits) *Engine

NewEngineWithLimits is NewEngine with explicit bounds. Engines with different Limits must not be shared: the bounds decide what a body turns into, so the decision memo is only sound within one set of them.

type Limits

type Limits = engine.Limits

Limits bounds the work one Apply may do, in bytes. The zero value is v0.3.0 behaviour exactly: no bound is applied unless it is configured.

Bounds are sizes and never durations on purpose. A wall-clock deadline would make the output depend on how loaded the host was, and the cache-safe contract requires that identical original bytes always produce byte-identical output — a request that compresses on an idle box and passes through on a busy one breaks every provider prompt cache downstream of it.

type Result

type Result = engine.Result

Result reports what one pipeline pass did to a request body.

func Process

func Process(body []byte) ([]byte, Result)

Process runs the default shared engine over a request body.