compress

README

nyarime/compress

Pure Go house compression codecs for the Nyarime stack. Apache-2.0 (same family as GoFEC).

简体中文

Philosophy

In-house first. We study klauspost/compress, libzstd, xz/7-Zip, and public corpora — then implement and tune our own encoders. This is not a long-term wrapper around klauspost.

Package	Role
zstd/	RFC 8878 house encoder/decoder (NYA CompressionID 1–5)
lzma2/	Raw LZMA2 streams + XZ/LZMA decode helpers (CompressionID 6)

NYA (GPL-3.0) links this library for archive payloads.

Status

v0.2.6 — LZMA2 level-9 optimal parse + 7-Zip comparison harness.

Recent LZMA2 work (v0.2.3–v0.2.6):

Release	Highlights
v0.2.3	Dual-encode guard: level 9 runs greedy + optimal, keeps smaller output
v0.2.4	Cached length price tables for optimal DP
v0.2.5	BT4 tree navigation fix; optimal prices all four rep slots
v0.2.6	Dense match cut-offs (17–80 B) for JSON/log optimal parse

Level-9 vs 7-Zip -mx9 (Aug 2026)

Measured with go test ./lzma2 -run TestBenchVs7zCorpora -count=1 -timeout 15m. See docs/BENCHMARK-7Z.md for corpora and reproduction.

corpus	nya%	7z%	gap
structured_text	1.1	1.6	−0.5pp
pseudo_enwik	0.7	1.1	−0.5pp
mixed_json_log	7.1	5.0	+2.1pp
silesia_dickens (1 MiB)	38.3	29.6	+8.8pp

Negative gap = house LZMA2 smaller than 7z on that corpus. Phase-4 target is gap ≤ 0 on Silesia + enwik9 at level 9 — see ROADMAP.md.

Quick start

go get github.com/nyarime/compress@v0.2.6

import "github.com/nyarime/compress/lzma2"

out, err := lzma2.Lzma2CompressOpts(data, lzma2.Lzma2Options{
    DictSize:     64 << 20,
    Depth:        256,
    NiceLen:      128,
    OptimalParse: true, // dual-encode vs greedy when set
})

Related

• GoFEC — FEC (Apache-2.0)
• NYA — archive + CLI
• ROADMAP.md — ratio milestones
• docs/BENCHMARK-7Z.md — 7z comparison harness