zstd

Documentation

Overview

Package zstd provides a Scrinium TransformerFactory for the zstd compression algorithm via github.com/klauspost/compress/zstd.

Wiring (typical host setup):

reg := core.NewTransformerRegistry().
    Register("zstd", zstd.New(zstd.Options{}))
store, _, _ := core.InitStore(ctx, drv,
    core.WithReadRegistry(reg), /* ... */)

Stream contract. The factory yields fresh per-operation Encoder and Decoder instances. Both run in O(1) memory: the Encoder wraps the input reader, returning a reader that produces a valid zstd frame; the Decoder wraps a zstd frame and exposes the original bytes. No part of the stream is buffered in full — data is pulled lazily by the consumer.

Bypass heuristics (per docs §7.1). Streams that look already-compressed (high Shannon entropy on a leading window) or are too small to amortise zstd-frame overhead are encoded at the fastest level — the resulting stage in the manifest looks ordinary and the decode path is unchanged. The thresholds (window size, entropy ceiling, microsize bypass) live on Options and are spec-defaulted; see Options for guidance.

Index

• func New(opts Options) core.TransformerFactory
• type Options

Functions

func New

func New(opts Options) core.TransformerFactory

New constructs a zstd TransformerFactory with the given options. Zero-valued fields are filled with the spec defaults.

Types

type Options

type Options struct {
	// EntropyWindowSize is the prefix size, in bytes, on which the
	// Encoder samples Shannon entropy to decide whether the input
	// looks already-compressed. Default: 8192 (8 KiB).
	EntropyWindowSize int

	// EntropyThreshold is the upper entropy bound, in bits per byte
	// (0..8), at which the Encoder switches to its fastest level.
	// Default: 7.7.
	EntropyThreshold float64

	// CompressorBypassLimit is the input-size threshold, in bytes,
	// below which compression is unprofitable (frame overhead
	// dominates) and the Encoder falls back to its fastest level.
	// Default: 512.
	CompressorBypassLimit int

	// Level is the default zstd level for streams that are not
	// bypassed. Default: zstd.SpeedDefault.
	Level zstd.EncoderLevel
}

Options configures the factory. Zero values pick spec defaults.