codec

Documentation

Overview

Package codec wraps the block compressors a tatami page payload can use behind one interface, so the container code never cares whether a block is raw, zstd, or dictionary-trained zstd. The enum IDs match the format canon: NONE=0, LZ4=1, ZSTD=2, ZSTD_DICT=3.

All codecs here are deterministic: the same input bytes and the same codec settings produce the same output bytes, which is what the M0 byte-stable round-trip oracle relies on. The zstd codec pins encoder concurrency to one and a fixed level so its output does not drift with GOMAXPROCS.

Index

• Constants
• type Codec
  • func ByID(id ID) (Codec, error)
  • func Default() (Codec, error)
  • func Identity() Codec
  • func NewZstdDict(dict []byte, level zstd.EncoderLevel) (Codec, error)
  • func Zstandard(level zstd.EncoderLevel) (Codec, error)
• type ID

Constants

const DefaultLevel = zstd.SpeedBetterCompression

DefaultLevel is the zstd level the writer pins for both page and blob-record compression, kept in one place so the block codec and the dictionary codec agree.

Types

type Codec

type Codec interface {
	// ID returns the on-disk identifier.
	ID() ID
	// Compress appends the compressed form of src to dst and returns dst. A
	// codec must be deterministic for a given build.
	Compress(dst, src []byte) []byte
	// Decompress appends the decompressed form of src to dst and returns it.
	// uncompressedSize is the known decoded length from the page header, used to
	// size the destination and to bound the output.
	Decompress(dst, src []byte, uncompressedSize int) ([]byte, error)
}

Codec compresses and decompresses a single page payload.

func ByID

func ByID(id ID) (Codec, error)

ByID returns the codec for an on-disk identifier. ZstdDict is intentionally absent: a dictionary codec needs its dictionary bytes, which the generic page path never has. Blob-region records carry the ZstdDict id but are always read through a Dict codec the reader builds from the loaded dictionary, never through ByID.

func Default

func Default() (Codec, error)

Default is the package default block codec for M0: deterministic zstd at a level that balances ratio against write CPU. Later milestones tune the level per region; for now one level serves every page.

func Identity

func Identity() Codec

Identity returns the no-op codec that stores payloads verbatim.

func NewZstdDict

func NewZstdDict(dict []byte, level zstd.EncoderLevel) (Codec, error)

NewZstdDict returns a deterministic zstd codec that compresses and decompresses against the raw content dictionary dict at the given level. An empty dictionary is rejected; the caller should fall back to a plain zstd codec when it has no dictionary to train. The returned codec's ID is ZstdDict.

func Zstandard

func Zstandard(level zstd.EncoderLevel) (Codec, error)

Zstandard returns a shared deterministic zstd codec at the given level. The codec is process-wide and safe for concurrent use through EncodeAll and DecodeAll.

type ID

type ID uint8

ID is the on-disk codec identifier stored in a page header.

const (
	None     ID = 0
	LZ4      ID = 1
	Zstd     ID = 2
	ZstdDict ID = 3
)