decode

package module
v0.1.0 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Nov 8, 2026 License: BSD-3-Clause Imports: 7 Imported by: 0

README

decode

ci

Video decoders for avkit, pure Go, no cgo. The first codec is H.264 (ITU-T H.264 | ISO/IEC 14496-10). Wave D1 of go-avkit/avkit#89.

r, _ := container.NewReader(file)                   // MP4, Matroska, ...
frames, err := decode.DecodeTrack(r, trackID, decode.Config{})
for _, f := range frames {                          // display order
    use(f.Y, f.Cb, f.Cr, f.YStride, f.CStride, f.Width, f.Height, f.PTS)
}

Streaming: decode.NewTrack(trackConfig, cfg) or decode.NewAVCC(avcC, cfg), then DecodeSample(sample.Data, pts) per sample and Flush() at the end. Raw .264 files: decode.New(cfg, 4) and DecodeAnnexB.

Frame carries the planes, strides, size, Chroma format, BitDepth and PTS. Its field names match go-videodsp/yuv.Picture for 8-bit 4:2:0, so converting to RGBA is a field copy; yuv.Picture has no bit depth or time, which is why it is not used as is.

What is decoded

The decoding is github.com/oops1/go.264 (BSD-2-Clause, pure Go with assembly kernels), pinned and forced onto its software path. Where it is wrong it is fixed upstream by pull request.

Profile State
Constrained Baseline, Baseline without FMO/ASO bit-exact
Main, progressive (CAVLC and CABAC, B pictures, weighted prediction, multiple references, long-term references, MMCO) bit-exact
High, 4:2:0 8-bit, progressive (8x8 transform, scaling matrices) bit-exact on the streams below; no JVT FRExt stream measured yet
Interlaced (PAFF, MBAFF), FMO/ASO, Extended (SP/SI, data partitioning) refused with ErrUnsupported
High 10, 4:2:2, 4:4:4, lossless refused with ErrUnsupported

Frame.BitDepth and Chroma exist so the later profiles need no API change.

Measured

Every picture's MD5 against ffmpeg -f framemd5 -pix_fmt yuv420p, single thread, and against the reference yuv shipped in the suite where there is one.

  • ITU-T H.264.1 conformance suite, AVCv1: 81 bitstreams tried, 73 match ffmpeg (and the suite's yuv where it ships one): 69 with go.264 v1.12.0 as pinned here, and 4 more (MR2..MR5_TANDBERG) only with oops1/go.264#2 "decoder: handle MMCO 5 and gaps in frame_num"; they are listed in pendingUpstream until it is released. The other 8 are not claimed: 5 interlaced, 3 use FMO/ASO (ffmpeg cannot decode two of them either). ffmpeg's MD5s are in testdata/jvt; the bitstreams are not in the repository (their redistribution terms are not stated), so TestJVTConformance runs when DECODE_JVT_DIR points at them (go run ./tools/jvt fetch downloads them).
  • 29 streams in testdata/streams (from go.264, BSD-2-Clause; Baseline, Main, High) run in CI against ffmpeg's MD5s in the .md5 files.
  • 5 real files (MP4 and Matroska, 720x300 to 1920x1080, x264), 900 pictures each: testdata/real-files.txt, hashes only.

Bounds

A stream is untrusted. Config.MaxFrameMBs (default 36864, a 4096x2304 picture) and MaxSampleBytes (default 64 MiB) are checked before anything sized by the stream is allocated; go.264 also refuses pictures past its level ceiling and bounds its reference frames by max_num_ref_frames. TestBombRefusedUnderMemoryLimit runs three bomb SPSs under a 256 MiB memory limit and asserts a refusal and under 8 MiB allocated. FuzzParseAVCC, FuzzDecodeSample and FuzzDecodeAnnexB have a committed corpus (testdata/fuzz, from go run ./tools/jvt seeds) and run in CI.

Bibliography

  • ITU-T H.264 (V16, 06/2026, in force), opened as the PDF. Read and checked: 7.3.2.1 (SPS syntax entry), 7.4.3 (frame_num semantics: a picture with memory_management_control_operation 5 is inferred to have had frame_num 0; Table 7-9), 8.2.1 (tempPicOrderCnt after operation 5; 8.2.1.1 to 8.2.1.3 prevPicOrderCntMsb/Lsb and prevFrameNumOffset after it), 8.2.5.1 to 8.2.5.4 (marking, gaps in frame_num, sliding window) and C.4.4 (output order and operation 5). The fix upstream (oops1/go.264#2) follows those clauses, including the rebased prevPicOrderCntLsb. ffmpeg (libavcodec h264_refs.c, h264_picture.c, h264_slice.c, h264_parse.c, opened) sets frame_num to 0 on MMCO_RESET and takes the frame order count as the Min of the two field counts; I found no rebase of prev_poc_lsb in those files. The spec text is the reference.
  • ITU-T H.264.1, conformance specification and the bitstream list at https://www.itu.int/wftp3/av-arch/jvt-site/draft_conformance/AVCv1/ (listing and each stream's readme opened).
  • ISO/IEC 14496-15 (avcC): paywalled, not opened. The record layout is that of mp4ff v0.62.0 avc/avcdecoderconfigurationrecord.go (opened), which cites 14496-15 itself; it is fuzzed here.

Performance

Not yet measured against ffmpeg: the machine's 1-minute load was 36 to 50 during this work, above the limit for a measurement (8). Tracked in the issues.

Development

go test -race ./...                       # library, 100% of statements
cd tools && go run ./covergate ../cover.out   # the gate (reads the profile)
cd tools && go test ./...                 # oracle harness

BSD-3-Clause. The streams in testdata/streams are from go.264 (BSD-2-Clause, testdata/streams/LICENSE.go264).

Documentation

Overview

Package decode turns the coded video samples avkit's container package demuxes into pictures.

The first codec is H.264 (ITU-T H.264 | ISO/IEC 14496-10), through github.com/oops1/go.264, a pure Go decoder that is checked here, picture for picture, against ffmpeg. See the README for the profiles that claim holds for and the conformance bitstreams it was measured on.

Input is what a container hands over: an AVCDecoderConfigurationRecord (ISO/IEC 14496-15) or the parameter sets of a container.TrackConfig, then one length-prefixed access unit per sample. Output is every picture in display order as planar YUV, with its chroma format, bit depth and presentation time.

Index

Constants

View Source
const (
	// DefaultMaxFrameMBs is 36864 macroblocks, a 4096x2304 picture: level 5.1
	// of H.264 Annex A.
	DefaultMaxFrameMBs = 36864
	// DefaultMaxSampleBytes is 64 MiB, far above any single access unit a
	// level 6.2 stream can carry at its maximum bit rate and frame rate.
	DefaultMaxSampleBytes = 64 << 20
)

Default limits. A stream is untrusted input: the picture size in its sequence parameter set sizes every buffer the decoder allocates, so it is bounded before the first of them is.

Variables

View Source
var (
	// ErrConfig means the codec configuration (an avcC record or a track's
	// parameter sets) is malformed.
	ErrConfig = errors.New("decode: invalid codec configuration")
	// ErrSample means a sample is not a well formed run of length-prefixed
	// NAL units.
	ErrSample = errors.New("decode: invalid sample")
	// ErrOverLimit means the stream asks for more than Config allows: a
	// picture larger than MaxFrameMBs, or a sample larger than MaxSampleBytes.
	// It is raised before anything sized by the stream is allocated.
	ErrOverLimit = errors.New("decode: stream exceeds the configured limits")
	// ErrUnsupported means the stream uses a coding tool this decoder does
	// not implement (see the README): interlaced coding, a chroma format
	// other than 4:2:0, a bit depth above 8, lossless transform bypass.
	ErrUnsupported = errors.New("decode: unsupported coding tool")
	// ErrClosed means the decoder was used after Close.
	ErrClosed = errors.New("decode: use of a closed decoder")
)

Errors a decoder can refuse with. Each is matched with errors.Is.

Functions

This section is empty.

Types

type AVCConfig

type AVCConfig struct {
	// Profile, Compatibility and Level are AVCProfileIndication,
	// profile_compatibility and AVCLevelIndication, the first three bytes of
	// the SPS after the NAL header.
	Profile, Compatibility, Level uint8
	// LengthSize is the size in bytes of each NAL unit's length prefix:
	// lengthSizeMinusOne + 1: 1, 2 or 4 (3 is reserved).
	LengthSize int
	// SPS and PPS are the parameter sets, as NAL units without a prefix.
	SPS, PPS [][]byte
}

AVCConfig is an AVCDecoderConfigurationRecord, the content of the avcC box (ISO/IEC 14496-15). The standard's text is paywalled and was not opened; the layout is that of mp4ff v0.62.0, avc/avcdecoderconfigurationrecord.go, which was.

func ParseAVCC

func ParseAVCC(b []byte) (AVCConfig, error)

ParseAVCC parses an AVCDecoderConfigurationRecord. It reads the record only as far as the decoder needs: the extension fields a High profile record appends (chroma format, bit depths, SPS extensions) are bounds-checked but not kept, since the sequence parameter set itself is authoritative.

type Chroma

type Chroma uint8

Chroma is the chroma sampling of a Frame.

const (
	Chroma400 Chroma = iota
	Chroma420
	Chroma422
	Chroma444
)

The chroma formats of H.264 (chroma_format_idc). Only Chroma420 is produced today; the others are named so that a Frame can describe the profiles that are not decoded yet without an API change.

type Config

type Config struct {
	// MaxFrameMBs is the largest picture accepted, in 16x16 macroblocks.
	MaxFrameMBs int
	// MaxSampleBytes is the largest sample (access unit) accepted.
	MaxSampleBytes int
	// EnforceLevel also refuses a stream that exceeds its own declared
	// level's limits (Annex A), where MaxFrameMBs alone would admit it.
	EnforceLevel bool
}

Config bounds what a stream may make the decoder allocate. A zero field takes its default.

type Decoder

type Decoder struct {
	// contains filtered or unexported fields
}

Decoder decodes one H.264 elementary stream. It is not safe for concurrent use.

func New

func New(cfg Config, lengthSize int) (*Decoder, error)

New returns a decoder for length-prefixed samples with lengthSize-byte lengths (1, 2 or 4 as avcC allows; MP4 and Matroska use 4). The parameter sets must then arrive in-band, or through Config via NewAVCC or NewTrack.

func NewAVCC

func NewAVCC(avcC []byte, cfg Config) (*Decoder, error)

NewAVCC returns a decoder configured by an avcC record, as MP4 stores it.

func NewTrack

func NewTrack(tc container.TrackConfig, cfg Config) (*Decoder, error)

NewTrack returns a decoder for an avkit container track. Samples are length-prefixed with four-byte lengths, which is what container.Reader hands over for MP4 and Matroska. The track's SPS and PPS are decoded first.

func (*Decoder) Close

func (d *Decoder) Close() error

Close releases the decoder.

func (*Decoder) DecodeAnnexB

func (d *Decoder) DecodeAnnexB(data []byte) ([]Frame, error)

DecodeAnnexB decodes part of a raw Annex B byte stream (start code delimited, as in a .264 file). The pictures' PTS is their index in display order. The chunk need not end on a NAL unit boundary.

func (*Decoder) DecodeSample

func (d *Decoder) DecodeSample(data []byte, pts int64) ([]Frame, error)

DecodeSample decodes one access unit in length-prefixed form, which is container.Sample.Data, and returns the pictures that became ready, in display order. pts is the sample's presentation time; it is attached to the pictures in increasing order as they come out, so it need not belong to the picture the sample itself contains. A picture can come out several samples late.

func (*Decoder) Flush

func (d *Decoder) Flush() ([]Frame, error)

Flush returns the pictures still held for reordering. Call it at the end of the stream.

type Frame

type Frame struct {
	// Y, Cb and Cr are the planes. A sample is one byte when BitDepth is 8
	// and a little-endian 16-bit word above that. A Chroma400 frame has no
	// Cb or Cr.
	Y, Cb, Cr []byte
	// YStride and CStride are the plane row lengths in bytes.
	YStride, CStride int
	// Width and Height are the luma size in samples.
	Width, Height int
	// Chroma is the chroma sampling.
	Chroma Chroma
	// BitDepth is the luma bit depth.
	BitDepth int
	// PTS is the presentation time in the track timescale: the decode time
	// of the sample plus its composition offset. For a raw Annex B stream it
	// is the picture's index in display order.
	PTS int64
}

Frame is one decoded picture, cropped to the stream's display window.

The field names and meaning match go-videodsp/yuv.Picture for 8-bit 4:2:0 so that converting is a field-by-field copy. A Frame owns its planes.

func DecodeTrack

func DecodeTrack(r *container.Reader, trackID uint32, cfg Config) ([]Frame, error)

DecodeTrack decodes every sample of one video track of a demuxed file and returns the pictures in display order with their presentation times in the track's timescale. It is the whole-file form of NewTrack and DecodeSample.

Directories

Path Synopsis
tools module

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL