h265

README

h265

Pure-Go (CGO=0) reader for the H.265/HEVC bitstream: the NAL layer, the parameter sets, the slice segment header, and the boundaries between coded pictures.

units, err := h265.SplitAnnexB(data)
sps, err := h265.ParseSPS(spsUnit)
pps, err := h265.ParsePPS(ppsUnit)
pics, err := h265.SplitPictures(units, pps)   // without decoding any

What it is not

It does not decode pictures. There is no CABAC, no residual, no transform, no prediction and no deblocking — nothing here turns a bitstream into pixels. It reads the syntax a caller needs in order to describe a track, cut a stream on picture boundaries, or hand slices to something that does decode.

What is in it

SplitAnnexB, SplitLengthPrefixed, Unit, UnitType	the NAL layer, both framings
ParseSPS, ParsePPS	the parameter sets
ParseSliceSegmentHeader, SliceSegmentHeader, SliceType	the slice segment header
SplitPictures, Picture	coded pictures, from first_slice_segment_in_pic_flag
POCCounter, POC	picture order counts, clause 8.3.1
ShortTermRPS, RefPic, LongTermRefPic	the reference picture sets a sequence carries
ReferencesOf, PictureRefs, LongTermRef	what one coded picture says it still needs
Derive, RefPicSet, LtPicture	the pictures a decoder must keep for it, clause 8.3.2
Lists, RefListEntry	the two lists a slice predicts from, clause 8.3.4, in the order the slice asked for
PredWeights, RefWeight	how a slice weighs each picture, 7.3.6.3
ShortHeaderError	a unit that ended inside the syntax, said as such

⛔ HEVC's slice types are not H.264's. Here B is 0, P is 1 and I is 2 — the reverse of H.264, where P is 0 and B is 1. A reader that shares a constant between the two formats reads every B slice as a P slice and every P slice as a B one, and the stream still parses.

The order a viewer sees

A stream sends only the LOW bits of each picture's order count, so the count has to be carried across pictures rather than read from one. POCCounter does that.

⛔ The picture the high bits come from is not simply the previous one. It is the previous picture of sub-layer zero that is neither a leading picture nor a sub-layer non-reference — which is exactly the set a thinned stream may have dropped. A counter that carried from every picture would make the order of what remains depend on what was thrown away.

An IDR states no count at all and is zero by definition; a BLA restarts the high bits, because a broken link is a splice.

The derivation is held against a second implementation: FFmpeg's ff_hevc_compute_poc2, transcribed into the test, over 263 168 carry-and-low -bit pairs at two widths. The one place the two part company is a negative carry — FFmpeg reconstructs the previous low bits as a remainder, which is negative there, and 84 of 1024 negative cases disagree. 8.3.1 keeps the low and high parts apart and the low part is what the slice header stated, so that is what this keeps.

The pictures a picture may still need

A sequence parameter set carries the short-term reference picture sets its slices may name. SPS.ShortTermRefPicSets holds them in order, each as the pictures already seen (Before) and those still to come (After), nearest first.

⛔ A set may be written as a DIFFERENCE from an earlier one, and the difference cannot even have its bits counted without the set it is a difference from — so the sets are read in order and every one is kept. The smallest difference the syntax can state is one: abs_delta_rps_minus1 has no way to say "the same set", so a predicted set always moves every entry, and an entry that crosses zero changes which half it belongs to.

⛔ Everything between the order-count width and the sets is conditional and of variable length — sub-layer buffering, block and transform geometry, the scaling lists, the PCM fields. A reader that walked any of them wrongly would read the sets from the wrong bits, and nothing in a set would show it. Each shape has a test that reads a known set back out of it.

The scaling lists are skipped with the reader the picture parameter set already had: a second copy of that walk would be a second thing to drift.

ReferencesOf reads what one picture states. A slice usually names one of the sequence's sets rather than restating it — which is why a sequence carries them — and may instead state one of its own, itself possibly a difference from one of the sequence's.

⛔ The index into those sets is as wide as the number of them needs, and one set means no index at all. A reader that always read bits would take the next field as the index and misread everything after it.

Long-term pictures come from two places at once and are counted as one run: some named out of the sequence's list, some stated by the slice. Each is named by the LOW bits of its order count, and states the high bits only when the low ones would be ambiguous.

An IDR says nothing, and that is not an error: it begins a coded video sequence, so nothing before it is available and nothing after it may reach back.

Derive turns that into the five lists 8.3.2 names, given the picture's own order count. The short-term entries are relative and become absolute here.

⛔ Curr and Foll are not the same thing. A picture in Curr may be predicted from by this picture. One in Foll may not — it is listed because a LATER picture will want it, and a decoder that dropped it would break that one instead. Both must be kept; only one may be used.

⛔ A long-term picture's count is exact only when the slice stated its high bits; otherwise it is the low bits, and the picture meant is whichever held matches them. LtPicture.Exact says which, because handing a decoder low bits as though they were a count is a wrong answer that looks like a right one.

The high bits are stated as a number of wraps that accumulates across the entries — each counted from the one before, not from zero — and the count starts afresh for the slice's own entries after the sequence's. Two entries each stating one wrap are 256 and 512 back, not 256 twice.

The two lists a slice predicts from

Lists builds RefPicList0 and RefPicList1 out of a derived set, clause 8.3.4. They hold the SAME pictures; the order is what distinguishes them.

refs, err := h265.ReferencesOf(first, sps, pps)
set := h265.Derive(refs, poc, sps)
l0, l1, err := h265.Lists(set, refs)

refs carries how many entries each list has and, where the slice stated one, the order they are taken in -- so the caller passes neither.

⛔ The two lists are not interchangeable. List 0 opens with the pictures BEFORE this one in output order, list 1 with those after. A caller that fed a B slice the same list twice, or swapped them, would predict from the wrong side and get an image that is wrong without being malformed.

A list is as long as the slice says, which has nothing to do with how many pictures are available. Shorter and it simply stops. Longer and the candidates are repeated, from the start, as many times as it takes — that is legal, and a list built to stop once would come out short.

An I slice gets no lists. A P slice gets list 0 only.

The order the slice asked for

A slice may state, for each position of each list, which picture goes there -- ref_pic_lists_modification(), 7.3.6.2. ReferencesOf reads it into ListEntryL0 and ListEntryL1, and Lists applies it.

⛔ An entry indexes the TEMPORARY list, which is as long as the greater of the list's own length and the pictures available. A slice may name a picture beyond the end of its own list, and a reader that cut the list to length first would refuse a legal stream.

⛔ nil is not an empty list. A slice that stated no modification leaves the default order standing; a list of no entries is a different thing, and the two must not be spelt the same way.

An entry is Ceil(Log2(NumPicTotalCurr)) bits wide, which holds values past the last picture — three pictures are named in two bits — so an index out of range is reachable from a field of conformant width.

⛔ An entry is bounded by the PICTURES, not by the temporary list it indexes (7.4.7.2). The temporary list is longer whenever the list has more entries than there are pictures, and bounding on it lets an index past the last picture through. FFmpeg bounds on the temporary list; GStreamer enforces the clause and cites it. Nothing is lost by refusing them: the temporary list past NumPicTotalCurr only repeats the candidates.

How a slice weighs what it predicts from

pred_weight_table(), 7.3.6.3, comes back as PictureRefs.Weights -- nil where the picture parameter set does not weight this kind of slice.

⛔ The weights are VALUES, not the differences the syntax carries. A weight is stated as a difference from the neutral 1 << denominator, and a chroma offset as a difference that still has the weight folded into it. Handed the raw numbers, a decoder scales by something close to nothing.

ChromaOffset = Clip(delta - ((128 * ChromaWeight) >> denom) + 128, -128, 127)

⛔ Every luma flag comes first, as one field of n bits, then every chroma flag, and only then the values. H.264 interleaves a flag with its own values; HEVC does not. A reader carrying the sibling's shape over takes the first entry's weight out of the second entry's flag -- a wrong number, not an error.

An entry that states nothing still comes back with the neutral weight and a zero offset, so a caller never has to ask whether a field was present.

The chroma fields follow ChromaArrayType, as the SAO flag does.

TemporalMVP, CollocatedFromL0 and CollocatedRefIdx say which picture the slice takes its motion from, and MaxMergeCand how many merge candidates it uses. The index is stated only where the list has more than one entry.

⛔ The offsets are bounded by the format, and two readers disagree on how much. 7.4.7.3 allows a luma offset in -128..127 and a chroma offset difference in -512..511 — four times the half range. FFmpeg checks neither: it bounds the chroma difference at ±2¹⁷, which guards its own arithmetic rather than stating the format's range, and does not check the luma offset at all. libde265 enforces both. This package follows the tighter pair, and the boundaries are pinned on both sides.

Not here: slice_qp_delta and the fields after it. ReferencesOf stops once it has everything about the pictures.

⛔ No sequence range extension. WpOffsetHalfRange is 1 << 7 unless high_precision_offsets_enabled_flag is set, and that flag lives in a sequence extension this package does not read — it sits past everything ParseSPS consumes. A stream that sets it states its offsets over a wider range and would be refused here, and nothing in this package can tell.

A picture may be carried by several slice segments, and only the one with first_slice_segment_in_pic_flag set begins a new picture. That is what SplitPictures counts, which is how a stream's pictures are counted without decoding any of them.

A ceiling on the sum, not on each half

⛔ 7.4.8 bounds num_negative_pics by sps_max_dec_pic_buffering_minus1 and num_positive_pics by what is LEFT of it, so the ceiling is on the sum. A bound on each half allows twice what any conforming stream may state: eight before and eight after is well inside sixteen each way and outside the format.

The ceiling is fifteen — sps_max_dec_pic_buffering_minus1 is at most MaxDpbSize - 1, and A.4.2 puts MaxDpbSize at sixteen whatever the level. GStreamer bounds NumDeltaPocs the same way.

A width is read before it is narrowed

⛔ log2_max_pic_order_cnt_lsb_minus4 is a BIT COUNT — a slice header reads its order count with it, so a reader given a wrong one is misaligned from that field onwards.

It was bounded, and the bound did not hold: the value was narrowed to uint8 before the check, and that conversion wraps. 1048576 becomes 0, so the width came out as 4 — the smallest legal one — and a value a million past the range read as conformant. The two bit depths were narrowed the same way and not bounded at all.

Every one of them is now read as a full uint32, checked against 7.4.3.2.1's range, and narrowed afterwards. Both boundaries are pinned.

Sibling

go-avkit/h264 is the same layer for H.264/AVC. Its clause 8.2 derivations have HEVC counterparts here — order counts, the reference set, the reference lists — with two exceptions: h264 also reads the prediction weights and applies the list reordering, and this package does neither yet.

Windows, macOS, Linux; six 64-bit architectures. 100% statement coverage, gated in CI.

Licence

BSD-3-Clause.

Documentation

Overview

Package h265 reads an H.265/HEVC bitstream in pure Go, with no libavcodec linkage and no external binaries.

The framing and the bit reading are go-avkit/bitstream's, shared with H.264: the two formats separate their units identically and differ in how the header bytes of a unit are read -- H.264 spends one and this spends two -- and in what the parameter sets hold. This package is that difference.

Index

• Constants
• Variables
• type LongTermRef
• type LongTermRefPic
• type LtPicture
• type POC
• type POCCounter
  • func (c *POCCounter) Next(u Unit, sps SPS, pps PPS) (POC, error)
  • func (c *POCCounter) Reset()
• type PPS
  • func ParsePPS(u Unit) (PPS, error)
• type Picture
  • func SplitPictures(units []Unit, pps PPS) ([]Picture, error)
• type PictureRefs
  • func ReferencesOf(u Unit, sps SPS, pps PPS) (PictureRefs, error)
• type PredWeights
• type RefListEntry
  • func Lists(set RefPicSet, refs PictureRefs) (l0, l1 []RefListEntry, err error)
• type RefPic
• type RefPicSet
  • func Derive(refs PictureRefs, poc int32, sps SPS) RefPicSet
  • func (s RefPicSet) Count() int
• type RefWeight
• type SPS
  • func ParseSPS(u Unit) (SPS, error)
• type ShortHeaderError
  • func (e *ShortHeaderError) Error() string
• type ShortTermRPS
  • func (s ShortTermRPS) Count() int
• type SliceSegmentHeader
  • func ParseSliceSegmentHeader(u Unit, pps PPS) (SliceSegmentHeader, error)
• type SliceType
  • func (t SliceType) String() string
• type Unit
  • func SplitAnnexB(data []byte) ([]Unit, error)
  • func SplitLengthPrefixed(data []byte, lengthSize int) ([]Unit, error)
  • func (u Unit) Unescape() []byte
• type UnitType
  • func (t UnitType) IsBLA() bool
  • func (t UnitType) IsIDR() bool
  • func (t UnitType) IsLeading() bool
  • func (t UnitType) IsRandomAccess() bool
  • func (t UnitType) IsSliceSegment() bool
  • func (t UnitType) IsSubLayerNonReference() bool

Examples

• Lists

Constants

const MaxRefIdxActive = 15

MaxRefIdxActive is the most entries a reference picture list may have, 7.4.3.3. It bounds what a parameter set may state and what Lists will build.

Variables

var (
	// ErrNotPPS means the unit handed over is not a picture parameter set.
	ErrNotPPS = errors.New("h265: unit is not a picture parameter set")
	// ErrUnsupportedPPS means the set does not end where a set must end, so
	// something in it was read wrongly.
	ErrUnsupportedPPS = errors.New("h265: picture parameter set was not consumed exactly")
	// ErrRefIdxRange means the set states more reference indices than a list
	// can hold. 7.4.3.3 puts num_ref_idx_lX_default_active_minus1 in 0..14; a
	// larger value is not a big stream, it is a few bytes asking a reader to
	// size a list from an attacker's number.
	ErrRefIdxRange = errors.New("h265: num_ref_idx_lX_default_active_minus1 out of range")
)

Errors a picture parameter set can be refused with.

var (
	// ErrNotSlice means the unit handed over does not carry a slice segment.
	ErrNotSlice = errors.New("h265: unit does not carry a slice segment")
	// ErrSliceHeader means a segment states something this reader does not follow.
	ErrSliceHeader = errors.New("h265: unsupported slice segment header")
)

Errors a slice segment header can be refused with.

var (
	// ErrNotSPS means the unit handed over is not a sequence parameter set.
	ErrNotSPS = errors.New("h265: unit is not a sequence parameter set")
	// ErrUnsupportedSPS means the set states something this reader does not
	// follow.
	ErrUnsupportedSPS = errors.New("h265: unsupported sequence parameter set")
)

Errors a sequence parameter set can be refused with.

var ErrNoOrder = errors.New("h265: the unit carries no picture order count")

ErrNoOrder means the unit states no order of its own.

var ErrRefLists = errors.New("h265: reference list cannot be built")

ErrRefLists means a slice asked for a reference list that cannot be built.

Types

type LongTermRef

type LongTermRef struct {
	POCLSB uint32
	Used   bool
	// MSBPresent says DeltaMSBCycle was stated. Without it the picture is the
	// one whose low bits match; with it, the one that many wraps further back.
	MSBPresent    bool
	DeltaMSBCycle uint32
	// FromSPS says the entry was named out of the sequence's own list rather
	// than stated by the slice.
	FromSPS bool
}

LongTermRef is one long-term picture a slice names.

A long-term picture is named by the LOW bits of its order count, which is enough while no two candidates share them. When two would, the slice states the high bits as well, which is what MSBPresent says.

type LongTermRefPic

type LongTermRefPic struct {
	POCLSB uint32
	Used   bool
}

LongTermRefPic is one long-term picture a sequence offers to its slices.

type LtPicture

type LtPicture struct {
	POC   int32
	Exact bool
}

LtPicture is one long-term picture a set names.

⛔ POC is a full order count only when Exact is set. Otherwise it is the LOW bits of one, and the picture meant is whichever held in the decoded picture buffer matches them -- which is enough while no two candidates share them, and is why the format lets a slice state the high bits when two would.

type POC

type POC struct {
	// Value is the count itself.
	Value int32
	// LSB is what the slice header stated, before the high bits were carried in.
	LSB uint32
	// Carried says this picture is the one later pictures count from. Only a
	// picture of sub-layer zero that is neither leading nor sub-layer
	// non-reference does that, and a counter that carried from every picture
	// would place everything after the first dropped one wrongly.
	Carried bool
}

POC is a picture's order count: where it belongs in the order a viewer sees, which is not the order it was coded in.

type POCCounter

type POCCounter struct {
	// contains filtered or unexported fields
}

POCCounter derives picture order counts across a coded video sequence.

⛔ It has to be a counter rather than a function because the format sends only the LOW bits of the count. The high bits are carried forward from an earlier picture, and the one they come from is not simply the previous picture: it is the previous one of sub-layer zero that is neither a leading picture nor a sub-layer non-reference. Those are exactly the pictures a thinned stream may have dropped, so counting from them would make the order depend on what was thrown away.

Feed it every coded picture of the sequence, in decode order, one slice per picture: the first slice segment of each. A picture carried by several segments is one picture here.

func (*POCCounter) Next

func (c *POCCounter) Next(u Unit, sps SPS, pps PPS) (POC, error)

Next gives the order count of one coded picture and remembers what the pictures after it will need.

The unit is wanted whole, not just its payload: the count depends on the unit TYPE, which says whether the high bits restart, and on the temporal identifier, which says whether this picture is one later ones count from.

func (*POCCounter) Reset

func (c *POCCounter) Reset()

Reset forgets every picture counted so far. It is what a seek is: the pictures before the new position are not the ones the counts should follow from.

type PPS

type PPS struct {
	ID                     uint32
	SPSID                  uint32
	DependentSlicesEnabled bool
	OutputFlagPresent      bool
	ExtraSliceHeaderBits   uint8
	SignDataHiding         bool
	CABACInitPresent       bool
	NumRefIdxL0            uint32
	NumRefIdxL1            uint32
	InitQP                 int32 // already un-biased
	ConstrainedIntraPred   bool
	TransformSkip          bool
	CUQPDeltaEnabled       bool
	CbQPOffset             int32
	CrQPOffset             int32
	SliceChromaQPOffsets   bool
	WeightedPred           bool
	WeightedBipred         bool
	TransquantBypass       bool
	TilesEnabled           bool
	EntropySyncEnabled     bool
	TileColumns            uint32
	TileRows               uint32
	UniformTileSpacing     bool
	DeblockingOverride     bool
	DeblockingDisabled     bool
	BetaOffsetDiv2         int32
	TCOffsetDiv2           int32
	ListsModification      bool
	SliceHeaderExtension   bool
}

PPS is what a picture parameter set says about the slices that refer to it.

Two of these fields are what a slice segment header cannot be read without: whether a segment may depend on the one before it, and how many bits of the header the stream reserves before the slice type. A reader without them is misaligned from the first field that matters.

func ParsePPS

func ParsePPS(u Unit) (PPS, error)

ParsePPS reads a picture parameter set.

⛔ Unlike H.264's, this needs no other set to be consumed exactly: HEVC states the number of scaling lists by the size class rather than by the chroma format, so the variable parts here depend only on fields the set itself carries.

type Picture

type Picture struct {
	Units []Unit
	Type  SliceType
	// TypeKnown is false when no segment of this picture stated a type, which
	// happens when a stream is joined part way through a picture.
	TypeKnown bool
	Sync      bool
	Bytes     int
}

Picture is one coded picture: the units that carry it, and what it is.

Units are in the order they arrived, parameter sets and all, so a caller writing a sample table has the bytes of a sample and nothing else to gather.

func SplitPictures

func SplitPictures(units []Unit, pps PPS) ([]Picture, error)

SplitPictures groups a stream's units into the pictures they carry.

⛔ Where H.264 needs several fields compared between neighbouring slices to find a boundary, HEVC states it outright: one flag per segment says whether it begins a picture. That is the whole rule here, and it is worth saying that the simpler format is simpler rather than carrying the other one's machinery over.

Parameter sets and delimiters that arrive before a picture's first segment belong to that picture: a decoder must hold them before the segment that uses them.

type PictureRefs

type PictureRefs struct {
	ShortTerm ShortTermRPS
	LongTerm  []LongTermRef
	// NamedSet says the short-term set was NAMED out of the sequence's sets
	// rather than stated by the slice, and SetIndex says which.
	NamedSet bool
	SetIndex int

	// Type is how the slice is coded. An I slice predicts from nothing, so the
	// fields below are not stated for one.
	Type SliceType
	// NumRefIdxL0Active and NumRefIdxL1Active are how many entries each
	// reference list has. They start at the picture parameter set's and a
	// slice may override them.
	NumRefIdxL0Active uint32
	NumRefIdxL1Active uint32
	// ListEntryL0 and ListEntryL1 are ref_pic_lists_modification, 7.3.6.2: the
	// position in the temporary list that each entry of the final list is
	// taken from. nil means the slice stated no modification for that list and
	// the default order stands -- which is NOT the same as an empty slice.
	ListEntryL0 []uint32
	ListEntryL1 []uint32

	// TemporalMVP says the slice predicts motion from another picture, which
	// is what the two fields below name.
	TemporalMVP bool
	// CollocatedFromL0 says which list the picture is taken from, and
	// CollocatedRefIdx its position in that list. Both are only meaningful
	// while TemporalMVP is set.
	CollocatedFromL0 bool
	CollocatedRefIdx uint32
	// Weights is pred_weight_table, nil where the picture parameter set does
	// not weight this kind of slice.
	Weights *PredWeights
	// MaxMergeCand is 5 - five_minus_max_num_merge_cand, in 1..5.
	MaxMergeCand uint8
}

PictureRefs is what one coded picture states about the pictures it may still need: a short-term set, and the long-term pictures it names.

func ReferencesOf

func ReferencesOf(u Unit, sps SPS, pps PPS) (PictureRefs, error)

ReferencesOf reads what a coded picture says it still needs.

An IDR says nothing: it begins a coded video sequence, so nothing before it is available and nothing after it may reach back. That is not an error and comes back as an empty set.

Pass the FIRST slice segment of the picture. A later segment states no set of its own -- the set belongs to the picture -- and its address, which this package cannot yet work out, sits in the way.

type PredWeights

type PredWeights struct {
	LumaLog2Denom   uint8
	ChromaLog2Denom uint8
	// L0 and L1 are as long as the lists they weigh.
	L0 []RefWeight
	L1 []RefWeight
}

PredWeights is pred_weight_table, 7.3.6.3: how a slice weighs each picture it predicts from.

⛔ The weights here are VALUES, not the deltas the syntax carries. The format states each as a difference from the neutral weight, and the offsets of the chroma components as a difference that has to be folded with the weight before it means anything. A caller handed the raw deltas would scale by something close to zero.

type RefListEntry

type RefListEntry struct {
	POC int32
	// LongTerm says the entry came from the long-term pictures, which a decoder
	// treats differently: a long-term reference is not scaled by distance the
	// way a short-term one is.
	LongTerm bool
	// Exact says POC is a full order count. A long-term entry whose slice did
	// not state its high bits carries only the low ones. See LtPicture.
	Exact bool
}

RefListEntry is one entry of a reference picture list.

func Lists

func Lists(set RefPicSet, refs PictureRefs) (l0, l1 []RefListEntry, err error)

Lists builds the reference picture lists a slice predicts from, clause 8.3.4.

⛔ The two lists hold the SAME pictures in a different order, and the order is the point: list 0 starts with what came before this picture, list 1 with what comes after. A decoder handed them the same way round predicts a B slice from the wrong side.

refs carries how many entries each list has and, where the slice stated one, the order its entries are taken in. A list SHORTER than the pictures available simply stops; a list LONGER repeats them, which is legal and is why the candidates are cycled rather than laid down once.

An I slice predicts from nothing and gets no lists. A P slice gets list 0.

Example

ExampleLists is the snippet the README shows, compiled. A documented call that does not build is a claim, not a capability.

package main

import (
	"fmt"

	"github.com/go-avkit/h265"
)

// ExampleLists is the snippet the README shows, compiled. A documented call
// that does not build is a claim, not a capability.
func main() {
	var sps h265.SPS
	// What ReferencesOf reads out of the slice segment header. A B slice that
	// names, for each of its three list 0 entries, the picture it wants there.
	refs := h265.PictureRefs{
		Type:              h265.SliceB,
		NumRefIdxL0Active: 3,
		NumRefIdxL1Active: 2,
		ShortTerm: h265.ShortTermRPS{
			Before: []h265.RefPic{{DeltaPOC: -8, Used: true}, {DeltaPOC: -4, Used: true}},
			After:  []h265.RefPic{{DeltaPOC: 4, Used: true}},
		},
		ListEntryL0: []uint32{2, 0, 1},
	}

	set := h265.Derive(refs, 16, sps)
	l0, l1, err := h265.Lists(set, refs)
	if err != nil {
		fmt.Println(err)
		return
	}
	fmt.Println("list 0:", order(l0))
	fmt.Println("list 1:", order(l1))
}

func order(l []h265.RefListEntry) []int32 {
	out := make([]int32, len(l))
	for i, e := range l {
		out[i] = e.POC
	}
	return out
}

Output:
list 0: [20 8 12]
list 1: [20 8]

type RefPic

type RefPic struct {
	// DeltaPOC is the entry's order count minus the current picture's, so it is
	// negative for a picture already seen and positive for one still to come.
	DeltaPOC int32
	Used     bool
}

RefPic is one entry of a short-term reference picture set: where the picture sits relative to the current one, and whether the current one may use it.

A set lists pictures the DECODER must keep, which is not the same as the pictures this one predicts from. An entry that is not Used is kept because a LATER picture will want it, and dropping it would break that one instead.

type RefPicSet

type RefPicSet struct {
	// StCurrBefore are pictures already decoded that this one may use, nearest
	// first; StCurrAfter are those that follow it in output order and precede
	// it in the stream.
	StCurrBefore []int32
	StCurrAfter  []int32
	// StFoll are short-term pictures to keep and not use.
	StFoll []int32
	// LtCurr and LtFoll are the long-term pictures, likewise split.
	LtCurr []LtPicture
	LtFoll []LtPicture
}

RefPicSet is what one coded picture needs kept, derived from what its slice stated: five lists, and the difference between them is the whole point.

⛔ Curr and Foll are NOT the same thing. A picture in Curr may be predicted from by this picture. A picture in Foll may not -- it is listed because a LATER picture will want it, and a decoder that dropped it would break that one instead. Both must be kept; only one may be used.

func Derive

func Derive(refs PictureRefs, poc int32, sps SPS) RefPicSet

Derive turns what a slice stated into the pictures a decoder must keep, given the order count of the picture that stated it.

The short-term entries are relative and become absolute here; the long-term ones are already counts, or the low bits of counts.

func (RefPicSet) Count

func (s RefPicSet) Count() int

Count is how many pictures the set names in all, which is what a decoded picture buffer must hold for this picture.

type RefWeight

type RefWeight struct {
	LumaStated   bool
	LumaWeight   int32
	LumaOffset   int32
	ChromaStated bool
	// ChromaWeight and ChromaOffset are Cb then Cr.
	ChromaWeight [2]int32
	ChromaOffset [2]int32
}

RefWeight is how one reference picture is weighted, 7.4.7.3.

Stated says the slice gave a weight for that component. Where it did not, the weight is the neutral one -- 1 << the denominator -- and the offset is zero, which this fills in so that a caller never has to ask whether a field was present before using it.

type SPS

type SPS struct {
	VPSID          uint8
	MaxSubLayers   uint8
	ProfileSpace   uint8
	Tier           uint8
	ProfileIDC     uint8
	LevelIDC       uint8
	ID             uint32
	ChromaFormat   uint8
	SeparatePlanes bool
	CodedWidth     uint32
	CodedHeight    uint32
	WinLeft        uint32
	WinRight       uint32
	WinTop         uint32
	WinBottom      uint32
	Width          uint32
	Height         uint32
	BitDepthLuma   uint8
	BitDepthChroma uint8
	Log2MaxPOCLSB  uint8
	// ShortTermRefPicSets are the sets a slice may name instead of stating one
	// of its own, in the order the sequence lists them. A set may be written as
	// a difference from an earlier one, so the order is load-bearing.
	// SAOEnabled and TemporalMVPEnabled are not used here, but a slice header
	// cannot be read without them: both decide whether a field is present,
	// and a reader without them is misaligned from that point on.
	SAOEnabled         bool
	TemporalMVPEnabled bool

	ShortTermRefPicSets []ShortTermRPS
	// LongTermRefPics are the long-term pictures the sequence offers, named by
	// the low bits of their order count.
	LongTermRefPics []LongTermRefPic
	// LongTermRefPicsPresent says the sequence allows long-term pictures at
	// all. It is not the same as offering any: a sequence may allow them and
	// list none, and then a slice states its own.
	LongTermRefPicsPresent bool
}

SPS is what a sequence parameter set says about the pictures that follow.

Width and Height are what a player shows and what a container states: the coded size less the conformance window. Unlike H.264 the coded size is stated in luma samples directly rather than in macroblocks, so there is no multiply -- but the window still has to be taken off, and its offsets are in CHROMA units.

func ParseSPS

func ParseSPS(u Unit) (SPS, error)

ParseSPS reads a sequence parameter set.

type ShortHeaderError

type ShortHeaderError struct {
	Unit  int
	Bytes int
}

ShortHeaderError means a unit is too short to hold the two bytes of header this format spends.

func (*ShortHeaderError) Error

func (e *ShortHeaderError) Error() string

type ShortTermRPS

type ShortTermRPS struct {
	Before []RefPic // negative deltas, nearest first: -1, -2, ...
	After  []RefPic // positive deltas, nearest first: 1, 2, ...
}

ShortTermRPS is a short-term reference picture set: the pictures a coded picture may still need, stated relative to its own order count.

The two halves are kept apart because the format states them apart and because they mean different things: Before holds pictures already decoded, After holds pictures that follow in order but precede this one in the stream. Both are ordered by distance, nearest first.

func (ShortTermRPS) Count

func (s ShortTermRPS) Count() int

Count is how many pictures the set names, which is the NumDeltaPocs the next set may predict from.

type SliceSegmentHeader

type SliceSegmentHeader struct {
	First        bool // first_slice_segment_in_pic_flag
	Dependent    bool // only meaningful when First is false
	PPSID        uint32
	Type         SliceType
	TypeRead     bool
	RandomAccess bool // the unit type says this picture can stand alone
	// NoOutputOfPriorPics is stated only by a random access picture.
	NoOutputOfPriorPics bool
}

SliceSegmentHeader is the beginning of a slice segment: which picture it belongs to, whether it starts one, and -- when it starts one -- how that picture is coded.

⛔ Type is read only for a segment that BEGINS a picture, and TypeRead says so. Reaching the type of a later segment means first reading its address, whose width comes from the coding tree block size in the sequence set -- a field this package does not read yet. Reporting a type it had not read would be worse than saying it did not: a picture's type would then come from whichever segment was looked at.

func ParseSliceSegmentHeader

func ParseSliceSegmentHeader(u Unit, pps PPS) (SliceSegmentHeader, error)

ParseSliceSegmentHeader reads the beginning of a slice segment.

It needs the picture parameter set, which states whether a segment may depend on the one before it and how many bits the stream reserves before the slice type. Both change where the type sits, so a reader without them is misaligned from the first field that matters.

type SliceType

type SliceType uint8

SliceType is how a slice is coded. HEVC numbers them the other way round from H.264, and naming them here rather than reusing a number is the point.

const (
	SliceB SliceType = 0
	SliceP SliceType = 1
	SliceI SliceType = 2
)

The slice types.

func (SliceType) String

func (t SliceType) String() string

String names a slice type the way tools report picture types.

type Unit

type Unit struct {
	Type UnitType
	// LayerID is which layer of a scalable stream this belongs to. A plain
	// stream states zero throughout, and a decoder that ignored it would mix the
	// layers of one that does not.
	LayerID uint8
	// TemporalID is the sub-layer, one more than the field on the wire. A stream
	// can be thinned by dropping the higher ones, which is what it is for.
	TemporalID uint8
	Payload    []byte
}

Unit is one NAL unit: its two header bytes read, and its payload still escaped.

Payload is a window into the caller's bytes and is not copied.

func SplitAnnexB

func SplitAnnexB(data []byte) ([]Unit, error)

SplitAnnexB finds the NAL units of a byte stream and reads each one's header.

func SplitLengthPrefixed

func SplitLengthPrefixed(data []byte, lengthSize int) ([]Unit, error)

SplitLengthPrefixed finds the units of the form an MP4 sample carries, where each is preceded by its length in lengthSize bytes, and reads each one's header.

lengthSize comes from the hvcC record.

func (Unit) Unescape

func (u Unit) Unescape() []byte

Unescape undoes the escaping inside this unit's payload.

type UnitType

type UnitType uint8

UnitType is what a NAL unit holds. Only the ones this package acts on are named; the rest travel as their number.

const (
	UnitTrailN    UnitType = 0
	UnitTrailR    UnitType = 1
	UnitTSAN      UnitType = 2
	UnitTSAR      UnitType = 3
	UnitSTSAN     UnitType = 4
	UnitSTSAR     UnitType = 5
	UnitRADLN     UnitType = 6
	UnitRADLR     UnitType = 7
	UnitRASLN     UnitType = 8
	UnitRASLR     UnitType = 9
	UnitBLAWLP    UnitType = 16
	UnitBLAWRADL  UnitType = 17
	UnitBLANLP    UnitType = 18
	UnitIDRWRADL  UnitType = 19
	UnitIDRNLP    UnitType = 20
	UnitCRA       UnitType = 21
	UnitVPS       UnitType = 32
	UnitSPS       UnitType = 33
	UnitPPS       UnitType = 34
	UnitAUD       UnitType = 35
	UnitEOS       UnitType = 36
	UnitEOB       UnitType = 37
	UnitFD        UnitType = 38
	UnitPrefixSEI UnitType = 39
	UnitSuffixSEI UnitType = 40
)

The unit types a reader has to tell apart.

⛔ A picture's type is not one flag here as it is in H.264. HEVC names sixteen kinds of slice segment, and which of them can be decoded without anything before it is what a sample table needs: the ones from BLA_W_LP to CRA are exactly that set, and IsRandomAccess is where the boundary is written down rather than scattered.

func (UnitType) IsBLA

func (t UnitType) IsBLA() bool

IsBLA says the stream was spliced here. A broken link picture is decoded as a random access point whose leading pictures may refer to material that is not there, and its order count restarts: the high bits are taken as zero however far the count had got.

func (UnitType) IsIDR

func (t UnitType) IsIDR() bool

IsIDR says the picture starts a new coded video sequence and states no order of its own: an IDR carries no slice_pic_order_cnt_lsb at all, and its count is zero by definition.

func (UnitType) IsLeading

func (t UnitType) IsLeading() bool

IsLeading says the picture precedes its random access point in output order while following it in decode order -- RADL if it can be decoded, RASL if it refers to material before the access point and so may not be.

func (UnitType) IsRandomAccess

func (t UnitType) IsRandomAccess() bool

IsRandomAccess says whether a picture of this type can be decoded with nothing before it, which is what a sample table calls a sync sample.

func (UnitType) IsSliceSegment

func (t UnitType) IsSliceSegment() bool

IsSliceSegment says whether a unit carries coded picture data.

func (UnitType) IsSubLayerNonReference

func (t UnitType) IsSubLayerNonReference() bool

IsSubLayerNonReference says no picture of the same sub-layer uses this one for prediction, which is what makes a stream thinnable: these can be dropped without disturbing anything left. The even types below 15 are the ones.