README
¶
h264
Pure-Go (CGO=0) reader for the H.264/AVC bitstream, and the derivations clause 8.2 defines on top of it: picture order counts, the reference picture set, the reference lists a slice predicts from, and the weights a bi-predicted block is combined with.
pred := h264.NewPredictor()
for _, u := range units {
h, ref, err := h264.ParseSliceReferences(u, sps, pps)
if err != nil || !h.BeginsPicture() {
continue
}
p, err := pred.Picture(u, h, ref, sps, pps) // POC, L0, L1, and a handle
}
What it is not
It does not decode pictures. There is no entropy decoding, no residual, no transform, no motion compensation and no deblocking here — nothing in this module turns a bitstream into pixels. It answers the questions a decoder has to answer before it can: which pictures are references, in what order a slice addresses them, what each picture's order count is, and how two predictions are weighted.
For pixels, compose it with a decoder. oops1/go.264
is a pure-Go H.264 decoder and the two agree on 785 reference lists across 18
streams and on all 65536 implicit-weight pairs — see below.
What is in it
SplitAnnexB, SplitLengthPrefixed, Unit |
the NAL layer |
ParseSPS, ParsePPS |
the parameter sets |
ParseSliceHeader, ParseSliceReferences |
the slice header, through dec_ref_pic_marking |
SplitPictures |
coded pictures, without decoding any |
POCCounter |
picture order counts, types 0 and 2 (§8.2.1) |
RefSet |
the reference picture set: sliding window and every marking operation (§8.2.5) |
InitialRefLists, ApplyRefListOps |
the reference lists and their modifications (§8.2.4) |
ImplicitWeighting, ExplicitWeighting, WeightingFor, DistScaleFactor |
the prediction weights (§8.4.2.3) |
Predictor |
the five above, composed in the one order that is correct |
ErrRefIdxRange |
a set or a slice stating more active references than a list may hold |
Order count type 1 is not derived. It needs offset_for_ref_frame and its
cycle, which the set reader does not read, so POCCounter returns ErrPOCType
rather than a number it cannot compute. Measured over 158 sequences from one
library: every one was type 0 or type 2, none was type 1.
Field coding is refused, not guessed. A field pair shares a frame number and is marked as a unit; nothing here implements that, and answering anyway would hand back a set that is quietly wrong.
Windows, macOS, Linux; six 64-bit architectures. 100% statement coverage per function, gated in CI.
A count from the stream sizes an allocation
⛔ num_ref_idx_lX_active_minus1 is a 32-bit syntax element and a reference
entry is 24 bytes. An 18-byte picture parameter set states 4 294 967 295 of
them. The allocation in ApplyRefListOps was measured linear — 1 Mi entries for
25.2 MB, 10 Mi for 251.7 MB — so that set asks for about 103 GB.
7.4.2.2 puts the field in 0..31. The bound is applied where the count is
read: in ParsePPS and in the slice header's own override. It used to live
in readWeightList alone, whose comment said as much — "how many weights a list
may hold" — and a slice that stated no weight table carried the count straight
past it to the list construction.
The check is made on the value before the +1 the syntax carries: at the
top of the range that increment wraps to zero, and a set claiming a list of no
entries would read as conformant.
A selector the consumer indexes with
⛔ cabac_init_idc picks one of THREE context tables and
disable_deblocking_filter_idc one of three behaviours. Neither was bounded,
and a consumer indexes with them: a slice stating 1 048 576 was read and
handed on. 7.4.3 puts both in 0..2.
⛔ SliceQPY indexes the quantisation tables, and it is a SUM — the picture
set's InitQP plus the slice's delta. Neither end was bounded, so a delta of
1000 gave a quantiser of 1026, twenty times the largest the tables hold.
The floor moves with the bit depth: QpBdOffsetY is six per bit past eight,
so a ten-bit sequence may legally state a quantiser down to −12. A bound written
as zero refuses that stream, and there is a witness for it.
Bounded with them, each because something later indexes a table with it: both deblocking offsets (−6..6), the picture and switching quantisers, and both chroma quantiser offsets (−12..12).
A width is read before it is narrowed
⛔ log2_max_frame_num_minus4 and log2_max_pic_order_cnt_lsb_minus4 are BIT
COUNTS — the slice header reads frame_num and the order count with them, so
a reader given a wrong one is misaligned from that field onwards. Each was kept
in a byte with no bound at all, and so were both bit depths.
A narrowing conversion does not merely lose a value: it maps out-of-range values
onto legal ones. uint8(1048576) is 0, and uint8(256) + 4 is 4 — the
smallest width the format allows — so a check made afterwards would pass them.
All four are now read as a full uint32, checked against 7.4.2.1.1's range, and
narrowed afterwards. Both boundaries are pinned.
A weight is a shift count and a multiplier
⛔ luma_log2_weight_denom leaves this package as a SHIFT COUNT — a
Weighting computes 1 << LogDenom and shifts a sample by it. A count the
stream chose freely is a shift by whatever it likes, and in Go that is not a
panic: it is a silently wrong number. Measured before the bound: a slice
declaring 1 048 576 was read and handed on.
⛔ A weight multiplies a sample before anything clamps it. One of 2³⁰ overflows the arithmetic rather than making a bright picture; that too was read and handed on.
7.4.3.2 puts both denominators in 0..7 and every weight and offset in
-128..127, and ParseSliceReferences now refuses the rest. Both boundaries
are pinned: -128 and 127 are read, 128 and -129 are not.
How it is verified
Every rule has a witness, and every witness was checked to fail when its rule is ablated — a test that passes against the broken code is not a test. Beyond that, three controls that do not depend on this code being right:
- The reference set against ffmpeg.
ffmpeg -threads 1 -debug mmcoprints its own reference set; it shares no code with this. 3110 reference pictures across 101 streams, 3110 agree, zero disagreements, and no disagreement about how many reference pictures a stream has either. - The reference lists against
oops1/go.264. 785 lists across 18 streams, zero disagreements. - The implicit weights, exhaustively. The clause clips
tbandtdto [-128, 127], so 65536 pairs is the whole domain rather than a sample: all 65536 agree withgo.264, plus the long-term fallback.
Those comparisons found defects in both directions. They caught Prediction
carrying the list ordering rather than the list the slice can index, and they
localised an output-reordering defect in go.264 that had been reported — by
this project — as a decoding defect it was not.
Licence
BSD-3-Clause.
Documentation
¶
Overview ¶
Package h264 reads an H.264/AVC bitstream in pure Go, with no libavcodec linkage and no external binaries.
The framing and the bit reading live in go-avkit/bitstream, which H.265 shares: the two formats separate their units identically and differ only in how the header bytes of a unit are read. This package is that difference, and what stands on it.
Index ¶
- Variables
- func DistScaleFactor(currPOC, poc0, poc1 int32) int32
- func ParseSliceReferences(u Unit, sps SPS, pps PPS) (SliceHeader, SliceReferences, error)
- type MarkOp
- type POC
- type POCCounter
- type PPS
- type Picture
- type PredWeights
- type Prediction
- type Predictor
- type RefListOp
- type RefPicture
- type RefSet
- type RefWeight
- type SPS
- type SliceHeader
- type SliceReferences
- type SliceType
- type Unit
- type UnitType
- type Weighting
Constants ¶
This section is empty.
Variables ¶
var DefaultWeighting = Weighting{LogDenom: 5, W0: 32, W1: 32}
DefaultWeighting is the plain average of two predictions, which is what the format falls back to whenever a derivation cannot be trusted.
var ErrNotPPS = errors.New("h264: unit is not a picture parameter set")
ErrNotPPS means the unit handed over is not a picture parameter set.
var ErrNotSPS = errors.New("h264: unit is not a sequence parameter set")
ErrNotSPS means the unit handed over is not a sequence parameter set.
var ErrNotSlice = errors.New("h264: unit is not a coded slice")
ErrNotSlice means the unit handed over does not carry a coded slice.
var ErrPOCType = fmt.Errorf("%w: picture order count type", ErrSliceHeader)
ErrPOCType means a sequence states its order in a shape this does not derive.
var ErrRefIdxRange = errors.New("h264: num_ref_idx_lX_active_minus1 out of range")
ErrRefIdxRange means a parameter set or a slice header states more active references than a list may hold. 7.4.2.2 and 7.4.3 put num_ref_idx_lX_active_minus1 in 0..31; a larger value is not a large stream, it is a few bytes asking a reader to size a list from the stream's number.
var ErrRefLists = fmt.Errorf("%w: reference lists", ErrSliceHeader)
ErrRefLists means a slice's lists cannot be built as stated.
var ErrRefSet = fmt.Errorf("%w: reference set", ErrSliceHeader)
ErrRefSet means a picture's marking cannot be applied as stated.
var ErrSliceGroups = errors.New("h264: slice groups are not supported")
ErrSliceGroups means the set uses more than one slice group, whose map this reader cannot consume.
var ErrSliceHeader = errors.New("h264: unsupported slice header")
ErrSliceHeader means a slice states something this reader does not follow.
var ErrUnsupportedPPS = errors.New("h264: picture parameter set was not consumed exactly")
ErrUnsupportedPPS means the set does not end where a set must end, so something in it was read wrongly.
var ErrUnsupportedSPS = errors.New("h264: unsupported sequence parameter set")
ErrUnsupportedSPS means the set states something this reader does not follow.
Functions ¶
func DistScaleFactor ¶
DistScaleFactor is how far the current picture sits between two references, scaled so that 128 means "all the way to the second".
It is the quantity two different parts of the format are built on -- the weights of implicit bi-prediction and the motion a temporal-direct block inherits -- so it is derived once here rather than twice at the places that need it.
⛔ Every shift here is an ARITHMETIC shift, and the distances are signed. In Go as in the format, -1 >> 2 is -1, while -1 / 4 is 0: the shift floors and the division truncates toward zero. Written with a division, 768 of the 2048 factors this can produce come out wrong -- every negative one, which is every reference pair the current picture does not sit between.
(The halving inside the absolute value is NOT such a trap: Abs(td/2) and Abs(td)/2 agree on all 256 values td can take, because truncation toward zero makes them the same. Checked, rather than assumed, before this was written.)
func ParseSliceReferences ¶
func ParseSliceReferences(u Unit, sps SPS, pps PPS) (SliceHeader, SliceReferences, error)
ParseSliceReferences reads a slice header whole: everything ParseSliceHeader reads, and then what the slice says about the pictures it predicts from -- how many references it uses, how its lists are built, the weights it states, and how it marks the reference set.
It is the same single reader, told to keep going. Picture segmentation must not fail because a slice's weight table is unreadable, and a decoder must not be handed a zero weight table because nobody asked for one -- which is why the answer comes in a type of its own.
Types ¶
type MarkOp ¶
type MarkOp struct {
// Operation is memory_management_control_operation. Five is the one that
// matters most to a reader: it empties the reference set and RESTARTS the
// picture order count at this picture.
Operation uint32
// Value is whichever argument the operation takes, and zero for those that
// take none.
Value uint32
// Extra is long_term_frame_idx for operation three, which takes two.
Extra uint32
}
MarkOp is one memory management control operation, as dec_ref_pic_marking states them.
type POC ¶
type POC struct {
// Top and Bottom are TopFieldOrderCnt and BottomFieldOrderCnt. A frame has
// both; a field has only its own, and the other is left at zero.
Top int32
Bottom int32
// Value is the count a picture is ordered by: the smaller of the two for a
// frame, and the one field's own for a field.
Value int32
}
POC is a picture's order count: where it belongs in the order a viewer sees, which is not the order it was coded in.
type POCCounter ¶
type POCCounter struct {
// contains filtered or unexported fields
}
POCCounter derives picture order counts across a coded video sequence.
⛔ It has to be a counter rather than a function because the format states the count RELATIVE to the pictures before it: type zero sends only the low bits and the high bits are carried forward, and type two derives everything from a frame number that wraps. A caller computing a count from one slice header alone gets the right answer until the first wrap and the wrong one afterwards.
⛔ Only a REFERENCE picture moves the carry forward. A disposable picture reads the carry and leaves it where it was, so a counter that updated from every picture would place every picture after the first disposable one wrongly.
Feed it every coded picture of the sequence, in decode order, one slice per picture. A picture carried by several slices is one picture here: pass the slice that begins it and nothing else, or the count advances once per slice.
func (*POCCounter) Next ¶
func (c *POCCounter) Next(u Unit, h SliceHeader, ref SliceReferences, sps SPS) (POC, error)
Next gives the order count of one coded picture, and remembers what the pictures after it will need.
The unit is wanted for nal_ref_idc, which decides whether this picture moves the carry; the references are wanted for the one marking operation that restarts the count.
func (*POCCounter) Reset ¶
func (c *POCCounter) Reset()
Reset forgets every picture counted so far. It is what a seek is: the pictures before the new position are not the ones the counts should follow from.
type PPS ¶
type PPS struct {
ID uint32
SPSID uint32
CABAC bool // entropy_coding_mode_flag
BottomFieldPicOrder bool
NumRefIdxL0 uint32
NumRefIdxL1 uint32
WeightedPred bool
WeightedBipredIDC uint8
InitQP int32 // already un-biased: pic_init_qp_minus26 + 26
InitQS int32
ChromaQPIndexOffset int32
DeblockingControl bool
ConstrainedIntraPred bool
RedundantPicCnt bool
Transform8x8 bool
SecondChromaQPOffset int32 // equals ChromaQPIndexOffset when not stated
}
PPS is what a picture parameter set says about the slices that refer to it.
CABAC is the field everything above this turns on: the residuals of a slice are coded one way or the other and nothing can read them without knowing which.
func ParsePPS ¶
ParsePPS reads a picture parameter set.
⛔ It needs the sequence parameter set the PPS refers to, and not for convenience: the number of 8x8 scaling lists a PPS carries depends on the SPS's chroma format, so a reader without it cannot consume them exactly and everything after them is read from the wrong bits. The caller holds the SPS already -- a stream states it first -- so this asks for it rather than guessing.
type Picture ¶
type Picture struct {
Units []Unit
Type SliceType
Sync bool
Bytes int // the units' payloads and headers, not the separators between them
}
Picture is one coded picture: the units that carry it, and what it is.
Units are the NAL units in the order they arrived, parameter sets and all, so a caller writing a sample table has the bytes of a sample and nothing else to gather. Sync says the picture can be decoded without any before it, which is what a sample table calls a sync sample and a player calls a seek point.
func SplitPictures ¶
SplitPictures groups a stream's NAL units into the pictures they carry.
⛔ A picture may be carried by several slices, so the units between one picture's first slice and the next one's belong together. The slice at macroblock zero is what marks the start, and it is the condition that does almost all the work: measured against ffprobe on 32 files, it alone put every picture boundary in the right place.
It is not the whole rule the format states, and the rest is here because that one condition cannot tell two cases apart. Two pictures in a row both begin at macroblock zero -- that is the normal case and the reason the condition works. But a REDUNDANT slice repeats a picture already carried, also from macroblock zero, and a field pair carries two pictures whose slices both start there. So a slice at macroblock zero begins a new picture only if something that identifies the picture has also changed: the frame number, the parameter set, the field it codes, whether it is an IDR or which one, the picture order count, or whether it is a reference at all.
Conditions the standard lists that are NOT here: nothing depends on memory management operations or on a stream switching between coded frames and coded fields mid-sequence, both of which need state this does not keep. A stream doing either would have its pictures grouped by the conditions above, which is right wherever one of them changes and wrong only where none does.
Parameter sets and delimiters that arrive before a picture's first slice belong to that picture, not the one before: a decoder must have them in hand before the slice that uses them.
type PredWeights ¶
type PredWeights struct {
LumaLog2Denom uint32
ChromaLog2Denom uint32
L0 []RefWeight
L1 []RefWeight
}
PredWeights are the weights a slice states for weighted prediction, one entry per active reference.
type Prediction ¶
type Prediction struct {
// POC is the picture's order count.
POC POC
// L0 and L1 are the reference lists the slice uses, after any modification
// it states. An I slice has neither; a P slice has only L0.
L0, L1 []RefPicture
// Handle is what the reference set gave this picture, or -1 when the
// picture was not kept as a reference.
Handle int
}
Prediction is what one coded picture predicts from.
type Predictor ¶
type Predictor struct {
// contains filtered or unexported fields
}
Predictor walks a stream's coded pictures in decode order and answers, for each one, what it predicts from.
It exists because the five steps below have to happen in one particular order, and nothing in their signatures says so:
- the picture order count, which is state carried from the picture before;
- the initial reference lists, FROM THE SET AS IT STANDS;
- the modifications the slice states, searched in that same set;
- the slice's own marking, applied to the set;
- the picture joining the set, if it is a reference.
⛔ Step 2 before step 5 is the whole point. A picture predicts from the set as it was BEFORE the picture joined it -- take the picture in first and it predicts from itself, which decodes to a plausible and wrong image rather than to an error. Three separate callers in this project's own harnesses had to rediscover that ordering from the doc comments, which is the signal that the composition belonged here.
It holds no buffers and copies no samples: a Prediction names pictures by the handles the set gave them, and a caller keeps its own frame store.
func NewPredictor ¶
func NewPredictor() *Predictor
NewPredictor is a predictor at the start of a stream.
func (*Predictor) Held ¶
func (p *Predictor) Held() []RefPicture
Held is the set of pictures currently kept as references, in the order they were taken in. It is what the next picture will predict from.
func (*Predictor) Picture ¶
func (p *Predictor) Picture(u Unit, h SliceHeader, ref SliceReferences, sps SPS, pps PPS) (Prediction, error)
Picture answers what this coded picture predicts from, and advances the state. Feed it every coded picture of the stream in decode order, after reading the header of the slice that begins it.
⛔ Pass the FIRST slice of each picture, the one with first_mb_in_slice zero (see SliceHeader.BeginsPicture). A picture carried by several slices states its order count and its marking once, in every slice; feeding each of them would count the picture several times -- sliding the window too often and applying one marking repeatedly, which for an operation that releases a picture means refusing the second time round.
type RefListOp ¶
type RefListOp struct {
// Kind is modification_of_pic_nums_idc: 0 and 1 move a short-term picture
// backwards or forwards from the one before, 2 names a long-term picture.
Kind uint32
// Value is abs_diff_pic_num_minus1 for kinds 0 and 1, and long_term_pic_num
// for kind 2.
Value uint32
}
SliceHeader is the beginning of a coded slice: what it belongs to and how it is coded, but not the coded data itself.
FirstMB is what says where a picture begins. A picture may be carried by several slices, and only the one starting at macroblock zero begins a new picture -- which is how a stream's pictures are counted without decoding any. RefListOp is one instruction for building a reference picture list, as ref_pic_list_modification states them.
type RefPicture ¶
RefPicture is one picture a slice may predict from.
The caller holds the samples; this package orders the pictures. ID is the caller's own handle, carried through untouched so that an ordered list can be turned back into buffers.
func ApplyRefListOps ¶
func ApplyRefListOps(list, refs []RefPicture, ops []RefListOp, active int, currPicNum uint32, maxPicNum uint32) ([]RefPicture, error)
ApplyRefListOps rewrites a list the way a slice's instructions ask.
list is the list to rewrite and refs is the whole reference set. active is num_ref_idx_lX_active: the length the list must have when this returns. maxPicNum is 1 << log2_max_frame_num for a frame-coded picture, and currPicNum is the slice's own frame number.
⛔ A picture an instruction names is looked for in the REFERENCE SET, not in the list being rewritten. The list is only as long as the slice is active plus one, and the first instruction's shift pushes its last entry off the end -- so a second instruction naming that picture finds nothing. Measured on a real stream: a P slice holding frames 8, 9, 10 and 11 asked for 11 and then for 9, and searching the list refused the second because placing the first had already dropped it.
⛔ The list is active+1 long while the instructions are applied, and is cut back at the end. The format says so, and it matters: an instruction that moves a picture forward needs somewhere to put the one it displaces, and a reader working in a fixed-length list drops that picture instead.
func InitialRefLists ¶
func InitialRefLists(h SliceHeader, refs []RefPicture, currPOC int32, currFrameNum uint32, maxFrameNum uint32) (l0, l1 []RefPicture, err error)
InitialRefLists builds the reference lists a slice starts from, before any instruction it states is applied.
refs are the pictures held as references, in any order. currFrameNum and currPOC describe the slice's own picture, and maxFrameNum is 1 << the set's log2_max_frame_num.
⛔ Only frames. A field-coded picture interleaves the two parities and splits every reference in two, which is a different derivation altogether -- and of 158 sequences measured in one library, every one was frame-only. Returning a frame-shaped answer for a field would be wrong in a way nothing here could see.
type RefSet ¶
type RefSet struct {
// contains filtered or unexported fields
}
RefSet is the set of pictures a stream currently holds as references.
Feed it every coded picture in decode order, after reading its slice header: it applies what that picture says about the pictures before it, and then takes the picture itself in if it is a reference. Pictures reports what a slice may predict from, which is what InitialRefLists wants.
⛔ The set is state, like the picture order counts, and for the same reason: a picture's marking speaks about what came before it. A caller asking which pictures a slice may use, without having walked the ones before it, is asking a question that has no answer.
func (*RefSet) AfterPicture ¶
func (s *RefSet) AfterPicture(u Unit, h SliceHeader, ref SliceReferences, p POC, sps SPS) (int, error)
AfterPicture applies one picture's marking and takes the picture in if it is a reference. It returns the handle the picture was given, or -1 when the picture was not kept.
⛔ The order is: apply what this picture SAYS about the others, then take this picture in. Doing it the other way round lets the sliding window throw out the picture being added, and lets operation one address the new picture by a number that was meant for an older one.
func (*RefSet) Pictures ¶
func (s *RefSet) Pictures() []RefPicture
Pictures is what a slice may predict from, in the order they were taken in. The ordering a slice actually uses is InitialRefLists'.
type RefWeight ¶
type RefWeight struct {
LumaStated bool
LumaWeight int32
LumaOffset int32
ChromaStated bool
// ChromaWeight and ChromaOffset are Cb then Cr.
ChromaWeight [2]int32
ChromaOffset [2]int32
}
RefWeight is the weighting of one reference picture.
type SPS ¶
type SPS struct {
Profile uint8
Level uint8 // level_idc, ten times the level: 41 is level 4.1
ID uint32
ChromaFormat uint8 // 0 monochrome, 1 4:2:0, 2 4:2:2, 3 4:4:4
SeparatePlanes bool
BitDepthLuma uint8
BitDepthChroma uint8
FrameMBSOnly bool
MBWidth uint32 // in macroblocks
MBHeight uint32 // in macroblocks, after the field doubling
CropLeft uint32 // in crop units, as the set states them
CropRight uint32
CropTop uint32
CropBottom uint32
Width uint32 // cropped, in luma samples
Height uint32
MaxNumRefFrames uint32
// What a slice header cannot be read without. A slice states its frame
// number and its picture order count in widths and shapes this set decides,
// so a reader holding only the picture size cannot walk one field of a slice.
Log2MaxFrameNum uint8
POCType uint32
Log2MaxPOCLSB uint8
POCAlwaysZero bool
}
SPS is what a sequence parameter set says about the pictures that follow.
Width and Height are the CROPPED size -- what a player shows and what a container states. The coded size is a whole number of macroblocks and is usually larger: 1080 lines are coded as 68 rows of 16 and cropped back by eight, which is why a reader that stopped before the cropping fields reports 1088 for most of the high-definition video there is.
type SliceHeader ¶
type SliceHeader struct {
FirstMB uint32
Type SliceType
AllOfType bool // slice_type was stated above 4: every slice of this picture is this type
PPSID uint32
ColourPlane uint8
FrameNum uint32
FieldPic bool
BottomField bool
IDR bool
IDRPicID uint32
POCLSB uint32
DeltaPOCBottom int32
DeltaPOC [2]int32
RedundantPicCnt uint32
}
func ParseSliceHeader ¶
func ParseSliceHeader(u Unit, sps SPS, pps PPS) (SliceHeader, error)
ParseSliceHeader reads the part of a slice header that says WHICH picture this slice belongs to: everything up to and including redundant_pic_cnt.
That is what access unit segmentation needs, and no more. What a slice says about the pictures it predicts from is read by ParseSliceReferences, which must not be made a condition of knowing where a picture begins.
func (SliceHeader) BeginsPicture ¶
func (h SliceHeader) BeginsPicture() bool
BeginsPicture says whether this slice starts a new coded picture.
type SliceReferences ¶
type SliceReferences struct {
// DirectSpatial is direct_spatial_mv_pred_flag, stated by a B slice only. It
// decides where a direct-mode block takes its motion from, and the two
// answers are not close: spatial reads the neighbours, temporal scales the
// motion of the picture that follows.
DirectSpatial bool
// NumRefIdxL0Active and NumRefIdxL1Active are how many references this slice
// uses, after any override it states. They come from the picture set when the
// slice overrides nothing, which is why they are here rather than left to the
// caller to work out.
NumRefIdxL0Active uint32
NumRefIdxL1Active uint32
// ModifyL0 and ModifyL1 are the reference list instructions, in order. Empty
// means the default order, which is not the same as "no references".
ModifyL0 []RefListOp
ModifyL1 []RefListOp
// Weights are the prediction weights, when the picture set asks this slice to
// state them explicitly. Nil when it does not -- which includes implicit
// weighting, where the weights come from the picture order counts and no
// table is sent.
Weights *PredWeights
// NoOutputOfPriorPics and LongTermReference are what an IDR states about the
// pictures before it.
NoOutputOfPriorPics bool
LongTermReference bool
// AdaptiveMarking is adaptive_ref_pic_marking_mode_flag: the slice says which
// pictures to release rather than letting the sliding window decide.
AdaptiveMarking bool
// Marking is what it says, in order.
Marking []MarkOp
// ResetsPOC reports that one of those operations is number five, which
// restarts the picture order count at this picture. A reader computing counts
// without honouring it is right until the first one and wrong afterwards.
ResetsPOC bool
// CABACInit is cabac_init_idc, which picks the context table a CABAC slice
// starts from. Only a CABAC slice that is not I states it.
CABACInit uint32
// QP is SliceQPY: the quantisation parameter this slice starts at, already
// built from the picture set's initial value and the slice's own delta.
//
// ⛔ It is also this reader's own witness. Every field before it has a width
// that depends on the fields before THAT, so a parse that drifted by one bit
// anywhere lands here with a number outside 0 to 51 -- which is how reading
// 1,344,537 slices of a real library told this syntax was read correctly, with
// no reference decoder to compare against.
QP int32
// DeblockingIDC is disable_deblocking_filter_idc, and the two offsets are the
// filter's own, as the slice states them.
DeblockingIDC uint32
AlphaC0OffsetDiv2 int32
BetaOffsetDiv2 int32
}
SliceReferences is what a slice says about the pictures it predicts from. It is a type of its own, and ParseSliceReferences the only thing that fills it, because a field read from a header that was never asked for it would be zero and look answered: a caller reading DirectSpatial off a plain ParseSliceHeader would be told "temporal" about every B slice in the stream.
type SliceType ¶
type SliceType uint8
SliceType is how a slice is coded.
type Unit ¶
type Unit struct {
Type UnitType
RefIDC uint8 // nal_ref_idc: how much a unit matters to what follows
Payload []byte
}
Unit is one NAL unit: its header and its payload, still escaped.
Payload is a window into the caller's bytes and is not copied. Unescape is what makes a readable copy, and it is separate because most units are walked past rather than read: a stream of a thousand slices costs one allocation per parameter set this way, not one per unit.
func SplitAnnexB ¶
SplitAnnexB finds the NAL units of a byte stream, the form a .264 file and an MPEG-TS stream carry, and reads each one's header.
The separating is bitstream's, which knows nothing about headers; the ONE byte of header is read here, because that is what differs between this format and H.265.
func SplitLengthPrefixed ¶
SplitLengthPrefixed finds the NAL units of the form an MP4 sample carries, where each is preceded by its length in lengthSize bytes, and reads each one's header.
lengthSize comes from the avcC record and is 1, 2 or 4 in practice.
type UnitType ¶
type UnitType uint8
UnitType is what a NAL unit holds. Only the ones this package acts on are named; the rest travel as their number.
type Weighting ¶
type Weighting struct {
// LogDenom is luma_log2_weight_denom, or 5 under implicit weighting.
LogDenom int32
// W0 and W1 are the weights of the two predictions, and O0 and O1 their
// offsets. A single prediction uses W0 and O0 alone.
W0, W1 int32
O0, O1 int32
}
Weighting is how a decoder combines one or two predictions into a block.
A sample is (w0*p0 + w1*p1 + ((o0+o1+1) << LogDenom)) >> (LogDenom+1) for a bi-predicted block, and (w0*p0 + (1 << (LogDenom-1))) >> LogDenom, plus o0, for a single prediction. The weights are held rather than applied here: this package reads and derives, and the arithmetic belongs where the samples are.
func ExplicitWeighting ¶
func ExplicitWeighting(w *PredWeights, idx0, idx1 int) Weighting
ExplicitWeighting is the weighting a slice's own table states for one pair of references, or for a single one.
⛔ An entry that states nothing is the DEFAULT weight, not a zero weight. A reader taking the zero value would multiply the prediction by nothing and show a grey block.
func ImplicitWeighting ¶
ImplicitWeighting derives the weights for a bi-predicted block when the picture set asks for implicit weighting -- weighted_bipred_idc of two, where no weight table is sent and the weights come from how far the current picture sits between its two references.
eitherLongTerm says whether either reference is a long-term picture. The counts of a long-term picture do not measure a distance in time, so the format does not scale by them.
⛔ The fallback is not an error path. The format falls back to a plain average whenever the scaled distance lands outside its own window, and a decoder that instead used the out-of-range number would weight one reference by more than everything -- visible, and on exactly the pictures that are furthest from their references.
func WeightingFor ¶
func WeightingFor(pps PPS, h SliceHeader, ref SliceReferences, currPOC, poc0, poc1 int32, idx0, idx1 int, eitherLongTerm bool) Weighting
WeightingFor gives the weighting a bi-predicted block uses, whichever mode the picture set asked for.
It is the question a decoder actually has, and asking it in one place keeps the three modes from being told apart differently in two.
func (Weighting) Trivial ¶
Trivial reports whether this weighting leaves the prediction alone: each weight is one, in its own denominator, and neither offset moves anything.
⛔ It is NOT the same as being equal to DefaultWeighting. A slice may state the identity weighting in any denominator it likes -- a weight of 1 against a denominator of 0, or 64 against 6, are both exactly one -- and a caller comparing structures calls those weighted. Measured: of 18 P slices in one real stream, 8 state the identity this way, and ffmpeg reports all 8 as unweighted while a comparison against DefaultWeighting reported all 8 as weighted.
A decoder uses it to skip the weighted path, which is the reason it is here rather than in whatever asked the question.
Source Files