Documentation
¶
Overview ¶
Package quality measures the full-reference quality (VMAF) of a distorted video against its reference, as fast as the requested precision allows.
In sampled mode the timeline is split into strata (snapped to keyframes, a zero-cost proxy for shots) and short clips are scored in parallel until the confidence interval of the mean is narrower than the target precision, or, with a fixed budget (Options.Sample), until a share of the frames or a number of clips per scene is scored. Exact mode scores every frame.
Other metrics (XPSNR, CAMBI, PSNR, PSNR-HVS, SSIM, MS-SSIM, CIEDE2000) and the VMAF of other viewing devices are measured on the same decoded frames, in the same libvmaf contexts, and estimated from the same clips.
Index ¶
- Constants
- Variables
- func AV2CTCMetrics() []string
- func HeadlineSeries() []string
- func Metrics() []string
- func ParseDevices(names []string) ([]string, error)
- func ParseMetrics(names []string) ([]string, error)
- func RawSeries(res *Result, name string) ([]float64, bool)
- type Banding
- type BandingSegment
- type DeviceResult
- type Drops
- type Engine
- type Estimate
- type FrameScore
- type HDRMetric
- type HDRReport
- type Input
- type Meter
- type MetricResult
- type Options
- type Progress
- type Result
- type Sample
- type SampleReport
- type SeriesInfo
- type Simulation
- type StratumResult
Examples ¶
Constants ¶
const ( BoundariesKeyframes = "keyframes" BoundariesShots = "shots+keyframes" )
Stratum boundaries of a fixed budget (SampleReport.Boundaries).
const ( VariancePooled = "pooled" VarianceSeparate = "separate" VarianceCollapsed = "collapsed" )
Variance estimators of a sampled measurement (SampleReport.Variance).
const ( SeriesWPSNRY = "wpsnr_y" SeriesWPSNRCb = "wpsnr_cb" SeriesWPSNRCr = "wpsnr_cr" SeriesDeltaEITP = "deltae_itp" SeriesDeltaEITPP99 = "deltae_itp_p99" )
Series of the HDR metrics, measured on PQ and HLG references (see package quality/hdr).
const ( // MetricVMAF is always measured; naming it is allowed and changes nothing. MetricVMAF = "vmaf" // MetricXPSNR is XPSNR per plane (Fraunhofer HHI), computed in Go and // identical to ffmpeg's xpsnr filter. MetricXPSNR = "xpsnr" // MetricCAMBI is Netflix's banding index, with the options of the VMAF // v1 models (free when a v1 model is scored). MetricCAMBI = "cambi" // MetricPSNR is PSNR per plane and the AV2 CTC weighted PSNR-YUV. MetricPSNR = "psnr" // MetricPSNRHVS is PSNR-HVS (libvmaf). MetricPSNRHVS = "psnr-hvs" // MetricSSIM is SSIM on luma (libvmaf float_ssim). MetricSSIM = "ssim" // MetricMSSSIM is multi-scale SSIM on luma (libvmaf float_ms_ssim). MetricMSSSIM = "ms-ssim" // MetricCIEDE2000 is the CIEDE2000 colour difference (libvmaf), as a // dB-like score: higher is better. MetricCIEDE2000 = "ciede2000" )
Metrics measured next to VMAF, as named in Options.Metrics. Every metric is computed on the frames VMAF already decodes (the sampled clips, or every frame), at the evaluation resolution of the model.
const ( SeriesXPSNRY = "xpsnr_y" SeriesXPSNRU = "xpsnr_u" SeriesXPSNRV = "xpsnr_v" SeriesCAMBI = "cambi" // SeriesCAMBISource is CAMBI on the reference frame, measured on the // banded frames only (FrameScore.Metrics): the banding the encode // inherited. SeriesCAMBISource = "cambi_source" SeriesPSNRY = "psnr_y" SeriesPSNRCb = "psnr_cb" SeriesPSNRCr = "psnr_cr" SeriesPSNRYUV = "psnr_yuv" SeriesPSNRHVS = "psnr_hvs" SeriesSSIM = "ssim" SeriesMSSSIM = "ms_ssim" SeriesCIEDE2000 = "ciede2000" )
Series names: the keys of MetricResult.Name and FrameScore.Metrics.
const ( ModeSampled = "sampled" ModeExact = "exact" )
Modes.
const ( // BandingThreshold is the CAMBI score above which banding is visible // (Netflix: below 5 it is imperceptible). BandingThreshold = 5.0 )
const DropMargin = 10.0
DropMargin is how far under the mean (VMAF) a frame must score to count as a drop: more than a quality step of a ladder, clearly visible.
Variables ¶
var ( // ErrUnknownMetric is returned for a metric name not in Metrics. ErrUnknownMetric = errors.New("unknown metric") // ErrUnknownDevice is returned for a device not in vmaf.Devices. ErrUnknownDevice = errors.New("unknown device") )
var ( // ErrFrameRateMismatch is returned when the two videos do not share a // frame rate: frames would be paired with the wrong counterpart. ErrFrameRateMismatch = errors.New("reference and distorted frame rates differ") // ErrNoFramesToCompare is returned when an input has no bitstream // report or no frame to score. ErrNoFramesToCompare = errors.New("no frames to compare") )
var ErrInvalidSample = errors.New("invalid sample budget")
ErrInvalidSample is returned for a malformed fixed sampling budget.
var ErrUnknownHDRMetric = errors.New("unknown HDR metric")
ErrUnknownHDRMetric is returned for an HDR metric mode other than pq and tonemap.
Functions ¶
func AV2CTCMetrics ¶
func AV2CTCMetrics() []string
AV2CTCMetrics returns the metric set of the AOM AV2 common test conditions (arXiv:2605.15800): PSNR per plane and weighted PSNR-YUV, PSNR-HVS, SSIM, MS-SSIM, CIEDE2000, VMAF and CAMBI, all computed by libvmaf.
func HeadlineSeries ¶
func HeadlineSeries() []string
HeadlineSeries returns the series that sum a metric up in one number, in report order: the luma series of per-plane metrics. Reports short on room, such as the rungs of a ladder, show only these.
func Metrics ¶
func Metrics() []string
Metrics returns the metric names, in report order (VMAF first).
func ParseDevices ¶
ParseDevices validates device names (case-insensitive) and returns them without duplicates, in display order.
func ParseMetrics ¶
ParseMetrics validates metric names (case-insensitive) and returns them without duplicates, in report order. "vmaf" is dropped: it is always measured.
Example ¶
Metrics other than VMAF and the VMAF of viewing devices are chosen by name; ParseMetrics validates them early.
package main
import (
"fmt"
"github.com/eko/qc/quality"
)
func main() {
metrics, err := quality.ParseMetrics([]string{"XPSNR", "cambi"})
fmt.Println(metrics, err)
_, err = quality.ParseMetrics([]string{"ssimulacra2"})
fmt.Println(err)
}
Output: [xpsnr cambi] <nil> unknown metric "ssimulacra2" (supported: vmaf, xpsnr, cambi, psnr, psnr-hvs, ssim, ms-ssim, ciede2000)
func RawSeries ¶
RawSeries returns the per-frame raw values of a series of a result: the quantity whose mean is estimated. It is the reported value, except for XPSNR whose distortion √WSSE is recovered from its dB value (exact below the maxDecibels cap). ok is false when no frame carries the series.
Types ¶
type Banding ¶
type Banding struct {
Threshold float64 `json:"threshold"`
BandedFrames int `json:"bandedFrames"`
// SourceFrames counts the banded frames whose reference frame is
// banded too (SeriesCAMBISource above the threshold): banding the
// encode inherited rather than made.
SourceFrames int `json:"sourceFrames,omitempty"`
Segments []BandingSegment `json:"segments,omitempty"`
}
Banding reports where CAMBI exceeds BandingThreshold among the scored frames. In sampled mode, unscored frames are not known to be clean.
type BandingSegment ¶
type BandingSegment struct {
media.Interval
Frames int `json:"frames"`
Mean float64 `json:"mean"`
Peak float64 `json:"peak"`
}
BandingSegment is a run of banded scored frames.
type DeviceResult ¶
type DeviceResult struct {
Device string `json:"device"`
Model vmaf.ModelSpec `json:"model"`
Estimate
Scored stats.Summary `json:"scored"`
}
DeviceResult is the VMAF of one viewing condition, with its own model.
type Drops ¶ added in v1.2.0
type Drops struct {
Margin float64 `json:"margin"`
// Threshold is the score under which a frame is a drop.
Threshold float64 `json:"threshold"`
Low float64 `json:"low"`
High float64 `json:"high"`
}
Drops is the share of the frames scoring more than Margin under the mean of the video. In sampled mode it is estimated like the mean, with its interval (Low and High equal Share in exact mode).
type Engine ¶
type Engine interface {
// ResolveBackend picks where the features of specs are extracted (see
// vmaf.ResolveBackend).
ResolveBackend(
requested vmaf.Backend,
specs []vmaf.ModelSpec,
) (vmaf.BackendChoice, error)
// LoadModels loads specs once per worker: each clip the worker scores
// then only creates a scorer.
LoadModels(
specs []vmaf.ModelSpec,
) (vmaf.Models, error)
}
Engine is the VMAF implementation a Meter scores with (libvmaf.Engine, which needs cgo). Keeping it behind this port lets the measurement logic build and be tested without libvmaf.
type Estimate ¶
type Estimate struct {
Mean float64 `json:"mean"`
Low float64 `json:"low"`
High float64 `json:"high"`
HalfWidth float64 `json:"halfWidth"`
}
Estimate is a mean with its confidence interval. Low and High are equal to Mean in exact mode.
type FrameScore ¶
type FrameScore struct {
Index int `json:"index"`
PTS media.Duration `json:"pts"`
Score float64 `json:"score"`
Metrics map[string]float64 `json:"metrics,omitempty"`
}
FrameScore is the VMAF of one frame, and the values of the other metrics and devices by series name (psnr_y, vmaf_phone...).
type HDRMetric ¶
type HDRMetric string
HDRMetric selects how VMAF is scored on an HDR (PQ or HLG) reference. VMAF's models were trained on SDR: there is no public HDR model.
const ( // HDRMetricPQ scores VMAF on the HDR signal as decoded (PQ or HLG code // values), the default: no extra work, fine for ranking encodes of one // title (a ladder), not calibrated as an absolute score on PQ. The HDR // metrics (wPSNR, ΔE ITP) are measured on the same frames. HDRMetricPQ HDRMetric = "pq" // HDRMetricToneMap scores VMAF on an SDR (BT.709) tone mapping of both // videos, done by ffmpeg while decoding: VMAF then sees what it was // trained on, as an SDR display would show the HDR picture. Slower: // tone mapping costs decoding time, and the HDR metrics need a second // decode of the scored clips. HDRMetricToneMap HDRMetric = "tonemap" )
HDR metric modes.
func ParseHDRMetric ¶
ParseHDRMetric reads an HDR metric mode: pq (or empty) or tonemap.
type HDRReport ¶
type HDRReport struct {
// Transfer is the reference's: media.TransferPQ or TransferHLG.
Transfer string `json:"transfer"`
// Metric is how VMAF was scored.
Metric HDRMetric `json:"metric"`
// Calibrated is false when VMAF was scored on the HDR signal, which its
// models were not trained for.
Calibrated bool `json:"vmafCalibrated"`
// Note explains in a sentence how to read VMAF here.
Note string `json:"note"`
}
HDRReport tells how a comparison with an HDR reference was measured.
type Input ¶
type Input struct {
Path string
Video media.VideoStream
Bitstream *bitstream.Report
}
Input is one side of the comparison.
type Meter ¶
type Meter struct {
// contains filtered or unexported fields
}
Meter computes VMAF between two videos.
type MetricResult ¶
type MetricResult struct {
Name string `json:"name"`
Estimate
// Scored summarises the per-frame values of the scored frames.
Scored stats.Summary `json:"scored"`
}
MetricResult is the measurement of one series of a metric (e.g. psnr_y).
type Options ¶
type Options struct {
// Model is a model name, JSON path or "auto" (see vmaf.ResolveModel).
Model string
ModelDirs []string
// Exact scores every frame instead of sampling. It takes precedence
// over Sample.
Exact bool
// Sample, when set, replaces the precision target by a fixed budget
// scored in a single round: Precision, MaxShare and Budget do not apply,
// and the result reports the interval the budget reaches.
Sample Sample
// Cuts are the shot cuts of the distorted video, as timestamps relative
// to its first frame (e.g. from its technical analysis). With a fixed
// budget they cut strata, next to its keyframes: scenes then follow real
// shots even when keyframes do not (fixed-GOP encodes).
Cuts []media.Duration
// Precision is the target half-width of the confidence interval of the
// mean, in VMAF points. Default 0.5.
Precision float64
// Confidence level of the interval. Default 0.95.
Confidence float64
// the precision would need more, every frame is scored instead (sampling
// then costs more than it saves). Default 0.4.
MaxShare float64
// Budget never falls back to exact scoring: when the precision would
// need more than MaxShare of the frames, sampling stops at that budget
// and reports the interval it reached.
Budget bool
// BitDepth is the depth frames are scored at: 8 or 10. 0 picks 10 when
// either video has more than 8 bits (VMAF v1 needs it to see banding).
BitDepth int
// ClipFrames is the number of frames of a sampled clip. Default 4.
ClipFrames int
// InitialClips sizes the first round on long videos: strata grow so that
// about this many clips are scored first. Default 120, enough for ±0.5
// when clip scores spread by 3 VMAF points within strata.
InitialClips int
// Workers is the number of clips scored concurrently (0 = auto).
Workers int
// Seed makes clip selection reproducible.
Seed uint64
// Metrics lists the metrics measured next to VMAF (see Metrics). They
// are estimated from the clips VMAF samples: VMAF alone drives the
// sampling and its stopping rule.
Metrics []string
// Backend is where libvmaf extracts the model features: the CPU (zero
// value), vmaf.BackendCUDA (an NVIDIA GPU; fails when the models or the
// build cannot) or vmaf.BackendAuto (the GPU when possible, the CPU
// otherwise, with the reason in Result.BackendNote). VMAF v1 models
// have no CUDA features in libvmaf 3.2.1 (see vmaf.CUDAUnsupported).
Backend vmaf.Backend
// Devices lists viewing conditions (vmaf.Devices) whose VMAF v1 model
// is scored next to the primary model. Models evaluated at the
// primary resolution share its libvmaf contexts; the others (4K
// against a 1080p primary, or the reverse) need a second pass on the
// same clips at their resolution.
Devices []string
// HDRMetric is how VMAF is scored on a PQ or HLG reference (default
// HDRMetricPQ). Such references also get the HDR metrics (wPSNR, ΔE
// ITP, see package quality/hdr) on the scored frames.
HDRMetric HDRMetric
// SkipHDRMetrics measures VMAF without the HDR metrics on an HDR
// reference, as the ladder's probes do: they only shape the curves.
SkipHDRMetrics bool
// Progress, when set, is called after each scored clip. Calls never
// overlap.
Progress func(Progress)
}
Options tunes the measurement. The zero value is valid.
type Progress ¶
type Progress struct {
FramesScored int
FramesTotal int
Round int
Estimated bool
Mean float64
HalfWidth float64
Mode string
// Sample is the fixed budget of the measurement, zero when sampling is
// driven by a precision.
Sample Sample
}
Progress reports the advancement of a measurement. Mean and HalfWidth are the running estimate, set once a sampling round completes (Estimated).
type Result ¶
type Result struct {
Model vmaf.ModelSpec `json:"model"`
BitDepth int `json:"bitDepth"`
Mode string `json:"mode"`
Mean float64 `json:"mean"`
Low float64 `json:"low"`
High float64 `json:"high"`
HalfWidth float64 `json:"halfWidth"`
Confidence float64 `json:"confidence"`
// HarmonicMean is the harmonic mean of the frame scores (libvmaf's
// convention), which low frames pull down more than they do the mean.
// In sampled mode it is estimated like the mean, from the same clips.
HarmonicMean float64 `json:"harmonicMean,omitempty"`
// Drops tells how much of the video scores well under its mean: what
// the mean hides.
Drops *Drops `json:"drops,omitempty"`
// Fallback explains why an exact measurement replaced sampling.
Fallback string `json:"fallback,omitempty"`
// Sample describes the fixed budget of a sampled measurement, nil when
// sampling was driven by a precision.
Sample *SampleReport `json:"sample,omitempty"`
// Plans counts the decoding plans used per round (sweep or seek runs).
Plans map[string]int `json:"plans,omitempty"`
// Scored summarises the frames actually scored. In sampled mode its low
// percentiles are estimates and may miss isolated bad frames.
Scored stats.Summary `json:"scored"`
FramesTotal int `json:"framesTotal"`
FramesScored int `json:"framesScored"`
FramesDecoded int `json:"framesDecoded"`
Rounds int `json:"rounds"`
Strata []StratumResult `json:"strata"`
Frames []FrameScore `json:"frames"`
// Metrics holds the other metrics, one entry per series (psnr_y...).
Metrics []MetricResult `json:"metrics,omitempty"`
// Devices holds the VMAF of each requested viewing device.
Devices []DeviceResult `json:"devices,omitempty"`
// Banding lists the banded segments when CAMBI is measured.
Banding *Banding `json:"banding,omitempty"`
// HDR tells how VMAF was scored on a PQ or HLG reference, and how to
// read it; nil for SDR.
HDR *HDRReport `json:"hdr,omitempty"`
// Backend is "cuda" when the model features were extracted on an
// NVIDIA GPU, empty on the CPU.
Backend string `json:"backend,omitempty"`
// BackendNote explains why a GPU measurement ran on the CPU instead.
BackendNote string `json:"backendNote,omitempty"`
// HWAccel is the hardware decoding mode of the decoder ("cuda",
// "cuda-scale", "videotoolbox"), empty on the CPU. Files the hardware
// cannot decode still fall back to the CPU.
HWAccel string `json:"hwaccel,omitempty"`
Elapsed media.Duration `json:"elapsed"`
}
Result is a VMAF measurement.
func (*Result) Device ¶
func (r *Result) Device( name string, ) (DeviceResult, bool)
Device returns the VMAF of a device (e.g. vmaf.DevicePhone), if it was measured.
func (*Result) GPUSummary ¶
GPUSummary says in a few words what ran on a GPU: NVDEC or VideoToolbox decoding, CUDA feature extraction, or a CUDA request that fell back to the CPU (BackendNote says why). It is empty for a CPU-only measurement.
type Sample ¶
type Sample struct {
Share float64 `json:"share,omitempty"`
// PerScene is the number of clips scored in every scene.
PerScene int `json:"perScene,omitempty"`
}
Sample is a fixed sampling budget: a share of the frames, or a number of clips in every scene, scored in a single round whatever precision it reaches. The zero value selects precision-driven sampling.
func ParseSample ¶
ParseSample reads a budget: a share of the frames ("5%", "0.5%") or a number of clips per scene ("2/scene", or "2-per-scene"). An empty string is the zero Sample.
Example ¶
A fixed budget replaces the precision target: a share of the frames, or a number of clips in every scene. The CLI's --sample flag takes the same syntax.
package main
import (
"fmt"
"github.com/eko/qc/quality"
)
func main() {
for _, s := range []string{"5%", "2/scene"} {
budget, err := quality.ParseSample(s)
if err != nil {
fmt.Println(err)
continue
}
fmt.Printf("%s: share %.2f, %d per scene\n", budget, budget.Share, budget.PerScene)
}
_, err := quality.ParseSample("0%")
fmt.Println(err != nil)
}
Output: 5%: share 0.05, 0 per scene 2/scene: share 0.00, 2 per scene true
type SampleReport ¶
type SampleReport struct {
Sample
// Clips is the number of clips scored.
Clips int `json:"clips"`
// Boundaries tells what strata follow: the keyframes of the distorted
// stream, or those and the detected shot cuts (Options.Cuts).
Boundaries string `json:"boundaries"`
// Variance is the estimator of the interval: the within-stratum
// variance pooled over strata (share budgets), each scene's own variance
// (per-scene budgets), or collapsed strata when each holds a single clip.
Variance string `json:"variance"`
// Clamped explains how the budget was adapted to the video.
Clamped string `json:"clamped,omitempty"`
}
SampleReport describes the fixed budget a measurement spent.
func (SampleReport) Summary ¶
func (r SampleReport) Summary() string
Summary describes the budget in a few words for reports: "5% of frames · 199 clips", "2/scene (keyframes) · 482 clips".
type SeriesInfo ¶
type SeriesInfo struct {
// Label is the display name.
Label string
// Unit is "dB" for decibel-like scores, "" otherwise.
Unit string
// LowerIsBetter is set for CAMBI: its worst frames are the highest.
LowerIsBetter bool
// Decimals is the precision worth displaying.
Decimals int
}
SeriesInfo tells how to read a series.
func DescribeSeries ¶
func DescribeSeries( name string, ) SeriesInfo
DescribeSeries returns how to read a series; unknown names are shown as is.
func (SeriesInfo) Format ¶
func (i SeriesInfo) Format( v float64, ) string
Format prints a value at the precision of the series.
func (SeriesInfo) ShortLabel ¶
func (i SeriesInfo) ShortLabel() string
ShortLabel is the label without its parenthesised hint ("CAMBI (banding)" is "CAMBI"), for column headers.
type Simulation ¶
type Simulation struct {
Runs int `json:"runs"`
True float64 `json:"true"`
RMSE float64 `json:"rmse"`
// MeanAbsErr is the mean absolute error of the estimate.
MeanAbsErr float64 `json:"meanAbsError"`
MeanHalf float64 `json:"meanHalfWidth"`
Coverage float64 `json:"coverage"`
// SampledCoverage excludes runs that fell back to exact scoring.
SampledCoverage float64 `json:"sampledCoverage"`
Fallbacks int `json:"fallbacks"`
Strata int `json:"strata"`
}
Simulation summarises repeated sampled measurements replayed on known per-frame scores: it checks that the confidence intervals are honest.
func Simulate ¶
func Simulate( scores []float64, pts, keyframes []media.Duration, opts Options, runs int, ) Simulation
Simulate replays the production sampling loop runs times (different seeds) on exact per-frame scores, with strata built from the distorted keyframes (or opts.Cuts with a fixed budget, as in a real measurement). Runs that fall back to exact scoring count as exact (share 1, no error). The share of frames counts scored frames only, not the warm-up frames a real measurement also decodes and scores.
func SimulateSeries ¶
func SimulateSeries( scores []float64, series map[string][]float64, pts, keyframes []media.Duration, opts Options, runs int, ) (Simulation, map[string]Simulation)
SimulateSeries is Simulate with other per-frame series (e.g. XPSNR or PSNR, as returned by RawSeries) measured on the clips the VMAF scores select, as in a real measurement. It also returns, per series, how its estimate and interval behave: VMAF alone drives the sampling, so the other series get no precision target, but their intervals must still cover their true mean ~95% of the time.
Source Files
¶
Directories
¶
| Path | Synopsis |
|---|---|
|
Package hdr measures the fidelity of HDR (PQ or HLG) video on decoded 4:2:0 frames, in pure Go, with two metrics of the HDR literature:
|
Package hdr measures the fidelity of HDR (PQ or HLG) video on decoded 4:2:0 frames, in pure Go, with two metrics of the HDR literature: |
|
Package xpsnr computes XPSNR, the extended perceptually weighted PSNR of Fraunhofer HHI (Helmrich et al.), on decoded 4:2:0 frames.
|
Package xpsnr computes XPSNR, the extended perceptually weighted PSNR of Fraunhofer HHI (Helmrich et al.), on decoded 4:2:0 frames. |