vmaf

Documentation

Overview

Package vmaf describes VMAF measurements without depending on libvmaf: models and the resolution they are evaluated at (ModelSpec, ResolveModel, DeviceModel), extra feature extractors (Extractor), where the model features are extracted (Backend, ResolveBackend) and the contract of a scoring engine (Models, Scorer).

It needs no cgo, so that packages orchestrating measurements build without a C toolchain. The libvmaf binding, which implements Models and Scorer, lives in vmaf/libvmaf.

Index

• Constants
• Variables
• func CUDAUnsupported(spec ModelSpec) ([]string, error)
• func DefaultModelDirs() []string
• func DeviceModel(device string, fps float64) (string, error)
• func Devices() []string
• type Backend
  • func ParseBackend(s string) (Backend, error)
  • func (b Backend) String() string
• type BackendChoice
  • func ResolveBackend(requested Backend, specs []ModelSpec, initCUDA func() error) (BackendChoice, error)
• type Extractor
• type ModelSpec
  • func ResolveModel(name string, sourceHeight int, fps float64, dirs []string) (ModelSpec, error)
• type Models
• type Scorer
• type ScorerConfig
• type Scores

Constants

const (
	// DevicePhone is a phone held at 5 picture heights (vmaf_v1.0.16_5d0h).
	DevicePhone = "phone"
	// DeviceTV is a 1080p TV at 3 picture heights (vmaf_v1.0.16_3d0h), the
	// default model.
	DeviceTV = "tv"
	// Device4K is a 4K TV at 3 picture heights (vmaf_v1.0.16_3d0h_2160),
	// evaluated at 2160p.
	Device4K = "4k"
)

Devices: the viewing conditions VMAF v1 has a model for.

const (
	FeaturePSNRY     = "psnr_y"
	FeaturePSNRCb    = "psnr_cb"
	FeaturePSNRCr    = "psnr_cr"
	FeaturePSNRHVS   = "psnr_hvs"
	FeatureSSIM      = "float_ssim"
	FeatureMSSSIM    = "float_ms_ssim"
	FeatureCIEDE2000 = "ciede2000"
	FeatureCAMBI     = "cambi"
)

Feature outputs read back per frame, by name in Scores.Features.

Variables

var (
	// ErrBackend is returned for an unknown backend name.
	ErrBackend = errors.New("unknown backend")
	// ErrCUDAUnavailable is returned when CUDA is requested from a binary
	// built without the cuda tag.
	ErrCUDAUnavailable = errors.New("built without CUDA support (build with -tags cuda against a CUDA-enabled libvmaf)")
	// ErrCUDAInit is returned when the CUDA driver or device cannot be
	// initialised: no NVIDIA GPU, no driver, or a container started
	// without --gpus.
	ErrCUDAInit = errors.New("CUDA initialisation failed (no usable NVIDIA GPU or driver)")
	// ErrCUDAModel is returned when a model needs features libvmaf only
	// extracts on the CPU.
	ErrCUDAModel = errors.New("model features have no CUDA extractor")
	// ErrModelFeatures is returned when libvmaf lacks an extractor of the
	// model's features: VMAF v1 needs speed_chroma, which libvmaf 3.2.0
	// only builds with -Denable_float=true (Homebrew's 3.2.0 bottle has
	// none); 3.2.1 always builds it.
	ErrModelFeatures = errors.New("libvmaf lacks an extractor of the model's features (VMAF v1 needs libvmaf ≥ 3.2.1, or 3.2.0 built with -Denable_float=true; or pick --model vmaf_v0.6.1)")
)

var (
	// ErrModelNotFound is returned when a model name cannot be resolved.
	ErrModelNotFound = errors.New("model not found")
	// ErrUnknownDevice is returned for a device without a VMAF v1 model.
	ErrUnknownDevice = errors.New("unknown device")
)

var ErrUnknownExtractor = errors.New("unknown extractor")

ErrUnknownExtractor is returned for an extractor a scoring engine does not know how to read back.

Functions

func CUDAUnsupported

func CUDAUnsupported(
	spec ModelSpec,
) ([]string, error)

CUDAUnsupported returns the features of the model that libvmaf cannot extract on CUDA, nil when the whole model runs on the GPU. With CUDA enabled, libvmaf looks every model feature up among its CUDA extractors only, so a single missing one makes the model fail to load: VMAF v1 (cambi, speed_chroma_uv, adm3, motion3) cannot run on CUDA at all.

func DefaultModelDirs

func DefaultModelDirs() []string

DefaultModelDirs returns the directories searched for model JSON files by default, in order: libvmaf's install locations (Homebrew, /usr/local, /usr).

func DeviceModel

func DeviceModel(
	device string,
	fps float64,
) (string, error)

DeviceModel returns the VMAF v1 model of a device, in its high frame rate variant above 30 fps (like the automatic model).

func Devices

func Devices() []string

Devices returns the supported devices, in display order.

Types

type Backend

type Backend string

Backend selects where the features of the models are extracted.

const (
	// BackendCPU extracts every feature on the CPU (the zero value).
	BackendCPU Backend = ""
	// BackendCUDA extracts the model features on an NVIDIA GPU. It needs a
	// binary built with the cuda tag (see vmaf/libvmaf) against a libvmaf
	// built with
	// -Denable_cuda=true, and models whose features all have a CUDA
	// extractor (see CUDAUnsupported). Extra extractors (PSNR, CAMBI...)
	// stay on the CPU, in the same context.
	BackendCUDA Backend = "cuda"
	// BackendAuto is BackendCUDA when it can score the models, BackendCPU
	// otherwise (see ResolveBackend). Scorers only take CPU or CUDA.
	BackendAuto Backend = "auto"
)

Backends.

func ParseBackend

func ParseBackend(
	s string,
) (Backend, error)

ParseBackend reads a backend name: cpu (or empty), cuda or auto.

func (Backend) String

func (b Backend) String() string

String returns the backend name, "cpu" for the zero value.

type BackendChoice

type BackendChoice struct {
	Backend Backend
	// Reason explains a fallback from BackendAuto to the CPU.
	Reason string
}

BackendChoice is the backend a measurement runs on, and why it differs from the requested one.

func ResolveBackend

func ResolveBackend(
	requested Backend,
	specs []ModelSpec,
	initCUDA func() error,
) (BackendChoice, error)

ResolveBackend picks the backend scoring specs. BackendCUDA fails when CUDA cannot score them; BackendAuto then falls back to the CPU and says why. initCUDA initialises the device (libvmaf.InitCUDA): it is only called once every model has CUDA features, so a CPU-only model never touches the GPU.

type Extractor

type Extractor string

Extractor is a libvmaf feature extractor measured next to the model features, on the same pictures.

const (
	// ExtractorPSNR measures PSNR per plane (FeaturePSNRY, Cb, Cr).
	ExtractorPSNR Extractor = "psnr"
	// ExtractorPSNRHVS measures PSNR-HVS (FeaturePSNRHVS).
	ExtractorPSNRHVS Extractor = "psnr_hvs"
	// ExtractorSSIM measures SSIM on luma (FeatureSSIM).
	ExtractorSSIM Extractor = "float_ssim"
	// ExtractorMSSSIM measures multi-scale SSIM on luma (FeatureMSSSIM).
	ExtractorMSSSIM Extractor = "float_ms_ssim"
	// ExtractorCIEDE2000 measures the CIEDE2000 colour difference, as a
	// dB-like score (FeatureCIEDE2000).
	ExtractorCIEDE2000 Extractor = "ciede"
	// ExtractorCAMBI measures banding (FeatureCAMBI) with the options of
	// the VMAF v1 models: free when a v1 model is scored.
	ExtractorCAMBI Extractor = "cambi"
)

Extractors of the libvmaf feature set.

type ModelSpec

type ModelSpec struct {
	Name string `json:"name"`
	// Source is a file path when FromPath is set, a built-in version otherwise.
	Source   string `json:"-"`
	FromPath bool   `json:"-"`
	Width    int    `json:"evalWidth"`
	Height   int    `json:"evalHeight"`
}

ModelSpec identifies a model and the resolution it must be evaluated at.

func ResolveModel

func ResolveModel(
	name string,
	sourceHeight int,
	fps float64,
	dirs []string,
) (ModelSpec, error)

ResolveModel picks the model to use. name may be "" or "auto" (VMAF v1: 4K model above 1080p, high frame rate variant above 30 fps), a model name found in dirs (e.g. "vmaf_v1.0.16_5d0h"), a path to a JSON model, or a libvmaf built-in version (e.g. "vmaf_v0.6.1").

type Models

type Models interface {
	// NewScorer returns a Scorer measuring every model and cfg.
	NewScorer(
		cfg ScorerConfig,
	) (Scorer, error)
	// Close releases the models.
	Close()
}

Models are VMAF models loaded once and shared by the scorers they create, so that scoring many short clips does not reload them. They must outlive their scorers and are not safe for concurrent use.

type Scorer

type Scorer interface {
	// Push scores the next pair. Frames must carry chroma and match the
	// configured geometry and bit depth; they are not retained.
	Push(
		ref, dist *frame.Frame,
	) error
	// Collect flushes the scorer and returns the values of every pushed
	// pair. It can only be called once.
	Collect() (Scores, error)
	// Close releases the scorer.
	Close()
}

Scorer computes per-frame VMAF, with one or more models, and extra features for one contiguous sequence of frame pairs.

type ScorerConfig

type ScorerConfig struct {
	// Extractors are measured in the same pass as the models.
	Extractors []Extractor
	// Width and Height are the geometry of the 4:2:0 frames.
	Width, Height int
	// BitDepth is 8, or 10 with 16-bit samples.
	BitDepth int
	// Threads is the engine's worker count.
	Threads int
	// Backend is where the model features are extracted: BackendCPU (the
	// zero value) or BackendCUDA. Resolve BackendAuto with ResolveBackend
	// first. Extractors always run on the CPU.
	Backend Backend
}

ScorerConfig describes what a Scorer measures next to its models.

type Scores

type Scores struct {
	// VMAF holds one score per pushed pair for each model, in the order
	// the models were loaded.
	VMAF [][]float64
	// Features holds one value per pushed pair for each extractor output.
	Features map[string][]float64
}

Scores are the per-frame results of a Scorer.