Documentation
¶
Overview ¶
Package hwmediacodec drives the operating system's hardware video decode and encode engines, and its audio codecs, from Go without cgo.
Native libraries are loaded at run time (dlopen on macOS, iOS, Linux and Android, LoadLibrary on Windows), so CGO_ENABLED=0 builds and cross-compilation work on the desktop platforms. On iOS and Android purego needs the cgo runtime, so mobile builds (as made by ebitenmobile) set CGO_ENABLED=1 with the platform's clang; the library itself still contains no cgo code. A platform without a usable engine reports an empty Probe result and ErrUnsupported from NewDecoder and NewEncoder; it never fails at link time.
In a browser (GOOS=js GOARCH=wasm, the target Ebitengine builds for the web) the engine is the WebCodecs API, with the platform's codecs behind it. Its results arrive on the page's event loop, which a WebAssembly program shares: Send blocks while the browser's codec is saturated and Flush until everything is delivered, and that is when the frames and packets Receive returns come in. A loop that only polls Receive, without ever blocking, starves the browser; the loops below do not.
Status ¶
H.264 and HEVC decoding and encoding are implemented on macOS (Apple Silicon) and iOS through VideoToolbox, on Android through the NDK MediaCodec API (the vendor's hardware codecs), on Linux and Windows (amd64) with the NVIDIA driver through NVDEC and NVENC, on Linux with an Intel GPU through Intel VPL (a VPL or Media SDK runtime), on Linux through VA-API (AMD Mesa and Intel drivers), and on Windows (amd64 and arm64) through Media Foundation (decoding with the Microsoft decoder transforms accelerated by Direct3D 11, encoding with the vendor's hardware encoder transforms), and in browsers through WebCodecs (the codecs the browser reports as hardware ones; RGBA and BGRA output converted by the browser, and the Native format for frames that stay on the GPU). Where the NVIDIA driver is installed its backend is tried first. Raw frames are NV12, RGBA or BGRA in CPU memory. VideoToolbox converts to and from the RGB formats itself and NVENC takes them as input; everywhere else the library converts in Go, so both formats work for decoding and encoding on every backend. Decoded frames are returned in display order (see NewDecoder and WithDecodeOrder; the Media Foundation and Intel VPL decoders and the browsers' reorder natively, so WithDecodeOrder has no effect there). AV1 decoding is available where the GPU has an AV1 decoder: on macOS with an Apple M3 or newer and on iOS from the A17 Pro on (8-bit and 10-bit Main profile), and on Linux and Windows through every backend (8-bit Main profile; NVIDIA RTX 30 series, Intel Tiger Lake, AMD RX 6000 series and newer; Media Foundation needs the "AV1 Video Extension" package), on Android where the SoC has an AV1 decoder, and in a browser that reports a hardware AV1 decoder. A Packet is then one temporal unit (an IVF frame or ISOBMFF sample; see package mediacontainer/ivf), each yielding one frame in presentation order. AV1 encoding is available through VA-API on Linux (8-bit Main profile, GPUs with an AV1 encoder such as the AMD RX 7000 series and Radeon 700M/800M); the other backends do not encode AV1 yet. The separate module github.com/shibukawa/hwmediacodec/ebitenvideo plays streams in Ebitengine.
AAC and Opus audio is decoded and encoded through the codecs of the platform (see NewAudioDecoder and NewAudioEncoder): AudioToolbox on macOS and iOS, MediaCodec on Android, Media Foundation on Windows (which has no Opus encoder), WebCodecs in the browser as far as the browser has the codecs, and on Linux, which has no codecs of its own, the libraries of the distribution where they are installed (libopus, libavcodec for AAC decoding, libfdk-aac for AAC encoding). They run in software; the library contains no codec and bundles none. HWMEDIACODEC_AUDIO_BACKENDS restricts and reorders the audio backends ("libopus", "libavcodec", "fdk-aac" on Linux) the way HWMEDIACODEC_BACKENDS does the video ones.
On Linux the VA-API backend opens the first usable DRM render node and the Intel VPL backend the first Intel one; set HWMEDIACODEC_VAAPI_DEVICE or HWMEDIACODEC_VPL_DEVICE to a /dev/dri/renderD* path to choose a GPU. The NVIDIA backend uses CUDA device 0; set HWMEDIACODEC_NVIDIA_DEVICE to another device ordinal on machines with several NVIDIA GPUs.
When several backends can serve a request the first registered one wins (on Linux: nvidia, vpl, vaapi; on Windows: nvidia, mediafoundation; one backend each on macOS, iOS, Android and in the browser); set HWMEDIACODEC_BACKENDS to a comma-separated list of backend names, for example "vaapi" or "vaapi,vpl", to restrict and reorder them.
Decoding ¶
dec, err := hwmediacodec.NewDecoder(ctx, hwmediacodec.H264)
...
r := annexb.NewReader(file, hwmediacodec.H264)
for {
au, err := r.Next()
if err == io.EOF { break }
if err := dec.Send(ctx, hwmediacodec.Packet{Data: au}); err != nil { ... }
for {
f, err := dec.Receive(ctx)
if errors.Is(err, hwmediacodec.ErrAgain) { break }
... use f ...
f.Release()
}
}
dec.Flush(ctx)
// Receive until io.EOF, then Close.
Encoding ¶
enc, err := hwmediacodec.NewEncoder(ctx, hwmediacodec.H264, 1920, 1080,
hwmediacodec.WithFrameRate(30), hwmediacodec.WithBitrate(4_000_000))
...
for i, f := range frames { // *hwmediacodec.Frame in NV12
f.PTS = int64(i) * int64(hwmediacodec.DefaultTimeScale) / 30
if err := enc.Send(ctx, f); err != nil { ... }
for {
p, err := enc.Receive(ctx)
if errors.Is(err, hwmediacodec.ErrAgain) { break }
out.Write(p.Data) // Annex-B access unit
}
}
enc.Flush(ctx)
// Receive until io.EOF, then Close.
Audio ¶
dec, err := hwmediacodec.NewAudioDecoder(ctx, track.Format()) // *mp4.AudioTrack
...
if err := dec.Send(ctx, packet); err != nil { ... }
f, err := dec.Receive(ctx) // interleaved samples in f.Data, see f.Int16s
enc, err := hwmediacodec.NewAudioEncoder(ctx, hwmediacodec.AAC, 48000, 2,
hwmediacodec.WithBitrate(128_000))
...
if err := enc.Send(ctx, hwmediacodec.S16Frame(samples, 48000, 2, pts)); err != nil { ... }
p, err := enc.Receive(ctx) // one raw AAC access unit; enc.Format() describes the stream
Index ¶
- Constants
- Variables
- func CopyFrames(ctx context.Context, dst FrameWriter, src FrameReader) (int64, error)
- func HasHardware(ctx context.Context, c Codec, dir Direction) bool
- type AudioDecoder
- type AudioDecoderOption
- type AudioEncoder
- type AudioEncoderOption
- type AudioFormat
- type AudioFrame
- type AudioOption
- type BackendError
- type Capability
- type Codec
- type DecodeReader
- type Decoder
- type DecoderOption
- type Direction
- type EncodeWriter
- type Encoder
- type EncoderOption
- func WithBFrames() EncoderOption
- func WithFrameRate(fps float64) EncoderOption
- func WithInputFormat(f PixelFormat) EncoderOption
- func WithKeyframeInterval(frames int) EncoderOption
- func WithLowLatency() EncoderOption
- func WithProfile(p Profile) EncoderOption
- func WithQuality(q float64) EncoderOption
- type Frame
- type FrameReader
- type FrameWriter
- type Option
- type Packet
- type PacketReader
- type PacketSeeker
- type PacketWriteCloser
- type PacketWriter
- type PacketWriterFunc
- type PixelFormat
- type Profile
- type RateControl
- type SampleFormat
- type SharedEncoderOption
- type UnsupportedError
Constants ¶
const ( H264 = codec.H264 HEVC = codec.HEVC AV1 = codec.AV1 Decode = codec.Decode Encode = codec.Encode NV12 = codec.NV12 RGBA = codec.RGBA BGRA = codec.BGRA Native = codec.Native VBR = codec.VBR CBR = codec.CBR ProfileDefault = codec.ProfileDefault ProfileBaseline = codec.ProfileBaseline ProfileMain = codec.ProfileMain ProfileHigh = codec.ProfileHigh )
const DefaultTimeScale int32 = 90000
DefaultTimeScale is the PTS unit used when WithTimeScale is not given.
Variables ¶
var ( ErrUnsupported = codec.ErrUnsupported ErrAgain = codec.ErrAgain ErrClosed = codec.ErrClosed ErrInvalidData = codec.ErrInvalidData )
Functions ¶
func CopyFrames ¶
func CopyFrames(ctx context.Context, dst FrameWriter, src FrameReader) (int64, error)
CopyFrames writes every frame of src to dst and releases it, until src reports io.EOF, and returns the number of frames copied. With a DecodeReader and an EncodeWriter this is a transcoder; Flush the EncodeWriter afterwards.
func HasHardware ¶
HasHardware reports whether Probe lists a hardware engine for codec c in the given direction on this machine, that is, whether NewDecoder or NewEncoder can succeed without WithSoftwareFallback. Use it to choose another path (a software codec, another format) before trying. It is false for the audio codecs, which never run in hardware.
Types ¶
type AudioDecoder ¶
type AudioDecoder = codec.AudioDecoder
Re-exported audio types. See the documentation on each in package codec.
func NewAudioDecoder ¶
func NewAudioDecoder(ctx context.Context, format AudioFormat, opts ...AudioDecoderOption) (AudioDecoder, error)
NewAudioDecoder opens a decoder for the stream format describes on the first backend that supports it. A container track reports its format (mp4.AudioTrack.Format, adts.Reader.Format) and an AudioEncoder reports the one of its output; for a stream from elsewhere fill in the codec, the sample rate and the channel count. The error matches ErrUnsupported (and is an *UnsupportedError) when no backend can serve the request.
The codecs are the operating system's own, which run in software on every platform: WithSoftwareFallback is not needed, and nothing is bundled with the library.
Every packet yields one frame of interleaved samples (S16 unless WithSampleFormat says otherwise) that carries the packet's PTS. The decoder returns all the samples a stream codes: the AudioFormat.Delay samples at its start, which lie before PTS 0 in a stream that starts there, and the padding of its last packet are for the caller to drop.
type AudioDecoderOption ¶
type AudioDecoderOption = codec.AudioDecoderOption
Re-exported audio types. See the documentation on each in package codec.
type AudioEncoder ¶
type AudioEncoder = codec.AudioEncoder
Re-exported audio types. See the documentation on each in package codec.
func NewAudioEncoder ¶
func NewAudioEncoder(ctx context.Context, c Codec, sampleRate, channels int, opts ...AudioEncoderOption) (AudioEncoder, error)
NewAudioEncoder opens an encoder for c that takes interleaved samples at the given rate and channel count (S16 unless WithSampleFormat says otherwise) on the first backend that supports it. The error matches ErrUnsupported (and is an *UnsupportedError) when no backend can serve the request, including a sample rate or a bitrate the codec does not offer.
Without options the encoder uses the backend's default bitrate. PTS and Packet.Duration count in samples (the time scale is the sample rate) unless WithTimeScale sets another unit. The first packets of a stream have a PTS before that of the first frame: an encoder starts AudioFormat.Delay samples early (see AudioEncoder.Format).
type AudioEncoderOption ¶
type AudioEncoderOption = codec.AudioEncoderOption
Re-exported audio types. See the documentation on each in package codec.
type AudioFormat ¶
type AudioFormat = codec.AudioFormat
Re-exported audio types. See the documentation on each in package codec.
type AudioFrame ¶
type AudioFrame = codec.AudioFrame
Re-exported audio types. See the documentation on each in package codec.
func F32Frame ¶
func F32Frame(samples []float32, sampleRate, channels int, pts int64) *AudioFrame
F32Frame wraps interleaved floating point samples as an encoder input frame without copying, for an encoder opened with WithSampleFormat(F32). The opposite direction is AudioFrame.Float32s.
func S16Frame ¶
func S16Frame(samples []int16, sampleRate, channels int, pts int64) *AudioFrame
S16Frame wraps interleaved 16-bit samples as an encoder input frame without copying: the frame shares the slice, which must stay untouched until AudioEncoder.Send has returned. The opposite direction is AudioFrame.Int16s.
type AudioOption ¶
type AudioOption = codec.AudioOption
Re-exported audio types. See the documentation on each in package codec.
func WithSampleFormat ¶
func WithSampleFormat(f SampleFormat) AudioOption
WithSampleFormat sets the format of the samples a decoder returns and an encoder is given: S16 (the default) or F32.
type BackendError ¶
type BackendError = codec.BackendError
Re-exported core types. See the documentation on each in package codec.
type Capability ¶
type Capability = codec.Capability
Re-exported core types. See the documentation on each in package codec.
func Probe ¶
func Probe(ctx context.Context) ([]Capability, error)
Probe reports every codec/direction pair the registered backends can serve on this machine: the hardware video engines first, then the audio codecs of the operating system (Capability.Hardware is false for those; test Codec.IsAudio to tell the two apart). It returns an empty (non-nil) slice, not an error, when nothing is present.
type DecodeReader ¶
type DecodeReader struct {
// contains filtered or unexported fields
}
DecodeReader couples a PacketReader to a Decoder and hands out the decoded frames one at a time: the Send/Receive loop of the Decoder documentation, written once. It does not own the decoder; Close that when done.
Open the decoder with WithTimeScale(src.TimeScale()) so that Frame.PTS is in the reader's units.
func NewDecodeReader ¶
func NewDecodeReader(dec Decoder, src PacketReader) *DecodeReader
NewDecodeReader returns a reader of the frames dec decodes from src.
type Decoder ¶
Re-exported core types. See the documentation on each in package codec.
func NewDecoder ¶
NewDecoder opens a hardware decoder for c on the first backend that supports it. The error matches ErrUnsupported (and is an *UnsupportedError) when no backend can serve the request.
Frames come back in display order: pictures are reordered by their picture order count, derived from the slice headers in Go, and held back no longer than the stream's reorder bound (num_reorder_frames in the SPS or, for H.264 streams without that VUI field, the DPB size of the level). WithDecodeOrder turns the reordering off.
type DecoderOption ¶
type DecoderOption = codec.DecoderOption
Re-exported core types. See the documentation on each in package codec.
func WithDecodeOrder ¶
func WithDecodeOrder() DecoderOption
WithDecodeOrder makes Receive return frames in decode order, as the backend produces them, instead of display order. It avoids the latency of the reorder buffer for streams without B-frames whose headers do not say so, and is what a transcoder that preserves timestamps wants.
func WithOutputFormat ¶
func WithOutputFormat(f PixelFormat) DecoderOption
WithOutputFormat requests a pixel format for decoded frames: NV12 (the default), RGBA or BGRA, or Native for frames that stay with the decoder (see Native; browsers only, ErrUnsupported elsewhere). RGBA and BGRA frames have one plane with rows of exactly 4*Width bytes, ready for ebiten.Image.WritePixels or image.RGBA. The backend converts where it can; for a backend that returns NV12 only the frames are converted in Go, with the colour matrix and range the stream declares (for a stream that declares none, BT.709 for pictures larger than 704x576 and BT.601 otherwise).
type EncodeWriter ¶
type EncodeWriter struct {
// contains filtered or unexported fields
}
EncodeWriter couples an Encoder to a PacketWriter: every frame written is encoded and the packets that become ready go to the writer, the Send/Receive loop of the Encoder documentation written once. It owns neither side; Flush it at the end, then close the encoder and the writer.
func NewEncodeWriter ¶
func NewEncodeWriter(enc Encoder, dst PacketWriter) *EncodeWriter
NewEncodeWriter returns a frame writer that encodes with enc into dst.
func (*EncodeWriter) Flush ¶
func (w *EncodeWriter) Flush(ctx context.Context) error
Flush makes the encoder emit the frames it is holding and writes their packets. The encoder can be reused afterwards; the next frame is a keyframe.
func (*EncodeWriter) Packets ¶
func (w *EncodeWriter) Packets() int64
Packets is the number of packets written so far.
func (*EncodeWriter) WriteFrame ¶
func (w *EncodeWriter) WriteFrame(ctx context.Context, f *Frame) error
WriteFrame implements FrameWriter. The pixel data is copied before it returns (see Encoder.Send).
type Encoder ¶
Re-exported core types. See the documentation on each in package codec.
func NewEncoder ¶
func NewEncoder(ctx context.Context, c Codec, width, height int, opts ...EncoderOption) (Encoder, error)
NewEncoder opens a hardware encoder for c at the given picture size on the first backend that supports it. The error matches ErrUnsupported (and is an *UnsupportedError) when no backend can serve the request, including when a requested control (profile, rate control mode) is not available.
Without options the encoder uses the backend's default rate control, no B-frames and the backend's keyframe placement. Set WithFrameRate so that rate control knows the frame rate, and give every frame an increasing PTS.
type EncoderOption ¶
type EncoderOption = codec.EncoderOption
Re-exported core types. See the documentation on each in package codec.
func WithBFrames ¶
func WithBFrames() EncoderOption
WithBFrames allows the encoder to use B-frames. Packets then arrive in decode order with DTS lagging PTS. B-frames are off by default.
func WithFrameRate ¶
func WithFrameRate(fps float64) EncoderOption
WithFrameRate tells the encoder the expected frame rate. It sets the duration of every frame and lets rate control plan for the rate.
func WithInputFormat ¶
func WithInputFormat(f PixelFormat) EncoderOption
WithInputFormat sets the pixel format of frames given to Encoder.Send: NV12 (the default), RGBA or BGRA. The alpha byte of RGBA and BGRA input is ignored. A backend whose encoder takes NV12 only gets RGBA and BGRA frames converted in Go (BT.709, video range) and declares that matrix in the stream where the encoder lets it.
func WithKeyframeInterval ¶
func WithKeyframeInterval(frames int) EncoderOption
WithKeyframeInterval makes every n-th frame a keyframe, counted from the first frame and from every Flush. The encoder may insert extra keyframes. 0 (the default) leaves keyframe placement to the backend.
func WithLowLatency ¶
func WithLowLatency() EncoderOption
WithLowLatency configures the encoder for live streaming: real-time rate control, no frame reordering and the backend's low-latency mode where it has one. WithKeyframeInterval and Frame.ForceKeyframe keep working.
func WithProfile ¶
func WithProfile(p Profile) EncoderOption
WithProfile selects the coding profile. Combinations the codec does not define (ProfileBaseline or ProfileHigh with HEVC) report ErrUnsupported.
func WithQuality ¶
func WithQuality(q float64) EncoderOption
WithQuality requests constant-quality coding with q in (0, 1] (higher is better) instead of a bitrate target. It cannot be combined with WithBitrate. Backends without a quality mode report ErrUnsupported.
type Frame ¶
Re-exported core types. See the documentation on each in package codec.
type FrameReader ¶
type FrameReader interface {
// ReadFrame returns the next frame, or io.EOF at the end. The caller
// owns the frame and must Release it.
ReadFrame(ctx context.Context) (*Frame, error)
}
FrameReader is a source of raw frames: a decoder fed from a PacketReader (DecodeReader), or anything that produces or transforms pictures.
type FrameWriter ¶
type FrameWriter interface {
// WriteFrame consumes f before it returns; the frame stays the
// caller's, who may release or reuse it afterwards.
WriteFrame(ctx context.Context, f *Frame) error
}
FrameWriter consumes raw frames: an encoder drained into a PacketWriter (EncodeWriter), or anything that shows or stores pictures.
type Option ¶
Re-exported core types. See the documentation on each in package codec.
func WithSoftwareFallback ¶
func WithSoftwareFallback() Option
WithSoftwareFallback lets a backend use the operating system's software codec when no hardware engine exists for the codec. By default the request is rejected with ErrUnsupported instead. It applies to the video decoders and encoders; the audio codecs run in software everywhere and take the option without needing it.
func WithTimeScale ¶
WithTimeScale sets the number of PTS units per second. The default is 90000 for video and the sample rate for an audio encoder, so that audio timestamps count samples. An audio decoder hands the PTS of each packet on unchanged, whatever its unit, and takes the option without effect.
type PacketReader ¶
type PacketReader interface {
// Codec of the stream.
Codec() Codec
// TimeScale is the number of units per second of the packets' PTS and
// DTS. Open the decoder with WithTimeScale(TimeScale()) so that the
// frames carry the same clock.
TimeScale() int32
// ReadPacket returns the next access unit, whose PTS counts from the
// start of the presentation. It returns io.EOF at the end of the
// stream.
ReadPacket() (Packet, error)
}
PacketReader hands out the access units of a stream in decode order: a demuxed video or audio track, a raw elementary stream. It is what decoders are fed from (see DecodeReader).
type PacketSeeker ¶
type PacketSeeker interface {
PacketReader
// SeekKeyframe repositions the reader at the last keyframe whose
// presentation time is not after t (at the first keyframe when t lies
// before it) and returns that keyframe's presentation time. The next
// ReadPacket returns the keyframe.
SeekKeyframe(t time.Duration) (time.Duration, error)
// Length returns the stream duration, or 0 when it is not known.
Length() time.Duration
}
PacketSeeker is a PacketReader that can be repositioned, which is what seeking and looping need.
type PacketWriteCloser ¶
type PacketWriteCloser interface {
PacketWriter
io.Closer
}
PacketWriteCloser is a PacketWriter that is finished with Close (the sample tables of a file are written, the viewers disconnected).
type PacketWriter ¶
PacketWriter receives encoded packets in decode order: a file, a segmenter, a network track. It is what encoders are drained into (see EncodeWriter and package capture).
type PacketWriterFunc ¶
PacketWriterFunc adapts a function to a PacketWriteCloser whose Close does nothing.
func (PacketWriterFunc) WritePacket ¶
func (f PacketWriterFunc) WritePacket(p Packet) error
WritePacket calls f.
type PixelFormat ¶
type PixelFormat = codec.PixelFormat
Re-exported core types. See the documentation on each in package codec.
type RateControl ¶
type RateControl = codec.RateControl
Re-exported core types. See the documentation on each in package codec.
type SampleFormat ¶
type SampleFormat = codec.SampleFormat
Re-exported audio types. See the documentation on each in package codec.
type SharedEncoderOption ¶
type SharedEncoderOption = codec.SharedEncoderOption
Re-exported audio types. See the documentation on each in package codec.
func WithBitrate ¶
func WithBitrate(bitsPerSecond int) SharedEncoderOption
WithBitrate sets the target bitrate in bits per second, for a video or an audio encoder. The rate control mode is VBR unless WithRateControl(CBR) is also given.
func WithRateControl ¶
func WithRateControl(rc RateControl) SharedEncoderOption
WithRateControl selects VBR (default) or CBR for the bitrate given with WithBitrate, for a video or an audio encoder. Backends that cannot do CBR report ErrUnsupported.
type UnsupportedError ¶
type UnsupportedError = codec.UnsupportedError
Re-exported core types. See the documentation on each in package codec.
Directories
¶
| Path | Synopsis |
|---|---|
|
Package capture records what a game or any other renderer draws.
|
Package capture records what a game or any other renderer draws. |
|
cmd
|
|
|
hwmediacodec
module
|
|
|
ebitenvideo
module
|
|
|
encoding
|
|
|
adts
Package adts reads and writes ADTS streams: AAC access units with a seven-byte header in front of each, which is the format of .aac files and of AAC in MPEG transport streams.
|
Package adts reads and writes ADTS streams: AAC access units with a seven-byte header in front of each, which is the format of .aac files and of AAC in MPEG transport streams. |
|
annexb
Package annexb splits H.264 and HEVC Annex-B byte streams into NAL units and groups them into access units (one coded picture each), which is the unit a hwmediacodec decoder consumes.
|
Package annexb splits H.264 and HEVC Annex-B byte streams into NAL units and groups them into access units (one coded picture each), which is the unit a hwmediacodec decoder consumes. |
|
examples
module
|
|
|
image
|
|
|
avif
Package avif reads and writes AVIF still images (HEIF with AV1 pictures) with the hardware codecs, in the manner of image/jpeg and image/png: Decode and DecodeConfig take an io.Reader, Encode an image.Image, and importing the package registers the format with the image package, so that image.Decode understands AVIF files:
|
Package avif reads and writes AVIF still images (HEIF with AV1 pictures) with the hardware codecs, in the manner of image/jpeg and image/png: Decode and DecodeConfig take an io.Reader, Encode an image.Image, and importing the package registers the format with the image package, so that image.Decode understands AVIF files: |
|
heif
Package heif reads and writes HEIC still images (HEIF with HEVC pictures) with the hardware codecs, in the manner of image/jpeg and image/png: Decode and DecodeConfig take an io.Reader, Encode an image.Image, and importing the package registers the format with the image package, so that image.Decode understands HEIC files:
|
Package heif reads and writes HEIC still images (HEIF with HEVC pictures) with the hardware codecs, in the manner of image/jpeg and image/png: Decode and DecodeConfig take an io.Reader, Encode an image.Image, and importing the package registers the format with the image package, so that image.Decode understands HEIC files: |
|
internal
|
|
|
aac
Package aac reads and writes the two small headers that describe an AAC stream: the AudioSpecificConfig of MPEG-4 (ISO/IEC 14496-3, section 1.6.2.1), which an MP4 file keeps in its esds box, and the ADTS frame header of raw .aac files and MPEG transport streams.
|
Package aac reads and writes the two small headers that describe an AAC stream: the AudioSpecificConfig of MPEG-4 (ISO/IEC 14496-3, section 1.6.2.1), which an MP4 file keeps in its esds box, and the ADTS frame header of raw .aac files and MPEG transport streams. |
|
audiocmp
command
Command audiocmp is the host half of the audio checks that run a codec somewhere else (a phone over adb, a Windows machine): it makes the test signal and compares raw 16-bit samples that came back with a reference.
|
Command audiocmp is the host half of the audio checks that run a codec somewhere else (a phone over adb, a Windows machine): it makes the test signal and compares raw 16-bit samples that came back with a reference. |
|
audiotoolbox
Package audiotoolbox implements the audio codecs of macOS and iOS on top of the AudioConverter API of Apple's AudioToolbox framework, loaded at run time with purego.
|
Package audiotoolbox implements the audio codecs of macOS and iOS on top of the AudioConverter API of Apple's AudioToolbox framework, loaded at run time with purego. |
|
audiotoolbox/sys
Package sys binds the AudioToolbox entry points used by the AudioToolbox backend (AudioConverter and AudioFormat) through purego, so that no cgo is required.
|
Package sys binds the AudioToolbox entry points used by the AudioToolbox backend (AudioConverter and AudioFormat) through purego, so that no cgo is required. |
|
audioutil
Package audioutil holds what the audio backends share: the trial that measures an encoder's delay, the queue that hands a decoder's samples out one frame per packet, and sample format conversion.
|
Package audioutil holds what the audio backends share: the trial that measures an encoder's delay, the queue that hands a decoder's samples out one frame per packet, and sample format conversion. |
|
av1
Package av1 parses the parts of an AV1 bitstream that a hardware decoder backend needs in order to describe a stream to the operating system: the OBU framing of a temporal unit (Section 5.3 of the AV1 specification), the sequence header (5.5), the frame header (5.9), the tile group layout (5.11.1) and the ISOBMFF AV1CodecConfigurationRecord ("av1C").
|
Package av1 parses the parts of an AV1 bitstream that a hardware decoder backend needs in order to describe a stream to the operating system: the OBU framing of a temporal unit (Section 5.3 of the AV1 specification), the sequence header (5.5), the frame header (5.9), the tile group layout (5.11.1) and the ISOBMFF AV1CodecConfigurationRecord ("av1C"). |
|
bitstream
Package bitstream reads bit fields and Exp-Golomb codes from H.264/HEVC raw byte sequence payloads.
|
Package bitstream reads bit fields and Exp-Golomb codes from H.264/HEVC raw byte sequence payloads. |
|
browserexec
command
Command browserexec runs a GOOS=js GOARCH=wasm test binary in a real browser, where WebCodecs exists (Node.js, which the Go toolchain's own go_js_wasm_exec uses, has none).
|
Command browserexec runs a GOOS=js GOARCH=wasm test binary in a real browser, where WebCodecs exists (Node.js, which the Go toolchain's own go_js_wasm_exec uses, has none). |
|
codec
Package codec holds the core types shared by the public hwmediacodec API and its platform backends.
|
Package codec holds the core types shared by the public hwmediacodec API and its platform backends. |
|
fdkaac
Package fdkaac implements AAC decoding and encoding on Linux on top of the system's libfdk-aac (the Fraunhofer FDK AAC library), loaded at run time with purego.
|
Package fdkaac implements AAC decoding and encoding on Linux on top of the system's libfdk-aac (the Fraunhofer FDK AAC library), loaded at run time with purego. |
|
h264
Package h264 parses the H.264 (Rec.
|
Package h264 parses the H.264 (Rec. |
|
hevc
Package hevc parses and writes HEVC (ITU-T H.265 / ISO/IEC 23008-2) parameter sets and slice segment headers and keeps the decoded picture buffer bookkeeping of the decoding process: picture order counts, reference picture sets and reference picture lists.
|
Package hevc parses and writes HEVC (ITU-T H.265 / ISO/IEC 23008-2) parameter sets and slice segment headers and keeps the decoded picture buffer bookkeeping of the decoding process: picture order counts, reference picture sets and reference picture lists. |
|
imageitem
Package imageitem is the ISOBMFF image-item plumbing shared by the image/heif (HEVC) and image/avif (AV1) packages: parsing the "meta" box into items and properties, grids, clean aperture, rotation and mirroring, driving the hardware decoder and encoder one keyframe per item, and writing the file.
|
Package imageitem is the ISOBMFF image-item plumbing shared by the image/heif (HEVC) and image/avif (AV1) packages: parsing the "meta" box into items and properties, grids, clean aperture, rotation and mirroring, driving the hardware decoder and encoder one keyframe per item, and writing the file. |
|
libavcodec
Package libavcodec implements AAC decoding on Linux on top of the system's libavcodec (FFmpeg), loaded at run time with purego.
|
Package libavcodec implements AAC decoding on Linux on top of the system's libavcodec (FFmpeg), loaded at run time with purego. |
|
libopus
Package libopus implements Opus decoding and encoding on Linux on top of the system's libopus, loaded at run time with purego.
|
Package libopus implements Opus decoding and encoding on Linux on top of the system's libopus, loaded at run time with purego. |
|
mediacodec
Package mediacodec implements the Android backend on top of the NDK MediaCodec C API (libmediandk.so), loaded at run time with purego.
|
Package mediacodec implements the Android backend on top of the NDK MediaCodec C API (libmediandk.so), loaded at run time with purego. |
|
mediacodec/sys
Package sys binds the Android NDK media library (libmediandk.so: the AMediaCodec and AMediaFormat C API) through purego, so that no cgo source is needed.
|
Package sys binds the Android NDK media library (libmediandk.so: the AMediaCodec and AMediaFormat C API) through purego, so that no cgo source is needed. |
|
mediafoundation
Package mediafoundation implements the Windows backend on top of Media Foundation decoder transforms accelerated through Direct3D 11 (DXVA).
|
Package mediafoundation implements the Windows backend on top of Media Foundation decoder transforms accelerated through Direct3D 11 (DXVA). |
|
mediafoundation/sys
Package sys binds the Media Foundation, Direct3D 11 and COM entry points used by the Media Foundation backend.
|
Package sys binds the Media Foundation, Direct3D 11 and COM entry points used by the Media Foundation backend. |
|
mediatest
Package mediatest drives ffmpeg, ffprobe and sips as test oracles for the container and image packages: it generates MP4 inputs, inspects outputs and converts pictures.
|
Package mediatest drives ffmpeg, ffprobe and sips as test oracles for the container and image packages: it generates MP4 inputs, inspects outputs and converts pictures. |
|
nvidia
Package nvidia implements the NVIDIA backend on Linux and on Windows (amd64): decoding through NVDEC (the cuvid parser and decoder in libnvcuvid / nvcuvid.dll) and encoding through NVENC (libnvidia-encode / nvEncodeAPI64.dll), both on top of the CUDA driver API in libcuda / nvcuda.dll.
|
Package nvidia implements the NVIDIA backend on Linux and on Windows (amd64): decoding through NVDEC (the cuvid parser and decoder in libnvcuvid / nvcuvid.dll) and encoding through NVENC (libnvidia-encode / nvEncodeAPI64.dll), both on top of the CUDA driver API in libcuda / nvcuda.dll. |
|
nvidia/sys
Package sys binds the NVIDIA driver libraries (libcuda, libnvcuvid and libnvidia-encode on Linux; nvcuda.dll, nvcuvid.dll and nvEncodeAPI64.dll on Windows) through purego so that no cgo is required.
|
Package sys binds the NVIDIA driver libraries (libcuda, libnvcuvid and libnvidia-encode on Linux; nvcuda.dll, nvcuvid.dll and nvEncodeAPI64.dll on Windows) through purego so that no cgo is required. |
|
opus
Package opus reads and writes the headers around Opus packets: the TOC byte that gives a packet's duration (RFC 6716, section 3.1) and the identification header "OpusHead" (RFC 7845, section 5.1).
|
Package opus reads and writes the headers around Opus packets: the TOC byte that gives a packet's duration (RFC 6716, section 3.1) and the identification header "OpusHead" (RFC 7845, section 5.1). |
|
pixconv
Package pixconv converts packed RGB pictures to NV12 in Go, for the backends whose encoder takes NV12 only (Media Foundation, VA-API and Intel VPL).
|
Package pixconv converts packed RGB pictures to NV12 in Go, for the backends whose encoder takes NV12 only (Media Foundation, VA-API and Intel VPL). |
|
reorder
Package reorder turns the decode-order output of a backend decoder into display order.
|
Package reorder turns the decode-order output of a backend decoder into display order. |
|
testutil
Package testutil generates test bitstreams and reference output with the ffmpeg command-line tool.
|
Package testutil generates test bitstreams and reference output with the ffmpeg command-line tool. |
|
vaapi
Package vaapi implements the Linux backend on top of VA-API (libva), loaded at run time with purego.
|
Package vaapi implements the Linux backend on top of VA-API (libva), loaded at run time with purego. |
|
vaapi/sys
Package sys binds libva (VA-API) through purego so that no cgo is required.
|
Package sys binds libva (VA-API) through purego so that no cgo is required. |
|
videotoolbox
Package videotoolbox implements the macOS backend on top of Apple's VideoToolbox framework, loaded at run time with purego.
|
Package videotoolbox implements the macOS backend on top of Apple's VideoToolbox framework, loaded at run time with purego. |
|
videotoolbox/sys
Package sys binds the CoreFoundation, CoreMedia, CoreVideo and VideoToolbox entry points used by the VideoToolbox backend through purego, so that no cgo is required.
|
Package sys binds the CoreFoundation, CoreMedia, CoreVideo and VideoToolbox entry points used by the VideoToolbox backend through purego, so that no cgo is required. |
|
vpl
Package vpl implements the Intel backend on Linux on top of the Intel Video Processing Library (VPL, the successor of the Media SDK and the API behind "Quick Sync Video").
|
Package vpl implements the Intel backend on Linux on top of the Intel Video Processing Library (VPL, the successor of the Media SDK and the API behind "Quick Sync Video"). |
|
vpl/sys
Package sys binds the Intel VPL dispatcher (libvpl) through purego so that no cgo is required.
|
Package sys binds the Intel VPL dispatcher (libvpl) through purego so that no cgo is required. |
|
webcodecs
Package webcodecs implements the browser backend on top of the WebCodecs API (VideoDecoder, VideoEncoder, VideoFrame, EncodedVideoChunk), reached through syscall/js in a GOOS=js GOARCH=wasm build.
|
Package webcodecs implements the browser backend on top of the WebCodecs API (VideoDecoder, VideoEncoder, VideoFrame, EncodedVideoChunk), reached through syscall/js in a GOOS=js GOARCH=wasm build. |
|
mediacontainer
|
|
|
ivf
Package ivf reads and writes IVF files, the minimal container for raw AV1 (and VP8/VP9) streams: a 32-byte file header followed by frames that each carry a 12-byte header with the frame size and timestamp.
|
Package ivf reads and writes IVF files, the minimal container for raw AV1 (and VP8/VP9) streams: a 32-byte file header followed by frames that each carry a 12-byte header with the frame size and timestamp. |
|
mp4
Package mp4 reads and writes MP4 files around hwmediacodec, which itself only handles elementary streams.
|
Package mp4 reads and writes MP4 files around hwmediacodec, which itself only handles elementary streams. |
|
webm
Package webm reads and writes WebM files around hwmediacodec, which itself only handles elementary streams.
|
Package webm reads and writes WebM files around hwmediacodec, which itself only handles elementary streams. |
|
net
|
|
|
hls
Package hls serves encoder output as a live HLS stream.
|
Package hls serves encoder output as a live HLS stream. |
|
webrtc
module
|
|
|
Package playback plays a video stream against a clock: it decodes on a background goroutine through hwmediacodec, a few frames ahead, and hands out the frame that is due as the caller advances the time.
|
Package playback plays a video stream against a clock: it decodes on a background goroutine through hwmediacodec, a few frames ahead, and hands out the frame that is due as the caller advances the time. |