Documentation
¶
Overview ¶
Package audiotoolbox decodes compressed audio and plays PCM on macOS through AudioToolbox, with no cgo.
It exists because of a measured hole, and it is the audio half of the one github.com/go-macos/videotoolbox fills for pictures. AVFoundation plays an MP4 end to end and will not demux Matroska at all, so a player that wants sound out of an MKV has to demux the file itself and hand the coded packets to a decoder directly. github.com/go-avkit/avkit/container does the demuxing — it already reports audio tracks, their codec, channel count, sample rate and coded packets, for MP4 and for Matroska alike. This package is the other half: AudioConverter to turn those packets into PCM, and AudioQueue to put the PCM on the system output.
Everything goes through github.com/ebitengine/purego, so a consumer still builds with CGO_ENABLED=0.
Two halves, used together or apart ¶
A Decoder turns one coded Packet into a Buffer of interleaved PCM. A Player takes interleaved PCM and plays it. Neither knows about the other: a caller that only wants samples never opens an output device, and a caller with PCM of its own never builds a decoder.
dec, _ := audiotoolbox.NewDecoder(cfg)
p, _ := audiotoolbox.NewPlayer(audiotoolbox.PlayerConfigFor(cfg))
p.Start()
for _, pkt := range packets {
buf, _ := dec.Decode(audiotoolbox.Packet{Data: pkt.Data})
p.Write(buf.PCM)
}
p.Drain()
The format is stated, never sniffed ¶
A Config says what the packets hold: which codec, at what sample rate, with how many channels. It is taken as stated, the way github.com/go-macos/videotoolbox takes its bitstream form as stated, and for the same reason: a demuxer already knows, and a decoder that guesses is a decoder that is occasionally, silently wrong. container.TrackConfig hands back every field this needs.
AAC in an MP4 or a Matroska file carries no in-band configuration — no ADTS header, nothing before the first packet — so the decoder is set up from an AudioSpecificConfig. When the demuxer states one, in Config.CodecConfig, it is used as it stands; when it does not, one is built from the sample rate, the channel count and the AAC profile the demuxer did state. See Config.MagicCookie.
Buffers alias, and the reason is one memcpy ¶
Decoder.Decode returns PCM that lives in a scratch buffer the decoder owns, and the next call overwrites it. That is deliberate: a player copies the samples into its own queue anyway, and allocating a fresh slice per packet — forty-seven of them a second at 48 kHz — buys nothing. Buffer.Clone copies for a caller that wants to keep one.
The clock ¶
Player.Played is how many seconds of audio have left the device. It is not a count of what was written: a player that synchronises video against bytes handed to the output is a player that drifts, because the output consumes them at its own rate. Audio is the master clock of every serious player — the ear hears a drift the eye does not — so this reads the AudioQueue's own timeline rather than guessing at it.
Player.Drain is what waits for the last of it. AudioQueueStop does not: asked to stop gently it returns at once and finishes in its own time, so a drain that trusted it would cut the tail off. This one waits for the queue to hand every buffer back, which is when the device has finished with them.
Index ¶
- Variables
- type Buffer
- type Codec
- type Config
- type Decoder
- type Packet
- type Player
- func (p *Player) Close() error
- func (p *Player) Config() PlayerConfig
- func (p *Player) Drain() error
- func (p *Player) Played() time.Duration
- func (p *Player) Queued() time.Duration
- func (p *Player) Start() error
- func (p *Player) Stop() error
- func (p *Player) Write(pcm []byte) (int, error)
- func (p *Player) Written() int64
- type PlayerConfig
- type SampleFormat
- type StatusError
- type WAV
Constants ¶
This section is empty.
Variables ¶
var ( // ErrUnsupported is returned by every entry point on non-darwin platforms. ErrUnsupported = errors.New("audiotoolbox: unsupported on this platform (darwin only)") // ErrClosed is returned when a Decoder or a Player is used after Close. ErrClosed = errors.New("audiotoolbox: closed") // ErrUnsupportedCodec is returned for a codec this package does not // describe to AudioToolbox. ErrUnsupportedCodec = errors.New("audiotoolbox: unsupported codec") // ErrConfig is returned for a configuration that does not describe a // stream: no sample rate, no channels, an impossible sample format. ErrConfig = errors.New("audiotoolbox: invalid configuration") // ErrPacket is returned when a packet cannot be submitted as given. ErrPacket = errors.New("audiotoolbox: invalid packet") )
Errors reported by the package. They are stable and may be tested with errors.Is.
Functions ¶
This section is empty.
Types ¶
type Buffer ¶
type Buffer struct {
// PCM is the decoded samples, interleaved, Frames*Channels*Format.Size()
// bytes of them.
PCM []byte
// Frames is how many sample frames PCM holds — samples per channel, not
// samples.
Frames int
// Channels is how many channels are interleaved in PCM.
Channels int
// SampleRate is the rate PCM is at, in Hz.
SampleRate int
// Format is the layout of one sample.
Format SampleFormat
// PTS is the presentation timestamp of the packet this came from.
PTS time.Duration
}
Buffer is the PCM one packet decoded into.
PCM is NOT copied out of the decoder: it aliases a scratch buffer the Decoder owns, and the next call to Decoder.Decode overwrites it. Use Buffer.Clone to keep one. A Player.Write consumes it before returning, so the common path never needs a copy.
func (Buffer) Clone ¶
Clone copies the samples out of the decoder's scratch buffer, so the result outlives the next Decoder.Decode.
type Codec ¶
type Codec uint8
Codec names the compressed form a decoder is set up for.
const ( // AAC is MPEG-4 Advanced Audio Coding, the "mp4a" of an MP4 sample entry // and the A_AAC of a Matroska track. It is what the overwhelming // majority of MP4 files carry. AAC Codec = iota + 1 // AC3 is Dolby Digital, the "ac-3" of an MP4 sample entry and the A_AC3 // of a Matroska track. It is what a great many films in MKV carry, in // 5.1. AC3 // EAC3 is Dolby Digital Plus, "ec-3". EAC3 // Opus is the IETF codec, "Opus" in an MP4 sample entry and A_OPUS in // Matroska. macOS decodes it, but only on a recent enough system; a // session that cannot be built says so rather than falling back. Opus )
The codecs this package describes to AudioToolbox. Every one of them is a codec github.com/go-avkit/avkit/container reports for a demuxed audio track, which is the whole point: what the demuxer produces is what this takes.
func CodecFor ¶
CodecFor maps the codec names container.TrackConfig reports onto a Codec. It reports false for anything else, which is how a caller finds out that a track it demuxed is not one this package decodes before it builds a decoder.
The names are the ones avkit normalises to: "mp4a" for AAC whether it came from an MP4 sample entry or from A_AAC/MPEG4/LC, "ac-3" for AC-3 whether from "ac-3" or A_AC3/BSID9, "ec-3", and "Opus" — with the capital, which is what the MP4 sample entry spells.
type Config ¶
type Config struct {
// Codec is the compressed form in the packets. It is taken as stated and
// never inferred.
Codec Codec
// SampleRate is the coded sample rate in Hz, as the container states it.
SampleRate int
// Channels is the coded channel count.
Channels int
// AudioObjectType is the AAC profile from the track's AudioSpecificConfig:
// 2 for AAC-LC, 5 for HE-AAC, 29 for HE-AAC v2. Zero means AAC-LC, which
// is what a Matroska track that states no configuration at all is.
// Ignored by every other codec.
AudioObjectType byte
// CodecConfig is the codec's own configuration record, as the demuxer
// read it: an AudioSpecificConfig for AAC, an OpusHead for Opus. When it
// is empty for AAC, one is built — see [Config.MagicCookie].
CodecConfig []byte
// OutputChannels is how many channels to decode into. Zero means
// Channels, which is the usual case; setting 2 for a 5.1 track asks
// AudioToolbox to downmix, which it will do for AC-3 and refuse for
// codecs whose decoder has no matrix.
OutputChannels int
// Output is the PCM sample format to decode into. The zero value is
// [Int16].
Output SampleFormat
}
Config describes the track a decoder is built for, the way a demuxer states it.
container.TrackConfig hands back every field but Codec, which CodecFor maps from its Codec string.
func (Config) AudioSpecificConfig ¶
AudioSpecificConfig is the two-or-so bytes of ISO/IEC 14496-3 configuration an AAC decoder is set up from.
Config.CodecConfig is used as it stands when the demuxer stated one: it came out of the file and is more authoritative than anything derived. When there is none — which is what a Matroska track with no CodecPrivate gives, and what every MP4 track avkit reads gives, because avkit reports the AAC profile rather than the record it came from — one is built from the profile, the sample rate and the channel count the demuxer DID state.
It is exported because a caller that wants to know what its decoder was set up from should be able to look, and because it is the one piece of this package worth checking against a hex dump.
func (Config) MagicCookie ¶
MagicCookie is the codec configuration AudioToolbox is set up from, or nil for a codec that needs none.
For AAC it is an MPEG-4 ES_Descriptor — an "esds" — wrapping the track's AudioSpecificConfig. The wrapping is not decoration, it is measured: offered the bare AudioSpecificConfig, AudioConverterSetProperty answers kAudioCodecBadPropertySizeError ('!dat') and the configuration is not applied. Wrapped, the same two bytes are taken.
For every other codec it is Config.CodecConfig as the demuxer read it, or nil when there is none. AC-3 and Enhanced AC-3 carry everything in their packets — the sync frame states the rate, the bitrate and the channel mode — so a cookie would describe what the decoder is about to read anyway.
type Decoder ¶
type Decoder struct {
// contains filtered or unexported fields
}
Decoder is one AudioConverter: a track's decoder, fed a coded packet at a time.
It is NOT safe for concurrent use.
func NewDecoder ¶
NewDecoder builds a decoder for a track.
func (*Decoder) CodecConfigRefused ¶
CodecConfigRefused reports what AudioToolbox answered when the codec configuration was offered to it, or nil when it was taken — and nil, too, for a codec that has none to offer.
A refusal is not fatal and is not treated as one: measured on a plain AAC MP4, a decoder whose cookie was refused still decoded every packet bit for bit against afconvert. It is reported because the alternative is a caller who believes the configuration took effect when it did not, and finds out from a 5.1 track that comes back as noise.
func (*Decoder) Config ¶
Config returns the configuration the decoder was built from, with the defaults resolved.
func (*Decoder) Decode ¶
Decode turns one coded packet into PCM.
The buffer it returns may hold no frames at all, and that is not an error: an AAC decoder emits nothing for the encoder delay at the start of a track, and a converter is entitled to hold a packet back. The samples alias the decoder's scratch buffer and the next call overwrites them.
type Packet ¶
type Packet struct {
// Data is the coded packet, as the demuxer produced it.
Data []byte
// PTS is when this packet's audio starts, relative to the start of the
// track. It travels through the decoder and comes back on the [Buffer];
// nothing here depends on it.
PTS time.Duration
}
Packet is one coded audio packet handed to the decoder — one AAC access unit, one AC-3 sync frame, one Opus packet. It is what container.Reader's Samples hands back for an audio track, one element at a time.
type Player ¶
type Player struct {
// contains filtered or unexported fields
}
Player plays interleaved PCM on the system output.
Player.Write is the only entry point that blocks: it waits for a queue buffer to come free, which is what paces a decode loop to real time without the caller sleeping on a guess.
It is NOT safe for concurrent use, except that Player.Played may be read from another goroutine — which is the point of a clock.
func NewPlayer ¶
func NewPlayer(cfg PlayerConfig) (*Player, error)
NewPlayer opens the system output. Nothing is heard until Player.Start.
func (*Player) Close ¶
Close stops the output and gives it back. It is safe to call more than once, and it does NOT drain: call Player.Drain first to hear the end.
func (*Player) Config ¶
func (p *Player) Config() PlayerConfig
Config returns the configuration the player was built from, with the defaults resolved.
func (*Player) Drain ¶
Drain waits for everything written to be played. A caller that has written its last buffer must call it, or Close cuts the tail off.
func (*Player) Played ¶
Played is how much audio has left the device since Player.Start.
It is the clock a player synchronises video against. It is read from the output's own timeline, not counted from what was written: the two differ by whatever is still sitting in the queue, which is a quarter of a second by default and the whole of the drift a naive player accumulates.
One honest caveat, measured: the timeline is the DEVICE's, and a device that runs out of audio does not stop — it plays silence and the clock goes on advancing. So a player that stops writing sees Played run past what it wrote. That is the right behaviour for a master clock, which must not stall, and Player.Queued is what says whether there is anything left to hear.
func (*Player) Queued ¶
Queued is how much audio has been written but not yet played — the latency between a decode and the sound.
func (*Player) Start ¶
Start begins playback. Writing before it is allowed and usual: filling the queue first is what stops the first buffers from being played half empty.
func (*Player) Write ¶
Write hands interleaved PCM to the output, blocking until it has all been queued. It returns the number of bytes written, which is len(pcm) unless it failed part way.
A partial frame is refused rather than queued: half a sample frame shifts every channel that follows it by one, and the noise that makes is not obviously a bug.
type PlayerConfig ¶
type PlayerConfig struct {
// SampleRate is the rate of the PCM that will be written, in Hz.
SampleRate int
// Channels is how many channels are interleaved in it.
Channels int
// Format is the layout of one sample. The zero value is [Int16].
Format SampleFormat
// BufferFrames is how many frames one output buffer holds; zero means
// 4096.
BufferFrames int
// BufferCount is how many of them the queue cycles through; zero means 3.
// Two is the minimum that lets one play while another is filled.
BufferCount int
// Volume is the output gain, 0 to 1. Zero means 1 — silence is asked for
// by not playing, not by a zero field nobody filled in.
Volume float64
}
PlayerConfig describes the output.
func PlayerConfigFor ¶
func PlayerConfigFor(cfg Config) PlayerConfig
PlayerConfigFor is the output that matches what a Decoder built from cfg produces, which is what a player feeding one decoder wants.
func (PlayerConfig) BytesPerFrame ¶
func (c PlayerConfig) BytesPerFrame() int
BytesPerFrame is how many bytes one sample frame takes.
type SampleFormat ¶
type SampleFormat uint8
SampleFormat is the layout of one PCM sample.
const ( // Int16 is signed 16-bit, little-endian, interleaved. It is the zero // value because it is what an output device wants, what a WAV file // holds, and what a reader can look at with any tool it already has. Int16 SampleFormat = iota // Float32 is 32-bit float, native-endian, interleaved, nominally in // [-1, 1]. It is what a mixer or a resampler would rather have. Float32 )
func (SampleFormat) Size ¶
func (f SampleFormat) Size() int
Size is how many bytes one sample of one channel takes.
type StatusError ¶
type StatusError struct {
// Op is the C function that failed.
Op string
// Status is the OSStatus it returned.
Status int32
}
StatusError carries an OSStatus from AudioToolbox, naming the call that returned it. AudioToolbox says everything through these codes, and half of them are four-character codes read as a signed integer — 1718449215 is 'fmt?', which is a great deal more informative.
func (*StatusError) Error ¶
func (e *StatusError) Error() string
type WAV ¶
type WAV struct {
// contains filtered or unexported fields
}
WAV is a RIFF/WAVE file being written.
It is here because a decoder nobody can check is a decoder nobody should trust. "It plays" is not a result — the reader of a report cannot hear it, and neither can a CI runner. A WAV file is: it opens in anything, its size can be divided by the frame size and compared against the track duration, and its samples can be looked at.
The writer needs an io.WriteSeeker because RIFF states the size of the file and of the data chunk in their headers, before either is known. WAV.Close goes back and fills them in; a file closed without it is a file every reader refuses.
func NewWAV ¶
func NewWAV(w io.WriteSeeker, cfg PlayerConfig) (*WAV, error)
NewWAV starts a WAV file for the PCM cfg describes. The header is written with placeholder sizes, which WAV.Close corrects.
func (*WAV) Close ¶
Close rewrites the header with the sizes now known. It does NOT close the underlying writer, which the caller opened and should close. Calling it twice is a no-op.