replay

package
v0.0.0-...-3c84d77 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Jun 25, 2026 License: MIT Imports: 25 Imported by: 0

Documentation

Overview

Package replay provides an instant replay system with variable-speed playback for live video switching.

Each source gets a GOP-aligned circular buffer that captures encoded H.264 frames with wall-clock timestamps. The Manager orchestrates mark-in/out points, playback, and per-source buffer lifecycle. The replayPlayer decodes clips, sorts by PTS, and re-encodes with frame duplication for slow-motion (0.25x-1x speed).

Audio is time-stretched for pitch-preserved slow-motion playback using a phase vocoder (STFT-based spectral processing) as the primary path, with WSOLA (Waveform Similarity Overlap-Add) as a fallback. Frame interpolation is pluggable via the FrameInterpolator interface, with alpha-blend and MCFI (motion-compensated frame interpolation) implementations available.

Key types:

  • Manager: Replay orchestration (mark-in/out, play, stop, buffer management)
  • Config: Buffer duration, codec factories, replay relay reference
  • Status: Current player state, mark points, active source
  • SourceBufferInfo: Per-source buffer fill level and time range
  • FrameInterpolator: Pluggable frame interpolation (blend, MCFI)

Replay output is routed to a dedicated "replay" relay so browsers can subscribe via MoQ for replay monitoring.

Index

Constants

This section is empty.

Variables

View Source 
var (
	ErrNoSource       = errors.New("replay: source not found")
	ErrNoMarkIn       = errors.New("replay: mark-in not set")
	ErrNoMarkOut      = errors.New("replay: mark-out not set")
	ErrInvalidMarks   = errors.New("replay: mark-out must be after mark-in")
	ErrPlayerActive   = errors.New("replay: player already active")
	ErrNoPlayer       = errors.New("replay: no active player")
	ErrEmptyClip      = errors.New("replay: clip contains no frames")
	ErrInvalidSpeed   = errors.New("replay: speed must be between 0.25 and 1.0")
	ErrBufferDisabled = errors.New("replay: buffer is disabled (0 duration)")
	ErrSourceMismatch = errors.New("replay: mark-out source must match mark-in source")
	ErrMaxSources     = errors.New("replay: maximum sources reached")
	ErrNotPlaying     = errors.New("replay: not playing")
	ErrNotPaused      = errors.New("replay: not paused")
	ErrInvalidSeek    = errors.New("replay: seek position must be between 0.0 and 1.0")
)

Sentinel errors for the replay subsystem.

View Source 
var ErrFrameRecycled = errors.New("filering: frame slot recycled by wrap")

ErrFrameRecycled is returned by Read when the requested frame slot has been overwritten by a circular wrap since it was written. The buffer metadata still references it by wall-clock time, but the bytes on disk no longer hold that frame, so reading them would yield silent corruption.

Functions

func PhaseVocoderTimeStretch 

func PhaseVocoderTimeStretch(input []float32, channels, sampleRate int, speed float64) []float32

PhaseVocoderTimeStretch performs high-quality pitch-preserved time-stretching using an STFT-based phase vocoder with identity phase locking and transient detection.

  • input: interleaved PCM samples
  • channels: number of audio channels (1 or 2)
  • sampleRate: sample rate in Hz
  • speed: playback speed (0.1-1.0)

Returns the time-stretched output samples.

func WSOLATimeStretch 

func WSOLATimeStretch(input []float32, channels, sampleRate int, speed float64) []float32

WSOLATimeStretch performs Waveform Similarity Overlap-Add time-stretching. Preserves pitch while changing duration.

  • input: interleaved PCM samples
  • channels: number of audio channels (1 or 2)
  • sampleRate: sample rate in Hz
  • speed: playback speed (0.25-1.0)

For speeds below 0.5x, uses cascaded stretching (two passes at sqrt(speed)) to avoid the artifacts that single-pass extreme stretching produces.

Returns the time-stretched output samples.

Types

type AdjustMarksRequest 

type AdjustMarksRequest struct {
	MarkIn  *int64 `json:"markIn,omitempty"`  // Unix ms
	MarkOut *int64 `json:"markOut,omitempty"` // Unix ms
}

AdjustMarksRequest is the JSON body for adjusting mark points.

type Config 

type Config struct {
	// BufferDurationSecs is the per-source buffer duration in seconds.
	// Default 300, max 300.
	BufferDurationSecs int

	// MaxSources is the maximum number of sources to buffer simultaneously.
	// Default 8.
	MaxSources int

	// MaxBufferBytes is the per-source byte limit for the replay buffer.
	// When exceeded, oldest GOPs are trimmed. Default 200MB. 0 disables.
	MaxBufferBytes int64

	// TmpfsDir is the directory for tmpfs-backed replay buffers.
	// Empty string = auto-detect (/dev/shm/switchframe/replay).
	// "none" = disable tmpfs, use in-memory buffers only.
	TmpfsDir string
}

Config holds configuration for the replay manager.

func DefaultConfig 

func DefaultConfig() Config

DefaultConfig returns the default replay configuration.

type FrameInterpolator 

type FrameInterpolator interface {
	Interpolate(frameA, frameB []byte, width, height int, alpha float64) []byte
}

FrameInterpolator generates an interpolated frame between two YUV420 frames.

type GPUPlayerConfig 

type GPUPlayerConfig struct {
	Clip         []bufferedFrame
	AudioClip    []bufferedAudioFrame
	Speed        float64
	Loop         bool
	InitialPTS   int64
	ShutterAngle float64 // 0.0-1.0 normalized (0.5 = 180°)

	GPUCtx *gpu.Context
	FRUC   *gpu.FRUC
	Pool   *gpu.FramePool

	Output         func(frame *media.VideoFrame)
	AudioOutput    func(frame *media.AudioFrame)
	RawVideoOutput func(yuv []byte, w, h int, pts int64)
	OnDone         func()
	OnReady        func()
	OnVideoInfo    func(sps, pps []byte, width, height int)
}

GPUPlayerConfig configures a GPU-accelerated replay player instance. The GPU player stays entirely on GPU: NVDEC decode → NVOFA optical flow → motion-blur CUDA kernel → NVENC encode. No CPU-side YUV touching in the hot path (except the optional RawVideoOutput callback which downloads to CPU).

type InterpolationMode 

type InterpolationMode string

InterpolationMode selects the frame interpolation algorithm. 

const (
	InterpolationNone          InterpolationMode = "none"           // frame duplication (current behavior)
	InterpolationBlend         InterpolationMode = "blend"          // alpha blend adjacent frames
	InterpolationMCFI          InterpolationMode = "mcfi"           // motion-compensated frame interpolation
	InterpolationHoldCrossfade InterpolationMode = "hold-crossfade" // hold clean frames, crossfade at transitions
)

type Manager 

type Manager struct {
	// contains filtered or unexported fields
}

Manager orchestrates the replay system: per-source buffers, viewers, mark-in/out points, and the active player.

func NewManager 

func NewManager(relay Relay, cfg Config, decoderFactory transition.DecoderFactory, encoderFactory transition.EncoderFactory) *Manager

NewManager creates a replay manager.

func (*Manager) AddSource 

func (m *Manager) AddSource(key string) error

AddSource registers a source for replay buffering. Returns an error if the maximum number of sources has been reached.

func (*Manager) Close 

func (m *Manager) Close()

Close stops any active player, waits for background export goroutines, and releases resources. Cleans up any tmpfs-backed ring buffer files.

func (*Manager) DebugSnapshot 

func (m *Manager) DebugSnapshot() map[string]any

DebugSnapshot returns debug information about the replay system.

func (*Manager) MarkIn 

func (m *Manager) MarkIn(source string) error

MarkIn sets the mark-in point to the current time for the given source.

func (*Manager) MarkOut 

func (m *Manager) MarkOut(source string) error

MarkOut sets the mark-out point to the current time.

func (*Manager) OnPlaybackLifecycle 

func (m *Manager) OnPlaybackLifecycle(onStart, onStop func())

OnPlaybackLifecycle registers callbacks invoked when playback starts and stops. onStart is called when the player transitions to playing (first frame decoded). onStop is called when the player finishes naturally or is stopped manually.

func (*Manager) OnStateChange 

func (m *Manager) OnStateChange(fn func())

OnStateChange registers a callback invoked when replay state changes.

func (*Manager) OnVideoInfoChange 

func (m *Manager) OnVideoInfoChange(fn func(sps, pps []byte, width, height int))

OnVideoInfoChange registers a callback invoked when the replay player produces its first keyframe with SPS/PPS. Used to set VideoInfo on the replay relay so MoQ subscribers can discover tracks.

func (*Manager) Pause 

func (m *Manager) Pause() error

Pause pauses the active player.

func (*Manager) PeekFrame 

func (m *Manager) PeekFrame(source string) ([]byte, error)

PeekFrame returns a JPEG thumbnail of the most recent frame for a source. Currently returns nil (no thumbnail available) — the frontend falls back to the replay canvas.

func (*Manager) Play 

func (m *Manager) Play(source string, speed float64, loop bool) error

Play starts playback of the marked clip at the given speed.

func (*Manager) QuickReplay 

func (m *Manager) QuickReplay(source string, seconds int, speed float64) error

QuickReplay combines mark + play in one call, auto-stopping any active player first. Sets marks to [now-seconds, now] and starts playback.

func (*Manager) RecordFrame 

func (m *Manager) RecordFrame(key string, frame *media.VideoFrame)

RecordFrame records a frame into the source's replay buffer. Called directly from the streamCallbackRouter's viewer.

func (*Manager) RemoveSource 

func (m *Manager) RemoveSource(key string)

RemoveSource stops buffering for a source.

func (*Manager) Resume 

func (m *Manager) Resume() error

Resume resumes the active player from a paused state.

func (*Manager) Seek 

func (m *Manager) Seek(position float64) error

Seek seeks the active player to the given position (0.0-1.0).

func (*Manager) SetAudioCodecFactories 

func (m *Manager) SetAudioCodecFactories(decFactory audio.DecoderFactory, encFactory audio.EncoderFactory)

SetAudioCodecFactories sets the AAC decoder/encoder factories used for WSOLA audio time-stretching during slow-motion playback.

func (*Manager) SetAudioOutput 

func (m *Manager) SetAudioOutput(fn func(frame *media.AudioFrame))

SetAudioOutput registers a callback for sending audio directly to the mixer, bypassing the relay encode/decode hop.

func (*Manager) SetGPUReplay 

func (m *Manager) SetGPUReplay(ctx *gpu.Context, fruc *gpu.FRUC, pool *gpu.FramePool)

SetGPUReplay enables GPU-accelerated replay with NVOFA motion blur. When set and NVOFA is available, Play() will use the GPU player instead of the CPU player. Pass nil values to disable GPU replay.

func (*Manager) SetMarks 

func (m *Manager) SetMarks(markInMs, markOutMs *int64) error

SetMarks adjusts the mark-in and/or mark-out points from Unix milliseconds.

func (*Manager) SetOnClipExported 

func (m *Manager) SetOnClipExported(fn func(source string, filePath string))

SetOnClipExported registers a callback invoked after Play() extracts a clip from the replay buffer. The callback receives the source name and a path to a temporary MPEG-TS file containing the muxed clip frames. The caller is responsible for moving or removing the temp file.

func (*Manager) SetPTSProvider 

func (m *Manager) SetPTSProvider(fn func() int64)

SetPTSProvider registers a function that returns the current program PTS. The replay player uses this to anchor its output PTS to the program timeline, preventing backward PTS jumps when cut to program.

func (*Manager) SetProxyEncoderFactory 

func (m *Manager) SetProxyEncoderFactory(factory transition.EncoderFactory)

SetProxyEncoderFactory sets a browser-compatible encoder factory (typically H.264) for dual-encode mode. When set, the main encoder factory creates the pipeline codec encoder (e.g. HEVC), and this factory creates the H.264 proxy for browser playback.

func (*Manager) SetRawVideoOutput 

func (m *Manager) SetRawVideoOutput(fn func(yuv []byte, w, h int, pts int64))

SetRawVideoOutput registers a callback for sending decoded YUV frames directly to the switcher pipeline (primary output path).

func (*Manager) SetShutterAngle 

func (m *Manager) SetShutterAngle(degrees float64)

SetShutterAngle sets the motion-blur shutter angle (0-360 degrees). Only affects GPU-accelerated replay. A shutter angle of 0 disables motion blur (pure interpolation). 180° is the film standard.

func (*Manager) SetSpeed 

func (m *Manager) SetSpeed(speed float64) error

SetSpeed changes the playback speed of the active player.

func (*Manager) ShutterAngle 

func (m *Manager) ShutterAngle() float64

ShutterAngle returns the current shutter angle in degrees.

func (*Manager) Status 

func (m *Manager) Status() Status

Status returns the current replay status for state broadcasts.

func (*Manager) Stop 

func (m *Manager) Stop() error

Stop stops the active player.

func (*Manager) Viewer 

func (m *Manager) Viewer(key string) *replayViewer

Viewer returns the replay viewer for the given source, or nil if the source is not registered for replay. The returned viewer implements distribution.Viewer and should be registered on the source's relay.

type MarkInRequest 

type MarkInRequest struct {
	Source string `json:"source"`
}

MarkInRequest is the JSON body for the mark-in endpoint.

type MarkOutRequest 

type MarkOutRequest struct {
	Source string `json:"source"`
}

MarkOutRequest is the JSON body for the mark-out endpoint.

type PlayRequest 

type PlayRequest struct {
	Source string  `json:"source"`
	Speed  float64 `json:"speed"`
	Loop   bool    `json:"loop"`
}

PlayRequest is the JSON body for the play endpoint.

type PlayerConfig 

type PlayerConfig struct {
	Clip           []bufferedFrame
	AudioClip      []bufferedAudioFrame
	Speed          float64
	Loop           bool
	InitialPTS     int64 // Starting PTS for output (anchors to program timeline).
	Interpolation  InterpolationMode
	DecoderFactory transition.DecoderFactory
	EncoderFactory transition.EncoderFactory
	Output         func(frame *media.VideoFrame)
	AudioOutput    func(frame *media.AudioFrame)
	OnDone         func()
	OnReady        func()                                   // Called when first GOP decoded and encoder created.
	OnVideoInfo    func(sps, pps []byte, width, height int) // Called once on first encoded keyframe.

	// RawVideoOutput sends decoded YUV directly to the switcher pipeline.
	// Called for every output frame (including slow-mo duplicates/interpolations).
	RawVideoOutput func(yuv []byte, w, h int, pts int64)

	// ProxyEncoderFactory creates a browser-compatible encoder (typically H.264)
	// for the browser relay output. When set, the main EncoderFactory creates the
	// pipeline codec encoder (e.g., HEVC), and this factory creates the browser proxy.
	// The proxy encoder's output goes to the Output callback (browser relay).
	// The main encoder's output goes to PipelineVideoOutput.
	// When nil, the main EncoderFactory is used for Output (legacy behavior).
	ProxyEncoderFactory transition.EncoderFactory

	// PipelineVideoOutput sends pipeline-codec encoded frames for recording/SRT.
	// Only used when ProxyEncoderFactory is set (dual-encode mode).
	PipelineVideoOutput func(frame *media.VideoFrame)

	// AudioDecoderFactory creates an AAC decoder for WSOLA pre-processing.
	// Required when Speed < 1.0 for pitch-preserved slow-motion audio.
	AudioDecoderFactory audio.DecoderFactory

	// AudioEncoderFactory creates an AAC encoder for WSOLA post-processing.
	AudioEncoderFactory audio.EncoderFactory
}

PlayerConfig configures a replay player instance.

type PlayerState 

type PlayerState string

PlayerState represents the current state of the replay player. 

const (
	PlayerIdle    PlayerState = "idle"
	PlayerLoading PlayerState = "loading"
	PlayerPlaying PlayerState = "playing"
	PlayerPaused  PlayerState = "paused"
)

type QuickReplayRequest 

type QuickReplayRequest struct {
	Seconds int     `json:"seconds"`
	Speed   float64 `json:"speed"`
	Source  string  `json:"source"` // Empty = current program source
}

QuickReplayRequest is the JSON body for the quick-replay endpoint.

type Relay 

type Relay interface {
	BroadcastVideo(frame *media.VideoFrame)
	BroadcastAudio(frame *media.AudioFrame)
}

Relay is the interface for the replay output relay.

type SeekRequest 

type SeekRequest struct {
	Position float64 `json:"position"` // 0.0-1.0
}

SeekRequest is the JSON body for the seek endpoint.

type SourceBufferInfo 

type SourceBufferInfo struct {
	Source       string  `json:"source"`
	FrameCount   int     `json:"frameCount"`
	GOPCount     int     `json:"gopCount"`
	DurationSecs float64 `json:"durationSecs"`
	BytesUsed    int64   `json:"bytesUsed"`
}

SourceBufferInfo describes the buffer state for a single source.

type SpeedRequest 

type SpeedRequest struct {
	Speed float64 `json:"speed"` // 0.25-1.0
}

SpeedRequest is the JSON body for the speed-change endpoint.

type Status 

type Status struct {
	State          PlayerState        `json:"state"`
	Source         string             `json:"source,omitempty"`
	Speed          float64            `json:"speed,omitempty"`
	Loop           bool               `json:"loop,omitempty"`
	Position       float64            `json:"position,omitempty"` // 0.0–1.0 playback progress
	MarkIn         *time.Time         `json:"markIn,omitempty"`
	MarkOut        *time.Time         `json:"markOut,omitempty"`
	MarkSource     string             `json:"markSource,omitempty"`
	ShutterAngle   float64            `json:"shutterAngle,omitempty"` // 0-360 degrees
	GPUAccelerated bool               `json:"gpuAccelerated,omitempty"`
	Buffers        []SourceBufferInfo `json:"buffers,omitempty"`
}

Status is the JSON-serializable status for the replay system, included in ControlRoomState for the browser.

func (Status) MarkInUnixMs 

func (rs Status) MarkInUnixMs() *int64

MarkInUnixMs returns the mark-in time as Unix milliseconds, or nil if not set.

func (Status) MarkOutUnixMs 

func (rs Status) MarkOutUnixMs() *int64

MarkOutUnixMs returns the mark-out time as Unix milliseconds, or nil if not set.

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.