inference

package
v0.1.0 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Aug 30, 2026 License: Apache-2.0 Imports: 41 Imported by: 0

Documentation

Overview

Package inference implements the LiveKit Cloud Inference adapters.

Index

Constants

View Source 
const (
	// DefaultURL is the production LiveKit Agent Gateway URL.
	DefaultURL = "https://agent-gateway.livekit.cloud/v1"
	// StagingURL is selected automatically for staging LiveKit projects.
	StagingURL = "https://agent-gateway.staging.livekit.cloud/v1"

	ProviderHeader = "X-LiveKit-Inference-Provider"
	PriorityHeader = "X-LiveKit-Inference-Priority"
)
View Source 
const (
	EOTInferenceMethod                 = "lk_eot_audio"
	DefaultTurnDetectorSampleRate      = 16000
	MinimumTurnDetectorSilenceDuration = 200 * time.Millisecond
	DefaultTurnPredictionTimeout       = time.Second
)

Audio end-of-turn protocol and production defaults.

View Source 
const (
	AdaptiveInterruptionSampleRate            = 16000
	DefaultMinimumInterruptionDuration        = 50 * time.Millisecond
	DefaultMaximumInterruptionAudioDuration   = 3 * time.Second
	DefaultInterruptionAudioPrefixDuration    = time.Second
	DefaultInterruptionDetectionInterval      = 100 * time.Millisecond
	DefaultRemoteInterruptionInferenceTimeout = 700 * time.Millisecond
)

Adaptive interruption wire and timing defaults.

View Source 
const (
	DefaultAvatarSampleRate = 16000
)
View Source 
const (
	InferenceVADSampleRate = 16000
)

Inference VAD protocol and production defaults.

View Source 
const MaxControlMessageBytes = 4 << 20

Variables

View Source 
var (
	// ErrTurnDetectorClosed reports use after detector shutdown.
	ErrTurnDetectorClosed = errors.New("turn detector is closed")
	// ErrTurnPredictionSuperseded cancels local work for a replaced request.
	ErrTurnPredictionSuperseded = errors.New("turn prediction superseded")
	// ErrLocalInferenceUnavailable selects the documented positive fallback.
	ErrLocalInferenceUnavailable = errors.New("local turn inference is unavailable")
)
View Source 
var (
	ErrInvalidEvent    = errors.New("invalid inference event")
	ErrControlTooLarge = errors.New("inference control message too large")
)
View Source 
var ErrAlignmentBufferLimit = errors.New("inference TTS: pending alignment buffer limit exceeded")

ErrAlignmentBufferLimit reports a server stream that sends unbounded timestamp metadata without enough audio to drain it.

View Source 
var ErrAvatarSessionAlreadyStarted = errors.New("inference avatar session may only be started once")
View Source 
var (
	// ErrInterruptionDetectorClosed reports use after detector shutdown.
	ErrInterruptionDetectorClosed = errors.New("adaptive interruption detector is closed")
)
View Source 
var ErrVADClosed = errors.New("inference VAD is closed")

ErrVADClosed reports an operation attempted after detector shutdown.

View Source 
var LocalTurnThresholds = defaultLocalTurnThresholds()

LocalTurnThresholds is a compatibility snapshot of the calibrated mini model table. Mutating it does not affect detector behavior.

Functions

func AccessToken 

func AccessToken(credentials Credentials, ttl time.Duration) (string, error)

AccessToken creates the short-lived inference-grant JWT used by all gateway transports. A non-positive TTL uses the cross-SDK ten-minute default.

func DecodeTTSAlignment 

func DecodeTTSAlignment(event TTSServerEvent, provider string) []agents.TimedString

DecodeTTSAlignment converts a gateway alignment event into SDK timestamps. Word alignment takes precedence over character alignment. Cartesia's words are space-delimited by convention, so the separator is restored here.

func DefaultURLFromEnvironment 

func DefaultURLFromEnvironment() string

DefaultURLFromEnvironment applies the same precedence as the TypeScript and Python SDKs: explicit inference URL, staging project detection, production.

func EstimateInterruptionProbability 

func EstimateInterruptionProbability(probabilities []float32, window time.Duration) float64

EstimateInterruptionProbability returns the n-th largest frame probability, where n is the number of 25 ms frames in the minimum-duration window. This is the conservative estimator used by the Python and TypeScript SDKs.

func MetadataHeaders 

func MetadataHeaders(metadata RequestMetadata) http.Header

MetadataHeaders returns a fresh map safe for caller mutation. AgentID should only be supplied after the room has connected.

func ParseAvatarModel 

func ParseAvatarModel(model string) (provider, avatarID string, err error)

ParseAvatarModel parses "provider" or "provider/<avatar-id>". Slashes in the provider-specific id are preserved for parity with the JS SDK.

func ParseSTTModelString 

func ParseSTTModelString(value string) (model string, language agents.LanguageCode)

ParseSTTModelString parses "provider/model:language" at the final colon. The final-colon rule preserves model identifiers which may contain colons.

func ParseTTSModelString 

func ParseTTSModelString(value string) (model, voice string)

ParseTTSModelString parses "provider/model:voice" at the final colon.

func STTAlignedTranscript 

func STTAlignedTranscript(models ...string) stt.AlignedTranscript

func STTDiarizationEnabled 

func STTDiarizationEnabled(options ModelOptions) bool

func STTSupportsKeyterms 

func STTSupportsKeyterms(model string) bool

func TTSHasAlignedTranscript 

func TTSHasAlignedTranscript(model string, options ModelOptions) bool

TTSHasAlignedTranscript reports whether the configured gateway adapter is explicitly requested to emit timestamps. Unknown providers are conservative.

Types

type AdaptiveInterruptionDetector 

type AdaptiveInterruptionDetector struct {
	// contains filtered or unexported fields
}

AdaptiveInterruptionDetector classifies overlap as interruption or backchannel.

func NewAdaptiveInterruptionDetector 

func NewAdaptiveInterruptionDetector(options AdaptiveInterruptionDetectorOptions) (*AdaptiveInterruptionDetector, error)

NewAdaptiveInterruptionDetector validates credentials and options without dialing.

func (*AdaptiveInterruptionDetector) Close 

func (d *AdaptiveInterruptionDetector) Close(ctx context.Context) error

func (*AdaptiveInterruptionDetector) Label 

func (d *AdaptiveInterruptionDetector) Label() string

func (*AdaptiveInterruptionDetector) Model 

func (d *AdaptiveInterruptionDetector) Model() string

func (*AdaptiveInterruptionDetector) OnError 

func (d *AdaptiveInterruptionDetector) OnError(fn func(InterruptionDetectionError)) func()

func (*AdaptiveInterruptionDetector) OnMetrics 

func (d *AdaptiveInterruptionDetector) OnMetrics(fn func(metrics.Interruption)) func()

func (*AdaptiveInterruptionDetector) OnOverlappingSpeech 

func (d *AdaptiveInterruptionDetector) OnOverlappingSpeech(fn func(OverlappingSpeechEvent)) func()

func (*AdaptiveInterruptionDetector) Provider 

func (d *AdaptiveInterruptionDetector) Provider() string

func (*AdaptiveInterruptionDetector) SampleRate 

func (d *AdaptiveInterruptionDetector) SampleRate() int

func (*AdaptiveInterruptionDetector) Stream 

func (d *AdaptiveInterruptionDetector) Stream(parent context.Context, options ...InterruptionStreamOptions) (*InterruptionStream, error)

func (*AdaptiveInterruptionDetector) UpdateOptions 

func (d *AdaptiveInterruptionDetector) UpdateOptions(options AdaptiveInterruptionUpdateOptions) error

type AdaptiveInterruptionDetectorOptions 

type AdaptiveInterruptionDetectorOptions struct {
	Threshold                   *float64
	MinimumInterruptionDuration time.Duration
	MaximumAudioDuration        time.Duration
	AudioPrefixDuration         time.Duration
	DetectionInterval           time.Duration
	InferenceTimeout            time.Duration
	BaseURL                     string
	Credentials                 Credentials
	ConnectOptions              agents.APIConnectOptions
	Metadata                    func(context.Context) RequestMetadata
	WebSocketDialer             *websocket.Dialer
	InputCapacity               int
	OutputCapacity              int
	WriteTimeout                time.Duration
}

AdaptiveInterruptionDetectorOptions configures the LiveKit barge-in model. A nil Threshold adopts the server-calibrated default.

type AdaptiveInterruptionUpdateOptions 

type AdaptiveInterruptionUpdateOptions struct {
	Threshold                   *float64
	MinimumInterruptionDuration *time.Duration
}

AdaptiveInterruptionUpdateOptions is a sparse live update that reconnects streams.

type AvatarSession 

type AvatarSession struct {
	// contains filtered or unexported fields
}

AvatarSession provisions and terminates one paid provider session through the LiveKit inference gateway. It is safe for concurrent inspection and Close calls. A failed create may be retried on the same instance using the same idempotency key; a successful create may never be started again.

func NewAvatarSession 

func NewAvatarSession(options AvatarSessionOptions) (*AvatarSession, error)

NewAvatarSession validates options without starting network or background work, keeping import and process start-up cost minimal.

func (*AvatarSession) AvatarIdentity 

func (s *AvatarSession) AvatarIdentity() string

func (*AvatarSession) AvatarName 

func (s *AvatarSession) AvatarName() string

func (*AvatarSession) Close 

func (s *AvatarSession) Close(ctx context.Context) error

Close terminates a provider session when the gateway supplied both required fields. Concurrent callers share one operation. A failed operation is returned to all of its waiters and a later Close call may retry it.

func (*AvatarSession) Provider 

func (s *AvatarSession) Provider() string

func (*AvatarSession) ProviderSessionID 

func (s *AvatarSession) ProviderSessionID() string

func (*AvatarSession) SessionID 

func (s *AvatarSession) SessionID() string

func (*AvatarSession) SessionInfo 

func (s *AvatarSession) SessionInfo() AvatarSessionInfo

func (*AvatarSession) Start 

func (s *AvatarSession) Start(ctx context.Context, options AvatarSessionStartOptions) (AvatarSessionInfo, error)

Start creates the remote session. The start claim is taken before any blocking work so overlapping callers cannot create two billed sessions.

func (*AvatarSession) Started 

func (s *AvatarSession) Started() bool

type AvatarSessionInfo 

type AvatarSessionInfo struct {
	SessionID         string `json:"session_id,omitempty"`
	ProviderSessionID string `json:"provider_session_id,omitempty"`
	AvatarIdentity    string `json:"avatar_identity,omitempty"`
	SampleRate        int    `json:"sample_rate,omitempty"`
}

AvatarSessionInfo is the authoritative gateway response retained by a successfully created AvatarSession.

type AvatarSessionOptions 

type AvatarSessionOptions struct {
	Model                     string
	AvatarParticipantIdentity string
	AvatarParticipantName     string
	LemonSlice                LemonSliceOptions
	ExtraKwargs               map[string]any
	BaseURL                   string
	Credentials               Credentials
	HTTPClient                *http.Client
	ConnectOptions            agents.APIConnectOptions
	Metadata                  RequestMetadata
	IdempotencyKey            string
}

AvatarSessionOptions configures the low-level inference gateway lifecycle. Media routing is intentionally supplied by voice/avatar.InferenceSession to avoid a Go package cycle between inference and voice.

type AvatarSessionStartOptions 

type AvatarSessionStartOptions struct {
	LiveKitURL    string
	RoomName      string
	RoomSID       string
	AgentIdentity string
}

AvatarSessionStartOptions are the room values used by the gateway to mint the avatar worker token. All values are required.

type Class 

type Class string

Class controls gateway scheduling. Low-priority work may yield to voice traffic and should only be used when no caller is synchronously waiting. 

const (
	ClassPriority Class = "priority"
	ClassStandard Class = "standard"
	ClassLow      Class = "low"
)

type CloudTurnTransportOptions 

type CloudTurnTransportOptions struct {
	BaseURL         string
	Credentials     Credentials
	ConnectOptions  agents.APIConnectOptions
	Metadata        func(context.Context) RequestMetadata
	WebSocketDialer *websocket.Dialer
	SendCapacity    int
	WriteTimeout    time.Duration
}

CloudTurnTransportOptions contains only cloud-specific state. Credentials are intentionally omitted from JSON snapshots by TurnDetector.MarshalJSON.

type ConnectionSettings 

type ConnectionSettings struct {
	TimeoutSeconds float64 `json:"timeout"`
	Retries        int     `json:"retries"`
}

type Credentials 

type Credentials struct {
	APIKey    agents.SecretString
	APISecret agents.SecretString
}

Credentials are resolved without retaining plaintext in formatted output. Explicit values take precedence over inference-specific and shared LiveKit environment variables.

func (Credentials) Resolve 

func (c Credentials) Resolve() (Credentials, error)

type EOTInferenceInput 

type EOTInferenceInput struct {
	PCM string `json:"pcm"`
}

EOTInferenceInput/Output are the shared-runner wire contract used by the local mini turn detector. PCM contains base64-encoded 16 kHz s16le audio so process IPC remains compact and language-compatible.

type EOTInferenceOutput 

type EOTInferenceOutput struct {
	Probability         float64 `json:"probability"`
	InferenceDurationMS float64 `json:"inferenceDurationMs"`
}

type FlushSentinel 

type FlushSentinel struct {
	Reason string
}

FlushSentinel marks a hard turn boundary for transports.

type InferenceVADStream 

type InferenceVADStream struct {
	*vadpkg.BaseStream
	// contains filtered or unexported fields
}

InferenceVADStream performs bounded windowing and speech-state detection.

func (*InferenceVADStream) Close 

func (s *InferenceVADStream) Close() error

func (*InferenceVADStream) Wait 

func (s *InferenceVADStream) Wait(ctx context.Context) error

type InterruptionDetectionError 

type InterruptionDetectionError struct {
	Timestamp   time.Time
	Label       string
	Err         error
	Recoverable bool
}

InterruptionDetectionError is emitted separately from stream termination so applications can observe recoverable retries without consuming the stream.

func (InterruptionDetectionError) Error 

func (e InterruptionDetectionError) Error() string

func (InterruptionDetectionError) Unwrap 

func (e InterruptionDetectionError) Unwrap() error

type InterruptionStream 

type InterruptionStream struct {
	// contains filtered or unexported fields
}

InterruptionStream is a bounded, cancellation-aware overlap detector stream.

func (*InterruptionStream) AgentSpeechEnded 

func (s *InterruptionStream) AgentSpeechEnded(ctx context.Context) error

func (*InterruptionStream) AgentSpeechStarted 

func (s *InterruptionStream) AgentSpeechStarted(ctx context.Context) error

func (*InterruptionStream) Close 

func (s *InterruptionStream) Close() error

func (*InterruptionStream) EndInput 

func (s *InterruptionStream) EndInput(ctx context.Context) error

func (*InterruptionStream) Flush 

func (s *InterruptionStream) Flush(ctx context.Context) error

func (*InterruptionStream) OverlapSpeechEnded 

func (s *InterruptionStream) OverlapSpeechEnded(ctx context.Context, endedAt time.Time, agentEnded bool) error

func (*InterruptionStream) OverlapSpeechStarted 

func (s *InterruptionStream) OverlapSpeechStarted(ctx context.Context, speechDuration time.Duration, startedAt time.Time) error

func (*InterruptionStream) PushAudio 

func (s *InterruptionStream) PushAudio(ctx context.Context, frame agents.AudioFrame) error

func (*InterruptionStream) Recv 

func (s *InterruptionStream) Recv(ctx context.Context) (OverlappingSpeechEvent, error)

func (*InterruptionStream) Wait 

func (s *InterruptionStream) Wait(ctx context.Context) error

type InterruptionStreamOptions 

type InterruptionStreamOptions struct {
	ConnectOptions agents.APIConnectOptions
	InputCapacity  int
	OutputCapacity int
}

InterruptionStreamOptions overrides per-stream connection and queue settings.

type LLM 

type LLM struct {
	*llmpkg.Base
	// contains filtered or unexported fields
}

LLM is an OpenAI-compatible LiveKit Inference client implemented directly on net/http to keep the dependency graph and process startup small.

func LLMFromModelString 

func LLMFromModelString(model string) (*LLM, error)

func NewLLM 

func NewLLM(options LLMOptions) (*LLM, error)

func (*LLM) Chat 

func (l *LLM) Chat(ctx context.Context, options llmpkg.ChatOptions) (llmpkg.LLMStream, error)

func (*LLM) Close 

func (l *LLM) Close(ctx context.Context) error

func (*LLM) UpdateOptions 

func (l *LLM) UpdateOptions(model string, modelOptions ModelOptions) error

UpdateOptions changes the persistent configuration used by subsequent chat calls. ModelOptions replaces, rather than merges, the previous options.

type LLMOptions 

type LLMOptions struct {
	Model            string
	Provider         string
	BaseURL          string
	Credentials      Credentials
	ModelOptions     ModelOptions
	StrictToolSchema bool
	InferenceClass   Class
	HTTPClient       *http.Client
	Metadata         func(context.Context) RequestMetadata
}

type LemonSliceOptions 

type LemonSliceOptions struct {
	ImageURL    string
	Prompt      string
	IdlePrompt  string
	IdleTimeout *time.Duration
}

LemonSliceOptions contains the gateway's first-class LemonSlice fields. Additional provider options belong in AvatarSessionOptions.ExtraKwargs.

type LocalEOTPredictor 

type LocalEOTPredictor interface {
	PredictEndOfTurn(context.Context, []int16) (float64, error)
	Close(context.Context) error
}

LocalEOTPredictor is the native/local model seam. Implementations should be safe for serialized calls; the SDK ensures at most one active prediction per stream and shares one lazily-created predictor across detector streams.

type LocalEOTPredictorFactory 

type LocalEOTPredictorFactory func(context.Context) (LocalEOTPredictor, error)

LocalEOTPredictorFactory lazily loads a shared mini-model predictor.

type LocalEOTPredictorFunc 

type LocalEOTPredictorFunc func(context.Context, []int16) (float64, error)

func (LocalEOTPredictorFunc) Close 

func (LocalEOTPredictorFunc) Close(context.Context) error

func (LocalEOTPredictorFunc) PredictEndOfTurn 

func (f LocalEOTPredictorFunc) PredictEndOfTurn(ctx context.Context, pcm []int16) (float64, error)

type ModelOptions 

type ModelOptions map[string]any

func MergeSTTKeyterms 

func MergeSTTKeyterms(model string, options ModelOptions, sessionTerms []string) (ModelOptions, bool)

MergeSTTKeyterms returns a provider-formatted shallow overlay and never mutates caller-owned options. Duplicate terms preserve first-seen order.

type OverlappingSpeechEvent 

type OverlappingSpeechEvent struct {
	Type               string        `json:"type"`
	CreatedAt          time.Time     `json:"createdAt"`
	DetectedAt         time.Time     `json:"detectedAt"`
	IsInterruption     bool          `json:"isInterruption"`
	AgentEnded         bool          `json:"agentEnded,omitempty"`
	TotalDuration      time.Duration `json:"-"`
	PredictionDuration time.Duration `json:"-"`
	DetectionDelay     time.Duration `json:"-"`
	OverlapStartedAt   *time.Time    `json:"-"`
	SpeechInput        []int16       `json:"-"`
	Probabilities      []float32     `json:"-"`
	Probability        float64       `json:"probability"`
	NumRequests        int64         `json:"numRequests"`
}

OverlappingSpeechEvent is the adaptive detector's interruption/backchannel verdict. Raw inference input is retained for diagnostics but omitted from JSON, matching the Python serializer.

func (OverlappingSpeechEvent) MarshalJSON 

func (event OverlappingSpeechEvent) MarshalJSON() ([]byte, error)

type RequestMetadata 

type RequestMetadata struct {
	RoomID      string
	JobID       string
	AgentID     string
	WorkerToken agents.SecretString
}

RequestMetadata mirrors the contextual headers added by the JavaScript and Python SDKs without coupling this package to a global job context.

type STT 

type STT struct {
	*stt.Base
	// contains filtered or unexported fields
}

STT implements stt.STT using the LiveKit Inference WebSocket protocol.

func NewSTT 

func NewSTT(options STTOptions) (*STT, error)

func STTFromModelString 

func STTFromModelString(model string) (*STT, error)

STTFromModelString parses provider/model:language and constructs an STT.

func (*STT) Close 

func (s *STT) Close(ctx context.Context) error

func (*STT) Recognize 

func (s *STT) Recognize(context.Context, []agents.AudioFrame, stt.RecognizeOptions) (stt.SpeechEvent, error)

func (*STT) Stream 

func (s *STT) Stream(ctx context.Context, options stt.StreamOptions) (stt.SpeechStream, error)

func (*STT) UpdateOptions 

func (s *STT) UpdateOptions(update STTUpdateOptions) error

UpdateOptions shallow-merges provider options and propagates a session.update to every active stream without reconnecting it.

func (*STT) UpdateSessionKeyterms 

func (s *STT) UpdateSessionKeyterms(keyterms []string) error

UpdateSessionKeyterms overlays framework-managed terms on user options. For an active utterance the update is deferred until EndOfSpeech.

type STTFallback 

type STTFallback struct {
	Models []STTFallbackModel `json:"models"`
}

type STTFallbackModel 

type STTFallbackModel struct {
	Model string       `json:"model"`
	Extra ModelOptions `json:"extra,omitempty"`
}

func STTFallbackFromString 

func STTFallbackFromString(value string) STTFallbackModel

STTFallbackFromString discards a language suffix just like agents-js. The primary stream language remains authoritative for all fallback models.

func (STTFallbackModel) MarshalJSON 

func (model STTFallbackModel) MarshalJSON() ([]byte, error)

MarshalJSON keeps the gateway's fallback shape exact: extra is always an object, even when the provider has no fallback-specific options.

type STTOptions 

type STTOptions struct {
	Model           string
	Language        agents.LanguageCode
	Encoding        string
	SampleRate      int
	BaseURL         string
	Credentials     Credentials
	ModelOptions    ModelOptions
	Fallback        []STTFallbackModel
	ConnectOptions  agents.APIConnectOptions
	Metadata        func(context.Context) RequestMetadata
	WebSocketDialer *websocket.Dialer
	InputCapacity   int
	OutputCapacity  int
	MaxMessageBytes int64
	SessionKeyterms []string
}

STTOptions configures the LiveKit Cloud Inference streaming STT adapter. Model may be "auto" or empty to let the gateway select a provider.

type STTServerEvent 

type STTServerEvent struct {
	Type       string          `json:"type"`
	SessionID  string          `json:"session_id,omitempty"`
	Transcript string          `json:"transcript,omitempty"`
	Language   string          `json:"language,omitempty"`
	Start      float64         `json:"start,omitempty"`
	Duration   float64         `json:"duration,omitempty"`
	Confidence float64         `json:"confidence,omitempty"`
	Words      []STTWord       `json:"words,omitempty"`
	SpeakerID  *string         `json:"speaker_id,omitempty"`
	Extra      json.RawMessage `json:"extra,omitempty"`
	Message    string          `json:"message,omitempty"`
	Code       *int            `json:"code,omitempty"`
	Unknown    json.RawMessage `json:"-"`
}

func DecodeSTTServerEvent 

func DecodeSTTServerEvent(data []byte) (STTServerEvent, error)

func (STTServerEvent) Known 

func (event STTServerEvent) Known() bool

type STTSessionCreate 

type STTSessionCreate struct {
	Type       string              `json:"type"`
	Model      string              `json:"model,omitempty"`
	Settings   STTSettings         `json:"settings"`
	Fallback   *STTFallback        `json:"fallback,omitempty"`
	Connection *ConnectionSettings `json:"connection,omitempty"`
}

func (STTSessionCreate) MarshalJSON 

func (event STTSessionCreate) MarshalJSON() ([]byte, error)

type STTSessionUpdate 

type STTSessionUpdate struct {
	Type     string            `json:"type"`
	Settings STTUpdateSettings `json:"settings"`
}

func (STTSessionUpdate) MarshalJSON 

func (event STTSessionUpdate) MarshalJSON() ([]byte, error)

type STTSettings 

type STTSettings struct {
	SampleRate string       `json:"sample_rate"`
	Encoding   string       `json:"encoding"`
	Language   string       `json:"language,omitempty"`
	Extra      ModelOptions `json:"extra"`
}

type STTUpdateOptions 

type STTUpdateOptions struct {
	Model        *string
	Language     *agents.LanguageCode
	ModelOptions ModelOptions
}

STTUpdateOptions is a sparse live update. Pointer fields distinguish an omitted value from an explicit empty value; ModelOptions are shallow-merged.

type STTUpdateSettings 

type STTUpdateSettings struct {
	Model    string       `json:"model,omitempty"`
	Language string       `json:"language,omitempty"`
	Extra    ModelOptions `json:"extra,omitempty"`
}

STTUpdateSettings is the hot-path settings payload accepted by the gateway. All fields are optional so callers can update one setting without resetting the others.

type STTWord 

type STTWord struct {
	Word       string          `json:"word"`
	Start      float64         `json:"start"`
	End        float64         `json:"end"`
	Confidence float64         `json:"confidence"`
	SpeakerID  *string         `json:"speaker_id,omitempty"`
	Extra      json.RawMessage `json:"extra,omitempty"`
}

type SpeechStream 

type SpeechStream struct {
	*stt.BaseStream
	// contains filtered or unexported fields
}

func (*SpeechStream) Wait 

func (s *SpeechStream) Wait(ctx context.Context) error

type StreamingTurnDetectionTransport 

type StreamingTurnDetectionTransport interface {
	Attach(*TurnDetectorStream)
	Run(context.Context) error
	RunInference(context.Context, string) error
	PushFrame(context.Context, agents.AudioFrame) error
	Flush(context.Context, FlushSentinel) error
	Close() error
}

StreamingTurnDetectionTransport is the extension seam used by the cloud and local transports. Run owns audio draining until the context is cancelled.

type SynthesizeStream 

type SynthesizeStream struct {
	*tts.BaseSynthesizeStream
	// contains filtered or unexported fields
}

func (*SynthesizeStream) Wait 

func (s *SynthesizeStream) Wait(ctx context.Context) error

type TTS 

type TTS struct {
	*tts.Base
	// contains filtered or unexported fields
}

TTS implements tts.TTS using pooled LiveKit Inference WebSockets.

func NewTTS 

func NewTTS(options TTSOptions) (*TTS, error)

func TTSFromModelString 

func TTSFromModelString(model string) (*TTS, error)

TTSFromModelString parses provider/model:voice and constructs a TTS.

func (*TTS) Capabilities 

func (t *TTS) Capabilities() tts.Capabilities

Capabilities reflects live option updates, including provider flags which enable alignment after construction.

func (*TTS) Close 

func (t *TTS) Close(ctx context.Context) error

func (*TTS) Prewarm 

func (t *TTS) Prewarm(ctx context.Context)

Prewarm opens one authenticated gateway connection in the background.

func (*TTS) Stream 

func (t *TTS) Stream(ctx context.Context, options tts.StreamOptions) (tts.SynthesizeStream, error)

func (*TTS) Synthesize 

func (t *TTS) Synthesize(context.Context, string, tts.SynthesizeOptions) (tts.ChunkedStream, error)

func (*TTS) UpdateOptions 

func (t *TTS) UpdateOptions(update TTSUpdateOptions) error

UpdateOptions updates future pooled sessions and live generation settings. Stale idle connections are rejected by generation on their next checkout; checked-out streams remain usable and receive generation_config updates.

type TTSAlignmentDecoder 

type TTSAlignmentDecoder interface {
	DecodeTTSAlignment(context.Context, TTSServerEvent, string) ([]agents.TimedString, error)
}

TTSAlignmentDecoder is the provider-alignment extension seam used by the inference TTS adapter. Implementations must not retain event slices and should return a newly owned result. The context is cancelled with the synthesis stream.

func DefaultTTSAlignmentDecoder 

func DefaultTTSAlignmentDecoder() TTSAlignmentDecoder

DefaultTTSAlignmentDecoder returns the stateless Cloud Inference provider decoder. A function keeps the process-wide default immutable and race-free.

type TTSAlignmentDecoderFunc 

type TTSAlignmentDecoderFunc func(context.Context, TTSServerEvent, string) ([]agents.TimedString, error)

TTSAlignmentDecoderFunc adapts a function into TTSAlignmentDecoder.

func (TTSAlignmentDecoderFunc) DecodeTTSAlignment 

func (f TTSAlignmentDecoderFunc) DecodeTTSAlignment(ctx context.Context, event TTSServerEvent, provider string) ([]agents.TimedString, error)

type TTSCharTimestamp 

type TTSCharTimestamp struct {
	Char  string  `json:"char"`
	Start float64 `json:"start"`
	End   float64 `json:"end"`
}

type TTSFallback 

type TTSFallback struct {
	Models []TTSFallbackModel `json:"models"`
}

type TTSFallbackModel 

type TTSFallbackModel struct {
	Model string       `json:"model"`
	Voice string       `json:"voice"`
	Extra ModelOptions `json:"extra,omitempty"`
}

func TTSFallbackFromString 

func TTSFallbackFromString(value string) TTSFallbackModel

func (TTSFallbackModel) MarshalJSON 

func (model TTSFallbackModel) MarshalJSON() ([]byte, error)

MarshalJSON keeps the gateway's fallback shape exact: extra is always an object, even when the provider has no fallback-specific options.

type TTSGenerationConfig 

type TTSGenerationConfig struct {
	Voice    string `json:"voice,omitempty"`
	Language string `json:"language,omitempty"`
	Model    string `json:"model,omitempty"`
}

type TTSInputTranscript 

type TTSInputTranscript struct {
	Type             string               `json:"type"`
	Transcript       string               `json:"transcript"`
	GenerationConfig *TTSGenerationConfig `json:"generation_config,omitempty"`
	Extra            ModelOptions         `json:"extra,omitempty"`
}

func (TTSInputTranscript) MarshalJSON 

func (event TTSInputTranscript) MarshalJSON() ([]byte, error)

type TTSOptions 

type TTSOptions struct {
	Model              string
	Voice              string
	Language           agents.LanguageCode
	Encoding           string
	SampleRate         int
	BaseURL            string
	Credentials        Credentials
	ModelOptions       ModelOptions
	Fallback           []TTSFallbackModel
	ConnectOptions     agents.APIConnectOptions
	Metadata           func(context.Context) RequestMetadata
	WebSocketDialer    *websocket.Dialer
	InputCapacity      int
	OutputCapacity     int
	MaxMessageBytes    int64
	MaxSessionDuration time.Duration
	// AlignmentDecoder customizes provider timestamp decoding. Nil uses the
	// cross-SDK Cloud Inference word/character decoder.
	AlignmentDecoder TTSAlignmentDecoder
	// MaxPendingAlignmentTokens bounds timestamp messages received before an
	// audio frame. Zero selects a production default.
	MaxPendingAlignmentTokens int
}

TTSOptions configures the LiveKit Cloud Inference streaming TTS adapter.

type TTSServerEvent 

type TTSServerEvent struct {
	Type      string             `json:"type"`
	SessionID string             `json:"session_id,omitempty"`
	Audio     string             `json:"audio,omitempty"`
	Message   string             `json:"message,omitempty"`
	Words     []TTSWordTimestamp `json:"words,omitempty"`
	Chars     []TTSCharTimestamp `json:"chars,omitempty"`
	Unknown   json.RawMessage    `json:"-"`
}

func DecodeTTSServerEvent 

func DecodeTTSServerEvent(data []byte) (TTSServerEvent, error)

func (TTSServerEvent) Known 

func (event TTSServerEvent) Known() bool

type TTSSessionCreate 

type TTSSessionCreate struct {
	Type       string              `json:"type"`
	SampleRate string              `json:"sample_rate"`
	Encoding   string              `json:"encoding"`
	Model      string              `json:"model,omitempty"`
	Voice      string              `json:"voice,omitempty"`
	Language   string              `json:"language,omitempty"`
	Extra      ModelOptions        `json:"extra"`
	Transcript string              `json:"transcript,omitempty"`
	Fallback   *TTSFallback        `json:"fallback,omitempty"`
	Connection *ConnectionSettings `json:"connection,omitempty"`
}

func (TTSSessionCreate) MarshalJSON 

func (event TTSSessionCreate) MarshalJSON() ([]byte, error)

type TTSUpdateOptions 

type TTSUpdateOptions struct {
	Model        *string
	Voice        *string
	Language     *agents.LanguageCode
	ModelOptions ModelOptions
}

TTSUpdateOptions is a sparse update. Pointer fields distinguish omission from an explicit empty value; ModelOptions are shallow-merged.

type TTSWordTimestamp 

type TTSWordTimestamp struct {
	Word  string  `json:"word"`
	Start float64 `json:"start"`
	End   float64 `json:"end"`
}

type ThresholdOptions 

type ThresholdOptions struct {
	// contains filtered or unexported fields
}

ThresholdOptions resolves the three threshold layers used by the audio turn detector: caller overrides, cloud/shipped defaults, and the materialized lookup table. It is safe for concurrent reads and runtime updates.

func NewThresholdOptions 

func NewThresholdOptions(model TurnDetectorModel, unlikely, backchannel ThresholdOverride) (*ThresholdOptions, error)

func (*ThresholdOptions) BackchannelOverrides 

func (o *ThresholdOptions) BackchannelOverrides() ThresholdOverride

func (*ThresholdOptions) DefaultThreshold 

func (o *ThresholdOptions) DefaultThreshold() (float64, bool)

func (*ThresholdOptions) Lookup 

func (o *ThresholdOptions) Lookup(language agents.LanguageCode) (float64, bool)

func (*ThresholdOptions) LookupBackchannel 

func (o *ThresholdOptions) LookupBackchannel(language agents.LanguageCode) (float64, bool)

func (*ThresholdOptions) Model 

func (o *ThresholdOptions) Model() TurnDetectorModel

func (*ThresholdOptions) Overrides 

func (o *ThresholdOptions) Overrides() ThresholdOverride

func (*ThresholdOptions) Snapshot 

func (o *ThresholdOptions) Snapshot() ThresholdSnapshot

func (*ThresholdOptions) Supports 

func (o *ThresholdOptions) Supports(language agents.LanguageCode) bool

func (*ThresholdOptions) Thresholds 

func (o *ThresholdOptions) Thresholds() map[string]float64

func (*ThresholdOptions) ToLocalFallback 

func (o *ThresholdOptions) ToLocalFallback()

ToLocalFallback performs the one-way cloud-to-mini transition. When cloud defaults are known, effective thresholds are rescaled per language so a caller override keeps the same ratio to the model's calibration.

func (*ThresholdOptions) UpdateBackchannelOverrides 

func (o *ThresholdOptions) UpdateBackchannelOverrides(value ThresholdOverride) error

func (*ThresholdOptions) UpdateOverrides 

func (o *ThresholdOptions) UpdateOverrides(value ThresholdOverride) error

func (*ThresholdOptions) UpdateServerDefaults 

func (o *ThresholdOptions) UpdateServerDefaults(thresholds map[string]float32, fallback float32, backchannel map[string]float32, backchannelFallback float32) error

UpdateServerDefaults adopts calibrated gateway values. A degenerate session is deliberately an error even with a caller override: it signals an incompatible or unhealthy gateway and triggers local fallback.

type ThresholdOverride 

type ThresholdOverride struct {
	Scalar    *float64           `json:"-"`
	Languages map[string]float64 `json:"-"`
}

ThresholdOverride represents the scalar-or-language-map union exposed by agents-js. A scalar applies to every language. Language values are layered over calibrated defaults. The zero value means no override.

Use ScalarThreshold when zero is an intentional override; Scalar is a pointer specifically to preserve unset versus an explicit zero.

func LanguageThresholds 

func LanguageThresholds(values map[string]float64) ThresholdOverride

LanguageThresholds returns an owned, normalized-on-use threshold map.

func ScalarThreshold 

func ScalarThreshold(value float64) ThresholdOverride

ScalarThreshold applies one threshold to every language, including zero.

func (ThresholdOverride) IsZero 

func (o ThresholdOverride) IsZero() bool

func (ThresholdOverride) MarshalJSON 

func (o ThresholdOverride) MarshalJSON() ([]byte, error)

MarshalJSON preserves the scalar-or-map union used by the TypeScript and Python SDKs instead of leaking the Go representation into config snapshots.

type ThresholdSnapshot 

type ThresholdSnapshot struct {
	Model                TurnDetectorModel  `json:"model"`
	Thresholds           map[string]float64 `json:"thresholds"`
	DefaultThreshold     *float64           `json:"defaultThreshold,omitempty"`
	Overrides            ThresholdOverride  `json:"overrides,omitzero"`
	BackchannelOverrides ThresholdOverride  `json:"backchannelOverrides,omitzero"`
}

ThresholdSnapshot is safe to serialize and never contains credentials or internal server state.

type TurnDetectionEvent 

type TurnDetectionEvent struct {
	Type                   string         `json:"type"`
	EndOfTurnProbability   float64        `json:"endOfTurnProbability"`
	LastSpeakingTime       time.Time      `json:"-"`
	LastSpeakingTimeMillis int64          `json:"lastSpeakingTimeMs"`
	DetectionDelay         *time.Duration `json:"-"`
	InferenceDuration      *time.Duration `json:"-"`
	BackchannelProbability *float64       `json:"backchannelProbability,omitempty"`
}

TurnDetectionEvent is emitted for one audio EOT prediction. Optional timing fields are pointers so an actual zero remains distinct from "not reported".

func (TurnDetectionEvent) MarshalJSON 

func (event TurnDetectionEvent) MarshalJSON() ([]byte, error)

type TurnDetector 

type TurnDetector struct {
	// contains filtered or unexported fields
}

TurnDetector implements the unified cloud v1 -> local v1-mini detector.

func NewTurnDetector 

func NewTurnDetector(options TurnDetectorOptions) (*TurnDetector, error)

func (*TurnDetector) BackchannelThreshold 

func (d *TurnDetector) BackchannelThreshold(language agents.LanguageCode) (float64, bool)

func (*TurnDetector) Close 

func (d *TurnDetector) Close(ctx context.Context) error

func (*TurnDetector) MarshalJSON 

func (d *TurnDetector) MarshalJSON() ([]byte, error)

func (*TurnDetector) Model 

func (d *TurnDetector) Model() string

func (*TurnDetector) OnMetrics 

func (d *TurnDetector) OnMetrics(fn func(metrics.EOTInference)) func()

func (*TurnDetector) Provider 

func (d *TurnDetector) Provider() string

func (*TurnDetector) SampleRate 

func (d *TurnDetector) SampleRate() int

func (*TurnDetector) Stream 

func (d *TurnDetector) Stream(parent context.Context, options ...TurnDetectorStreamOptions) (*TurnDetectorStream, error)

func (*TurnDetector) SupportsLanguage 

func (d *TurnDetector) SupportsLanguage(language agents.LanguageCode) bool

func (*TurnDetector) Thresholds 

func (d *TurnDetector) Thresholds() map[string]float64

func (*TurnDetector) UnlikelyThreshold 

func (d *TurnDetector) UnlikelyThreshold(language agents.LanguageCode) (float64, bool)

func (*TurnDetector) UpdateOptions 

func (d *TurnDetector) UpdateOptions(options TurnDetectorUpdateOptions) error

type TurnDetectorModel 

type TurnDetectorModel string

TurnDetectorModel is the model identifier reported in metrics and usage. 

const (
	TurnDetectorModelV1     TurnDetectorModel = "turn-detector-v1"
	TurnDetectorModelV1Mini TurnDetectorModel = "turn-detector-v1-mini"
)

type TurnDetectorOptions 

type TurnDetectorOptions struct {
	Version               TurnDetectorVersion
	UnlikelyThreshold     ThresholdOverride
	BackchannelThreshold  ThresholdOverride
	BaseURL               string
	Credentials           Credentials
	SampleRate            int
	ConnectOptions        agents.APIConnectOptions
	Metadata              func(context.Context) RequestMetadata
	WebSocketDialer       *websocket.Dialer
	LocalPredictor        LocalEOTPredictor
	LocalPredictorFactory LocalEOTPredictorFactory
	// Executor selects the worker-global runner registered under
	// EOTInferenceMethod. When nil, local fallback discovers an executor from
	// the stream context propagated by JobContext.
	Executor               ipc.InferenceExecutor
	KeepLocalPredictorOpen bool
	InputCapacity          int
}

TurnDetectorOptions maps the agents-js TurnDetector constructor while using typed credentials and context-aware local inference.

type TurnDetectorStream 

type TurnDetectorStream struct {
	// contains filtered or unexported fields
}

TurnDetectorStream is a bounded per-session audio EOT stream.

func (*TurnDetectorStream) BackchannelThreshold 

func (s *TurnDetectorStream) BackchannelThreshold(language agents.LanguageCode) (float64, bool)

func (*TurnDetectorStream) BeginPrediction 

func (s *TurnDetectorStream) BeginPrediction(ctx context.Context) (*TurnPrediction, error)

func (*TurnDetectorStream) CancelInference 

func (s *TurnDetectorStream) CancelInference(timedOut bool)

func (*TurnDetectorStream) Close 

func (s *TurnDetectorStream) Close() error

func (*TurnDetectorStream) EndInput 

func (s *TurnDetectorStream) EndInput(ctx context.Context) error

func (*TurnDetectorStream) Flush 

func (s *TurnDetectorStream) Flush(ctx context.Context, reason string) error

func (*TurnDetectorStream) IsFallback 

func (s *TurnDetectorStream) IsFallback() bool

func (*TurnDetectorStream) Model 

func (s *TurnDetectorStream) Model() string

func (*TurnDetectorStream) Predict 

func (s *TurnDetectorStream) Predict(ctx context.Context) (TurnDetectionEvent, error)

func (*TurnDetectorStream) PredictionTimeout 

func (s *TurnDetectorStream) PredictionTimeout() time.Duration

func (*TurnDetectorStream) Provider 

func (s *TurnDetectorStream) Provider() string

func (*TurnDetectorStream) PushAudio 

func (s *TurnDetectorStream) PushAudio(ctx context.Context, frame agents.AudioFrame) error

func (*TurnDetectorStream) ResolvePrediction 

func (s *TurnDetectorStream) ResolvePrediction(requestID string, probability float64, details TurnPredictionDetails)

ResolvePrediction accepts a transport result and ignores stale request IDs.

func (*TurnDetectorStream) SupportsLanguage 

func (s *TurnDetectorStream) SupportsLanguage(language agents.LanguageCode) bool

func (*TurnDetectorStream) UnlikelyThreshold 

func (s *TurnDetectorStream) UnlikelyThreshold(language agents.LanguageCode) (float64, bool)

func (*TurnDetectorStream) Wait 

func (s *TurnDetectorStream) Wait(ctx context.Context) error

type TurnDetectorStreamOptions 

type TurnDetectorStreamOptions struct {
	ConnectOptions agents.APIConnectOptions
	InputCapacity  int
	Transport      StreamingTurnDetectionTransport
}

TurnDetectorStreamOptions overrides per-stream transport settings.

type TurnDetectorUpdateOptions 

type TurnDetectorUpdateOptions struct {
	UnlikelyThreshold    *ThresholdOverride
	BackchannelThreshold *ThresholdOverride
}

TurnDetectorUpdateOptions is a sparse live threshold update.

type TurnDetectorVersion 

type TurnDetectorVersion string

TurnDetectorVersion is the public model selection option. 

const (
	TurnDetectorV1     TurnDetectorVersion = "v1"
	TurnDetectorV1Mini TurnDetectorVersion = "v1-mini"
)

type TurnPrediction 

type TurnPrediction struct {
	// contains filtered or unexported fields
}

TurnPrediction is the Go equivalent of the SDK Future returned by predict(). Wait is cancellation-aware and a deadline triggers the documented cloud to local timeout fallback.

func (*TurnPrediction) Wait 

func (p *TurnPrediction) Wait(ctx context.Context) (TurnDetectionEvent, error)

type TurnPredictionDetails 

type TurnPredictionDetails struct {
	InferenceDuration      *time.Duration
	DetectionDelay         *time.Duration
	BackchannelProbability *float64
}

TurnPredictionDetails carries optional transport timing and backchannel data.

type VAD 

type VAD struct {
	// contains filtered or unexported fields
}

VAD implements vad.VAD through a lazily initialized local predictor.

func NewVAD 

func NewVAD(options VADOptions) (*VAD, error)

NewVAD validates options without loading the native model.

func (*VAD) Capabilities 

func (v *VAD) Capabilities() vadpkg.Capabilities

func (*VAD) Close 

func (v *VAD) Close(ctx context.Context) error

func (*VAD) Label 

func (v *VAD) Label() string

func (*VAD) MinSilenceDuration 

func (v *VAD) MinSilenceDuration() (time.Duration, bool)

func (*VAD) Model 

func (v *VAD) Model() string

func (*VAD) OnMetrics 

func (v *VAD) OnMetrics(fn func(metrics.VAD)) func()

func (*VAD) Provider 

func (v *VAD) Provider() string

func (*VAD) Stream 

func (v *VAD) Stream(parent context.Context) (vadpkg.VADStream, error)

func (*VAD) UpdateOptions 

func (v *VAD) UpdateOptions(update VADUpdateOptions) error

type VADInferenceInput 

type VADInferenceInput struct {
	StreamID  string                `json:"streamId"`
	Operation VADInferenceOperation `json:"operation"`
	PCM       []byte                `json:"pcm,omitempty"`
}

VADInferenceInput is the stable local-runner/IPC request. PCM contains little-endian 16 kHz signed PCM and is present only for predict.

type VADInferenceOperation 

type VADInferenceOperation string
const (
	// VADInferenceMethod is the shared-runner method for stateful Silero VAD.
	// A runner must isolate recurrent state by StreamID and implement init,
	// predict, reset, and close operations.
	VADInferenceMethod = "lk_vad"

	VADOperationInit    VADInferenceOperation = "init"
	VADOperationPredict VADInferenceOperation = "predict"
	VADOperationReset   VADInferenceOperation = "reset"
	VADOperationClose   VADInferenceOperation = "close"
)

type VADInferenceOutput 

type VADInferenceOutput struct {
	Probability   float64 `json:"probability"`
	WindowSamples int     `json:"windowSamples,omitempty"`
}

VADInferenceOutput returns one probability. Init may override the standard 512-sample Silero window for compatible runner implementations.

type VADModel 

type VADModel string

VADModel identifies a bundled local voice-activity model. 

const VADModelSilero VADModel = "silero"

VADModelSilero selects the cross-SDK Silero model.

type VADOptions 

type VADOptions struct {
	Model                  VADModel
	MinimumSpeechDuration  time.Duration
	MinimumSilenceDuration time.Duration
	PrefixPaddingDuration  time.Duration
	MaximumBufferedSpeech  time.Duration
	ActivationThreshold    float64
	DeactivationThreshold  *float64
	PredictorFactory       VADPredictorFactory
	// Executor selects the worker-global runner registered under
	// VADInferenceMethod. When nil, each stream discovers the executor from
	// its context, matching TurnDetector and process-isolated job behavior.
	// PredictorFactory and Executor are mutually exclusive.
	Executor       ipc.InferenceExecutor
	StreamCapacity int
}

VADOptions configures local inference VAD. Zero values select the pinned SDK defaults.

type VADPredictor 

type VADPredictor interface {
	WindowSamples() int
	Predict(context.Context, []int16) (float64, error)
	Reset() error
	Close() error
}

VADPredictor is the stateful local-model seam. A factory must create an independent predictor per stream because Silero carries recurrent state. Predict must not retain the supplied window.

type VADPredictorFactory 

type VADPredictorFactory func(context.Context) (VADPredictor, error)

VADPredictorFactory lazily creates one stateful predictor per stream.

type VADUpdateOptions 

type VADUpdateOptions struct {
	MinimumSpeechDuration  *time.Duration
	MinimumSilenceDuration *time.Duration
	PrefixPaddingDuration  *time.Duration
	MaximumBufferedSpeech  *time.Duration
	ActivationThreshold    *float64
	DeactivationThreshold  *float64
}

VADUpdateOptions is a sparse, concurrency-safe live update.

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.