frames

package
v0.2.0 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: May 16, 2026 License: Apache-2.0 Imports: 3 Imported by: 0

README

Frames

Package frames defines frame types for the Voxray pipeline: audio, text, system, control, and vendor-specific (RTVI, DTMF). The serialize subpackage provides wire serialization (JSON, protobuf, Twilio, Telnyx, etc.).

Purpose

  • Frame: Interface implemented by all frames; FrameType(), ID(), PTS(), Metadata().
  • Base: Common id, pts, metadata; embed in concrete frames. IDs are unique (atomic counter).
  • Concrete frames: System (Start, End, Cancel, Error, Stop), control (Sync, Interruption, user/bot turn), data (Text, Transcription, AudioRaw, TTSAudioRaw, LLM*, TransportMessage, DTMF, etc.).
  • Serialization: serialize.Serializer (Serialize/Deserialize); JSON envelope, protobuf, and vendor formats (Twilio, Telnyx, Plivo, Vonage, Exotel, Genesys).

Frame type hierarchy

graph TD
    Frame["Frame interface"] --> Base["Base\nid, pts, metadata"]
    Base --> System["SystemFrame"]
    Base --> Data["DataFrame"]
    Base --> Control["ControlFrame"]
    System --> Start["StartFrame"]
    System --> End["EndFrame"]
    System --> Cancel["CancelFrame"]
    System --> Error["ErrorFrame"]
    Data --> Text["TextFrame"]
    Data --> Transcription["TranscriptionFrame"]
    Data --> AudioRaw["AudioRawFrame"]
    Data --> TTSAudio["TTSAudioRawFrame"]
    Data --> LLMCtx["LLMContextFrame\nLLMTextFrame\n..."]
    Control --> Sync["SyncFrame"]
    Control --> Interruption["InterruptionFrame"]
    Control --> UserTurn["UserStartedSpeakingFrame\nUserStoppedSpeakingFrame\nUserIdleFrame"]
    Control --> BotTurn["BotStartedSpeakingFrame\nBotStoppedSpeakingFrame"]

Serializers

graph TD
    Serializer["Serializer interface"] --> JSON["JSONSerializer"]
    Serializer --> Protobuf["ProtobufSerializer"]
    Serializer --> Twilio["twilio.Serializer"]
    Serializer --> Telnyx["telnyx.Serializer"]
    Serializer --> Plivo["plivo.Serializer"]
    Serializer --> Vonage["vonage.Serializer"]
    Serializer --> Exotel["exotel.Serializer"]
    Serializer --> Genesys["genesys.Serializer"]

Exported symbols (root package)

Symbol Type Description
Frame interface FrameType, ID, PTS, Metadata
Base struct NewBase, NewBaseWithID, ID, SetPTS, Metadata
SystemFrame, DataFrame, ControlFrame struct Base embeddings
StartFrame, NewStartFrame struct/func Pipeline init; audio rates, flags
EndFrame, NewEndFrame struct/func Normal end
CancelFrame, NewCancelFrame struct/func Stop with reason
ErrorFrame, NewErrorFrame struct/func Error, Fatal, Processor
TextFrame, NewTextFrame struct/func Text, SkipTTS, AppendToContext
TranscriptionFrame, NewTranscriptionFrame struct/func STT output; UserID, Timestamp, Finalized
AudioRawFrame, NewAudioRawFrame struct/func PCM audio; SampleRate, NumChannels
TTSAudioRawFrame, NewTTSAudioRawFrame struct/func TTS output
LLMContext, LLMContextFrame, LLMRunFrame, LLMTextFrame, etc. struct/func LLM context, run, messages, tools, text
UserStartedSpeakingFrame, UserStoppedSpeakingFrame, UserIdleFrame struct/func User turn control
BotStartedSpeakingFrame, BotStoppedSpeakingFrame struct/func Bot turn control
InputDTMFFrame, OutputDTMFUrgentFrame struct/func DTMF I/O
RTVIClientMessageFrame, RTVIServerMessageFrame struct RTVI protocol
InterruptionFrame, NewInterruptionFrame struct/func Barge-in clear
Service switcher, transport message, etc. struct/func See frames.go, llm.go

Subpackage serialize

Symbol Description
Serializer Serialize(Frame) ([]byte, error), Deserialize([]byte) (Frame, error)
SerializerWithSetup Optional Setup(StartFrame)
SerializerWithMessageType Optional SerializeWithType → (data, binary)
JSONSerializer, ProtobufSerializer Built-in
twilio, telnyx, plivo, vonage, exotel, genesys Vendor Serializer + Params

Concurrency

  • Frame ID generation uses atomic.AddUint64; safe for concurrent frame creation.
  • Frames are intended to be immutable after creation (except metadata); no internal locking.

Files (root)

File Description
frames.go Frame, Base, system/data/control frames, audio, text, transcription, error, service switcher
llm.go LLMContext, LLM* frames, TTSSpeakFrame, FunctionCallResultFrame
user_turn.go UserStartedSpeaking, UserStoppedSpeaking, UserIdle
bot_turn.go BotStartedSpeaking, BotStoppedSpeaking
vad.go VADParamsUpdate, VADUserStarted/StoppedSpeaking, UserSpeakingFrame
dtmf.go InputDTMFFrame, OutputDTMFUrgentFrame
rtvi.go RTVIClientMessageFrame, RTVIServerMessageFrame

See also

Documentation

Overview

Package frames defines DTMF and IVR-related frame types.

Package frames defines frame types for the Voxray pipeline (audio, text, system, control).

Package frames defines VAD/turn-related control frames.

Index

Constants

This section is empty.

Variables

View Source 
var ValidKeypadRunes = "0123456789*#"

ValidKeypadRunes is the set of valid single-keypad characters.

Functions

This section is empty.

Types

type AggregatedTextFrame 

type AggregatedTextFrame struct {
	TextFrame
	AggregatedBy string `json:"aggregated_by,omitempty"`
}

AggregatedTextFrame is text emitted after aggregation (e.g. by IVR pattern aggregator); AggregatedBy names the aggregation type.

func NewAggregatedTextFrame 

func NewAggregatedTextFrame(text, aggregatedBy string) *AggregatedTextFrame

NewAggregatedTextFrame creates an AggregatedTextFrame.

func (*AggregatedTextFrame) FrameType 

func (*AggregatedTextFrame) FrameType() string

type AudioRawFrame 

type AudioRawFrame struct {
	DataFrame
	Audio       []byte `json:"-"` // not JSON-encoded in wire format typically; use base64 if needed
	SampleRate  int    `json:"sample_rate"`
	NumChannels int    `json:"num_channels"`
	NumFrames   int    `json:"num_frames"`
}

AudioRawFrame holds raw PCM audio.

func NewAudioRawFrame 

func NewAudioRawFrame(audio []byte, sampleRate, numChannels int, numFrames int) *AudioRawFrame

NewAudioRawFrame creates an AudioRawFrame; num_frames is derived from len(audio)/(channels*2) if 0.

func (*AudioRawFrame) FrameType 

func (*AudioRawFrame) FrameType() string

type Base 

type Base struct {
	// contains filtered or unexported fields
}

Base holds common frame fields. Embed in concrete frame types.

func NewBase 

func NewBase() Base

NewBase returns a Base with a new ID.

func NewBaseWithID 

func NewBaseWithID(id uint64) Base

NewBaseWithID returns a Base with the given ID (e.g. when restoring from wire).

func (*Base) ID 

func (b *Base) ID() uint64

ID returns the frame ID.

func (*Base) Metadata 

func (b *Base) Metadata() map[string]any

Metadata returns the metadata map.

func (*Base) PTS 

func (b *Base) PTS() *int64

PTS returns presentation timestamp (nanoseconds); may be nil.

func (*Base) SetPTS 

func (b *Base) SetPTS(ns int64)

SetPTS sets the presentation timestamp.

type BotStartedSpeakingFrame 

type BotStartedSpeakingFrame struct {
	ControlFrame
}

BotStartedSpeakingFrame signals that the bot has started speaking (e.g., first TTS audio is being sent). Used by observers for turn and latency tracking.

func NewBotStartedSpeakingFrame 

func NewBotStartedSpeakingFrame() *BotStartedSpeakingFrame

NewBotStartedSpeakingFrame creates a BotStartedSpeakingFrame.

func (*BotStartedSpeakingFrame) FrameType 

func (*BotStartedSpeakingFrame) FrameType() string

type BotStoppedSpeakingFrame 

type BotStoppedSpeakingFrame struct {
	ControlFrame
}

BotStoppedSpeakingFrame signals that the bot has stopped speaking (e.g., TTS has finished a segment). Used by observers for turn tracking.

func NewBotStoppedSpeakingFrame 

func NewBotStoppedSpeakingFrame() *BotStoppedSpeakingFrame

NewBotStoppedSpeakingFrame creates a BotStoppedSpeakingFrame.

func (*BotStoppedSpeakingFrame) FrameType 

func (*BotStoppedSpeakingFrame) FrameType() string

type CancelFrame 

type CancelFrame struct {
	SystemFrame
	Reason any `json:"reason,omitempty"`
}

CancelFrame signals the pipeline to stop.

func NewCancelFrame 

func NewCancelFrame(reason any) *CancelFrame

NewCancelFrame creates a CancelFrame.

func (*CancelFrame) FrameType 

func (*CancelFrame) FrameType() string

type ControlFrame 

type ControlFrame struct{ Base }

ControlFrame carries control information (pause, resume, etc.).

func (*ControlFrame) FrameType 

func (*ControlFrame) FrameType() string

type DataFrame 

type DataFrame struct{ Base }

DataFrame is processed in order and carries data (text, audio, etc.).

func (*DataFrame) FrameType 

func (*DataFrame) FrameType() string

type EndFrame 

type EndFrame struct {
	SystemFrame
}

EndFrame signals that the pipeline has ended normally (all processing complete).

func NewEndFrame 

func NewEndFrame() *EndFrame

NewEndFrame creates an EndFrame.

func (*EndFrame) FrameType 

func (*EndFrame) FrameType() string

type ErrorFrame 

type ErrorFrame struct {
	SystemFrame
	Error     string `json:"error"`
	Fatal     bool   `json:"fatal"`
	Processor string `json:"processor,omitempty"`
}

ErrorFrame notifies upstream of an error.

func NewErrorFrame 

func NewErrorFrame(err string, fatal bool, processor string) *ErrorFrame

NewErrorFrame creates an ErrorFrame.

func (*ErrorFrame) FrameType 

func (*ErrorFrame) FrameType() string

type Frame 

type Frame interface {
	FrameType() string
	ID() uint64
	PTS() *int64
	Metadata() map[string]any
}

Frame is the interface implemented by all pipeline frames.

type FunctionCallResultFrame 

type FunctionCallResultFrame struct {
	DataFrame
	FunctionName string `json:"function_name"`
	ToolCallID   string `json:"tool_call_id"`
	Arguments    string `json:"arguments,omitempty"`
	Result       any    `json:"result"`
	RunLLM       *bool  `json:"run_llm,omitempty"`
}

FunctionCallResultFrame is the result of a tool/function call.

func (*FunctionCallResultFrame) FrameType 

func (*FunctionCallResultFrame) FrameType() string

type InputDTMFFrame 

type InputDTMFFrame struct {
	ControlFrame
	Digit KeypadEntry `json:"digit"`
}

InputDTMFFrame carries an inbound DTMF keypress from the transport (e.g. telephony). Pipeline/processors can use this for IVR or user input.

func NewInputDTMFFrame 

func NewInputDTMFFrame(digit KeypadEntry) (*InputDTMFFrame, error)

NewInputDTMFFrame creates an InputDTMFFrame. Returns error if digit is not valid (0-9, *, #).

func (*InputDTMFFrame) FrameType 

func (*InputDTMFFrame) FrameType() string

type InterruptionFrame 

type InterruptionFrame struct {
	ControlFrame
}

InterruptionFrame signals the transport to clear the playback buffer (barge-in). Used by telephony serializers (Twilio, Telnyx, Plivo, Vonage, Exotel) for "clear" events.

func NewInterruptionFrame 

func NewInterruptionFrame() *InterruptionFrame

NewInterruptionFrame creates an InterruptionFrame.

func (*InterruptionFrame) FrameType 

func (*InterruptionFrame) FrameType() string

type KeypadEntry 

type KeypadEntry string

KeypadEntry represents a valid DTMF key (0-9, *, #). 

const (
	KeypadStar  KeypadEntry = "*"
	KeypadPound KeypadEntry = "#"
)

Valid DTMF characters.

func ParseKeypadEntry 

func ParseKeypadEntry(s string) (KeypadEntry, error)

ParseKeypadEntry parses a string into a KeypadEntry. The input may be a single character or multiple digits (e.g. "1", "123"). Each rune must be 0-9, *, or #. For multi-digit sequences, only the first character is used for the KeypadEntry; use ParseKeypadSequence for full sequences.

func (KeypadEntry) String 

func (k KeypadEntry) String() string

String returns the keypad character as string.

type LLMContext 

type LLMContext struct {
	Messages   []map[string]any `json:"messages"`
	Tools      []map[string]any `json:"tools,omitempty"`
	ToolChoice string           `json:"tool_choice,omitempty"` // "none", "auto", "required", or JSON object
}

LLMContext holds messages, optional tools, and tool choice for an LLM.

func (*LLMContext) AddAudioMessage 

func (c *LLMContext) AddAudioMessage(text, data, format string)

AddAudioMessage appends a user message with input_audio content. data is base64-encoded audio; format e.g. "wav".

func (*LLMContext) AddImageMessage 

func (c *LLMContext) AddImageMessage(text, url string)

AddImageMessage appends a user message with image_url content (e.g. for vision). url is data URL or HTTP URL.

type LLMContextFrame 

type LLMContextFrame struct {
	DataFrame
	Context *LLMContext `json:"context"`
}

LLMContextFrame carries context for the LLM (messages, tools).

func NewLLMContextFrame 

func NewLLMContextFrame(ctx *LLMContext) *LLMContextFrame

NewLLMContextFrame creates an LLMContextFrame.

func (*LLMContextFrame) FrameType 

func (*LLMContextFrame) FrameType() string

type LLMContextSummaryRequestFrame 

type LLMContextSummaryRequestFrame struct {
	DataFrame
	RequestID            string      `json:"request_id"`
	Context              *LLMContext `json:"context"`
	MinMessagesToKeep    int         `json:"min_messages_to_keep"`
	TargetContextTokens  int         `json:"target_context_tokens"`
	SummarizationPrompt  string      `json:"summarization_prompt,omitempty"`
	SummarizationTimeout int         `json:"summarization_timeout,omitempty"` // seconds
}

LLMContextSummaryRequestFrame requests context summarization (e.g. when token/message limit reached).

func (*LLMContextSummaryRequestFrame) FrameType 

func (*LLMContextSummaryRequestFrame) FrameType() string

type LLMContextSummaryResultFrame 

type LLMContextSummaryResultFrame struct {
	DataFrame
	RequestID           string `json:"request_id"`
	Summary             string `json:"summary"`
	LastSummarizedIndex int    `json:"last_summarized_index"`
	Error               string `json:"error,omitempty"`
}

LLMContextSummaryResultFrame carries the result of context summarization.

func (*LLMContextSummaryResultFrame) FrameType 

func (*LLMContextSummaryResultFrame) FrameType() string

type LLMFullResponseEndFrame 

type LLMFullResponseEndFrame struct {
	DataFrame
}

LLMFullResponseEndFrame marks the end of a complete LLM response stream. Processors that aggregate LLM text should flush on this frame.

func NewLLMFullResponseEndFrame 

func NewLLMFullResponseEndFrame() *LLMFullResponseEndFrame

NewLLMFullResponseEndFrame creates a LLMFullResponseEndFrame.

func (*LLMFullResponseEndFrame) FrameType 

func (*LLMFullResponseEndFrame) FrameType() string

type LLMFullResponseStartFrame 

type LLMFullResponseStartFrame struct {
	DataFrame
}

LLMFullResponseStartFrame marks the start of a complete LLM response stream. Used by extensions (e.g. voicemail classifier, IVR) to know when to begin aggregating tokens.

func NewLLMFullResponseStartFrame 

func NewLLMFullResponseStartFrame() *LLMFullResponseStartFrame

NewLLMFullResponseStartFrame creates a LLMFullResponseStartFrame.

func (*LLMFullResponseStartFrame) FrameType 

func (*LLMFullResponseStartFrame) FrameType() string

type LLMMessagesAppendFrame 

type LLMMessagesAppendFrame struct {
	DataFrame
	Messages []map[string]any `json:"messages"`
	RunLLM   *bool            `json:"run_llm,omitempty"`
}

LLMMessagesAppendFrame appends messages to context.

func (*LLMMessagesAppendFrame) FrameType 

func (*LLMMessagesAppendFrame) FrameType() string

type LLMMessagesUpdateFrame 

type LLMMessagesUpdateFrame struct {
	DataFrame
	Messages []map[string]any `json:"messages"`
	RunLLM   *bool            `json:"run_llm,omitempty"`
}

LLMMessagesUpdateFrame replaces current context messages.

func (*LLMMessagesUpdateFrame) FrameType 

func (*LLMMessagesUpdateFrame) FrameType() string

type LLMRunFrame 

type LLMRunFrame struct {
	DataFrame
}

LLMRunFrame triggers the LLM to run on current context.

func NewLLMRunFrame 

func NewLLMRunFrame() *LLMRunFrame

NewLLMRunFrame creates an LLMRunFrame.

func (*LLMRunFrame) FrameType 

func (*LLMRunFrame) FrameType() string

type LLMSetToolChoiceFrame 

type LLMSetToolChoiceFrame struct {
	DataFrame
	ToolChoice string `json:"tool_choice"` // "none", "auto", "required", or JSON object
}

LLMSetToolChoiceFrame sets tool choice (none, auto, required, or specific).

func (*LLMSetToolChoiceFrame) FrameType 

func (*LLMSetToolChoiceFrame) FrameType() string

type LLMSetToolsFrame 

type LLMSetToolsFrame struct {
	DataFrame
	Tools []map[string]any `json:"tools"`
}

LLMSetToolsFrame sets tools for function calling.

func (*LLMSetToolsFrame) FrameType 

func (*LLMSetToolsFrame) FrameType() string

type LLMTextFrame 

type LLMTextFrame struct {
	TextFrame
}

LLMTextFrame is text emitted by the LLM.

func (*LLMTextFrame) FrameType 

func (*LLMTextFrame) FrameType() string

type ManuallySwitchServiceFrame 

type ManuallySwitchServiceFrame struct {
	SystemFrame
	ServiceName string `json:"service_name"`
}

ManuallySwitchServiceFrame requests switching to a service by name.

func NewManuallySwitchServiceFrame 

func NewManuallySwitchServiceFrame(serviceName string) *ManuallySwitchServiceFrame

NewManuallySwitchServiceFrame creates a frame to request switching to the named service.

func (*ManuallySwitchServiceFrame) FrameType 

func (*ManuallySwitchServiceFrame) FrameType() string

type OutputAudioRawFrame 

type OutputAudioRawFrame struct {
	AudioRawFrame
	TransportDestination string `json:"transport_destination,omitempty"`
}

OutputAudioRawFrame is audio destined for transport output.

func (*OutputAudioRawFrame) FrameType 

func (*OutputAudioRawFrame) FrameType() string

type OutputDTMFUrgentFrame 

type OutputDTMFUrgentFrame struct {
	ControlFrame
	Button KeypadEntry `json:"button"`
}

OutputDTMFUrgentFrame carries a DTMF keypress for the transport to play (e.g. for IVR navigation). Transport implementations may emit the actual DTMF tone or forward it to the telephony layer.

func NewOutputDTMFUrgentFrame 

func NewOutputDTMFUrgentFrame(button KeypadEntry) (*OutputDTMFUrgentFrame, error)

NewOutputDTMFUrgentFrame creates an OutputDTMFUrgentFrame. Returns error if button is not a valid keypad entry (0-9, *, #).

func (*OutputDTMFUrgentFrame) FrameType 

func (*OutputDTMFUrgentFrame) FrameType() string

type RTVIClientMessageFrame 

type RTVIClientMessageFrame struct {
	SystemFrame
	MsgID string         `json:"msg_id"`
	Type  string         `json:"type"` // e.g. "client-ready", "send-text"
	Data  map[string]any `json:"data"` // message payload
}

RTVIClientMessageFrame carries a parsed RTVI client message from the transport into the pipeline. The transport/serializer parses RTVI JSON and pushes this frame; RTVIProcessor handles it (e.g. send-text -> TranscriptionFrame, client-ready -> store version).

func NewRTVIClientMessageFrame 

func NewRTVIClientMessageFrame(msgID, typ string, data map[string]any) *RTVIClientMessageFrame

NewRTVIClientMessageFrame creates an RTVIClientMessageFrame.

func (*RTVIClientMessageFrame) FrameType 

func (*RTVIClientMessageFrame) FrameType() string

type RTVIServerMessageFrame 

type RTVIServerMessageFrame struct {
	SystemFrame
	Type  string         `json:"type"` // e.g. "bot-ready", "error"
	MsgID string         `json:"id"`   // RTVI message id (wire field "id")
	Data  map[string]any `json:"data"`
}

RTVIServerMessageFrame carries an RTVI server message to be sent to the client. When the pipeline uses an RTVI serializer, this frame is serialized as RTVI JSON (label, type, id, data). RTVIProcessor pushes this for bot-ready and error so they go out over the transport.

func NewRTVIServerMessageFrame 

func NewRTVIServerMessageFrame(typ, msgID string, data map[string]any) *RTVIServerMessageFrame

NewRTVIServerMessageFrame creates an RTVIServerMessageFrame.

func (*RTVIServerMessageFrame) FrameType 

func (*RTVIServerMessageFrame) FrameType() string

type ServiceMetadataFrame 

type ServiceMetadataFrame struct {
	SystemFrame
	ServiceName string         `json:"service_name"`
	Meta        map[string]any `json:"metadata,omitempty"`
}

ServiceMetadataFrame carries metadata from a service (e.g. capabilities).

func NewServiceMetadataFrame 

func NewServiceMetadataFrame(serviceName string, meta map[string]any) *ServiceMetadataFrame

NewServiceMetadataFrame creates a ServiceMetadataFrame.

func (*ServiceMetadataFrame) FrameType 

func (*ServiceMetadataFrame) FrameType() string

type ServiceSwitcherRequestMetadataFrame 

type ServiceSwitcherRequestMetadataFrame struct {
	SystemFrame
	ServiceName string `json:"service_name"`
}

ServiceSwitcherRequestMetadataFrame requests metadata from a specific service.

func NewServiceSwitcherRequestMetadataFrame 

func NewServiceSwitcherRequestMetadataFrame(serviceName string) *ServiceSwitcherRequestMetadataFrame

NewServiceSwitcherRequestMetadataFrame creates a request for the named service's metadata.

func (*ServiceSwitcherRequestMetadataFrame) FrameType 

func (*ServiceSwitcherRequestMetadataFrame) FrameType() string

type StartFrame 

type StartFrame struct {
	SystemFrame
	AudioInSampleRate  int  `json:"audio_in_sample_rate"`
	AudioOutSampleRate int  `json:"audio_out_sample_rate"`
	AllowInterruptions bool `json:"allow_interruptions"`
	EnableMetrics      bool `json:"enable_metrics"`
	EnableUsageMetrics bool `json:"enable_usage_metrics"`
}

StartFrame initializes the pipeline (first frame pushed).

func NewStartFrame 

func NewStartFrame() *StartFrame

NewStartFrame creates a StartFrame with defaults.

func (*StartFrame) FrameType 

func (*StartFrame) FrameType() string

type StopFrame 

type StopFrame struct {
	SystemFrame
}

StopFrame signals the pipeline to stop (processors keep connections open).

func NewStopFrame 

func NewStopFrame() *StopFrame

NewStopFrame creates a StopFrame.

func (*StopFrame) FrameType 

func (*StopFrame) FrameType() string

type SyncFrame 

type SyncFrame struct {
	ControlFrame
}

SyncFrame is a control frame used to synchronize parallel pipeline processing. When a branch emits SyncFrame it signals that it has finished processing the current batch.

func NewSyncFrame 

func NewSyncFrame() *SyncFrame

NewSyncFrame creates a SyncFrame.

func (*SyncFrame) FrameType 

func (*SyncFrame) FrameType() string

type SystemFrame 

type SystemFrame struct{ Base }

SystemFrame is a high-priority frame (e.g. Start, Cancel, Error).

func (*SystemFrame) FrameType 

func (*SystemFrame) FrameType() string

type TTSAudioRawFrame 

type TTSAudioRawFrame struct {
	OutputAudioRawFrame
}

TTSAudioRawFrame is TTS-generated audio.

func NewTTSAudioRawFrame 

func NewTTSAudioRawFrame(audio []byte, sampleRate int) *TTSAudioRawFrame

NewTTSAudioRawFrame creates a TTS audio frame (mono 16-bit PCM).

func (*TTSAudioRawFrame) FrameType 

func (*TTSAudioRawFrame) FrameType() string

type TTSSpeakFrame 

type TTSSpeakFrame struct {
	DataFrame
	Text string `json:"text"`
}

TTSSpeakFrame asks TTS to speak the given text.

func NewTTSSpeakFrame 

func NewTTSSpeakFrame(text string) *TTSSpeakFrame

NewTTSSpeakFrame creates a TTSSpeakFrame.

func (*TTSSpeakFrame) FrameType 

func (*TTSSpeakFrame) FrameType() string

type TextFrame 

type TextFrame struct {
	DataFrame
	Text                    string `json:"text"`
	SkipTTS                 *bool  `json:"skip_tts,omitempty"`
	IncludesInterFrameSpace bool   `json:"includes_inter_frame_spaces"`
	AppendToContext         bool   `json:"append_to_context"`
}

TextFrame carries text (e.g. from LLM or for TTS).

func NewTextFrame 

func NewTextFrame(text string) *TextFrame

NewTextFrame creates a TextFrame with a new Base.

func (*TextFrame) FrameType 

func (*TextFrame) FrameType() string

type TranscriptionFrame 

type TranscriptionFrame struct {
	TextFrame
	UserID    string `json:"user_id"`
	Timestamp string `json:"timestamp"`
	Language  string `json:"language,omitempty"`
	Finalized bool   `json:"finalized"`
}

TranscriptionFrame is STT output (user speech).

func NewTranscriptionFrame 

func NewTranscriptionFrame(text, userID, timestamp string, finalized bool) *TranscriptionFrame

NewTranscriptionFrame creates a TranscriptionFrame.

func (*TranscriptionFrame) FrameType 

func (*TranscriptionFrame) FrameType() string

type TransportMessageFrame 

type TransportMessageFrame struct {
	DataFrame
	Message map[string]any `json:"message"`
}

TransportMessageFrame carries a generic transport message (e.g. for provider protocols). Used for binary MessageFrame wire format and JSON envelope round-trip.

func NewTransportMessageFrame 

func NewTransportMessageFrame(msg map[string]any) *TransportMessageFrame

NewTransportMessageFrame creates a TransportMessageFrame with the given message.

func (*TransportMessageFrame) FrameType 

func (*TransportMessageFrame) FrameType() string

type UserIdleFrame 

type UserIdleFrame struct {
	ControlFrame
}

UserIdleFrame signals that the user has been idle (not speaking) for a configured timeout period after the bot has finished speaking. It can be used to trigger follow-up prompts or other behaviors.

func NewUserIdleFrame 

func NewUserIdleFrame() *UserIdleFrame

NewUserIdleFrame creates a UserIdleFrame.

func (*UserIdleFrame) FrameType 

func (*UserIdleFrame) FrameType() string

type UserSpeakingFrame 

type UserSpeakingFrame struct {
	ControlFrame
}

UserSpeakingFrame is emitted periodically by VADProcessor while speech is detected.

func NewUserSpeakingFrame 

func NewUserSpeakingFrame() *UserSpeakingFrame

NewUserSpeakingFrame creates a UserSpeakingFrame.

func (*UserSpeakingFrame) FrameType 

func (*UserSpeakingFrame) FrameType() string

type UserStartedSpeakingFrame 

type UserStartedSpeakingFrame struct {
	ControlFrame
}

UserStartedSpeakingFrame signals that the user has started speaking. It is a lightweight control frame that can be used by downstream processors to react to user turn start events (e.g., cancel TTS, adjust UI state, etc.).

func NewUserStartedSpeakingFrame 

func NewUserStartedSpeakingFrame() *UserStartedSpeakingFrame

NewUserStartedSpeakingFrame creates a UserStartedSpeakingFrame.

func (*UserStartedSpeakingFrame) FrameType 

func (*UserStartedSpeakingFrame) FrameType() string

type UserStoppedSpeakingFrame 

type UserStoppedSpeakingFrame struct {
	ControlFrame
}

UserStoppedSpeakingFrame signals that the user has stopped speaking (i.e., the user turn has ended as determined by the turn analyzer and strategies).

func NewUserStoppedSpeakingFrame 

func NewUserStoppedSpeakingFrame() *UserStoppedSpeakingFrame

NewUserStoppedSpeakingFrame creates a UserStoppedSpeakingFrame.

func (*UserStoppedSpeakingFrame) FrameType 

func (*UserStoppedSpeakingFrame) FrameType() string

type VADParamsUpdateFrame 

type VADParamsUpdateFrame struct {
	ControlFrame
	// StopSecs is silence duration in seconds to end turn (e.g. 2.0 for IVR).
	StopSecs float64 `json:"stop_secs"`
	// StartSecs is VAD start trigger time in seconds (pre-speech padding).
	StartSecs float64 `json:"start_secs,omitempty"`
}

VADParamsUpdateFrame updates VAD/turn parameters (e.g. stop_secs for IVR mode). Processors such as TurnProcessor apply these to their Analyzer when received.

func NewVADParamsUpdateFrame 

func NewVADParamsUpdateFrame(stopSecs, startSecs float64) *VADParamsUpdateFrame

NewVADParamsUpdateFrame creates a VADParamsUpdateFrame.

func (*VADParamsUpdateFrame) FrameType 

func (*VADParamsUpdateFrame) FrameType() string

type VADUserStartedSpeakingFrame 

type VADUserStartedSpeakingFrame struct {
	ControlFrame
	StartSecs float64 `json:"start_secs,omitempty"`
}

VADUserStartedSpeakingFrame is emitted by VADProcessor when speech is detected. StartSecs may carry the analyzer's pre-speech padding (for context).

func NewVADUserStartedSpeakingFrame 

func NewVADUserStartedSpeakingFrame(startSecs float64) *VADUserStartedSpeakingFrame

NewVADUserStartedSpeakingFrame creates a VADUserStartedSpeakingFrame.

func (*VADUserStartedSpeakingFrame) FrameType 

func (*VADUserStartedSpeakingFrame) FrameType() string

type VADUserStoppedSpeakingFrame 

type VADUserStoppedSpeakingFrame struct {
	ControlFrame
	StopSecs float64 `json:"stop_secs,omitempty"`
}

VADUserStoppedSpeakingFrame is emitted by VADProcessor when speech ends. StopSecs may carry the analyzer's silence duration that triggered the stop.

func NewVADUserStoppedSpeakingFrame 

func NewVADUserStoppedSpeakingFrame(stopSecs float64) *VADUserStoppedSpeakingFrame

NewVADUserStoppedSpeakingFrame creates a VADUserStoppedSpeakingFrame.

func (*VADUserStoppedSpeakingFrame) FrameType 

func (*VADUserStoppedSpeakingFrame) FrameType() string

Source Files

View all Source files

Directories

Path Synopsis
proto
wire
Package wire provides frame wire formats.
 Package wire provides frame wire formats.
Package serialize provides frame serialization interfaces and implementations.
 Package serialize provides frame serialization interfaces and implementations.
exotel
Package exotel provides Exotel Media Streams WebSocket protocol serializer.
 Package exotel provides Exotel Media Streams WebSocket protocol serializer.
genesys
Package genesys provides Genesys AudioHook WebSocket protocol serializer.
 Package genesys provides Genesys AudioHook WebSocket protocol serializer.
plivo
Package plivo provides Plivo Audio Streaming WebSocket protocol serializer.
 Package plivo provides Plivo Audio Streaming WebSocket protocol serializer.
telnyx
Package telnyx provides Telnyx WebSocket protocol serializer.
 Package telnyx provides Telnyx WebSocket protocol serializer.
twilio
Package twilio provides Twilio Media Streams WebSocket protocol serializer.
 Package twilio provides Twilio Media Streams WebSocket protocol serializer.
vonage
Package vonage provides Vonage Audio Connector WebSocket serializer.
 Package vonage provides Vonage Audio Connector WebSocket serializer.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.