frames

package
v0.0.5 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Jul 13, 2026 License: BSD-2-Clause Imports: 8 Imported by: 0

Documentation

Overview

Package frames defines the Frame type and the core frame categories — system, data and control — that flow through a jargo pipeline.

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

This section is empty.

Types

type AudioBufferStartRecordingFrame added in v0.0.4 

type AudioBufferStartRecordingFrame struct {
	BaseControlFrame
	UninterruptibleMixin
}

AudioBufferStartRecordingFrame instructs audio-buffer processors to start recording. It is a control frame and is uninterruptible so a barge-in does not drop it.

func NewAudioBufferStartRecordingFrame added in v0.0.4 

func NewAudioBufferStartRecordingFrame() *AudioBufferStartRecordingFrame

NewAudioBufferStartRecordingFrame builds an AudioBufferStartRecordingFrame.

type AudioBufferStopRecordingFrame added in v0.0.4 

type AudioBufferStopRecordingFrame struct {
	BaseControlFrame
	UninterruptibleMixin
}

AudioBufferStopRecordingFrame instructs audio-buffer processors to stop recording and flush the buffered audio. It is a control frame and is uninterruptible.

func NewAudioBufferStopRecordingFrame added in v0.0.4 

func NewAudioBufferStopRecordingFrame() *AudioBufferStopRecordingFrame

NewAudioBufferStopRecordingFrame builds an AudioBufferStopRecordingFrame.

type AudioRawData 

type AudioRawData struct {
	// Audio is raw PCM audio: 16-bit signed samples, interleaved by channel.
	Audio []byte
	// SampleRate is the audio sample rate in Hz.
	SampleRate int
	// NumChannels is the number of interleaved audio channels.
	NumChannels int
	// contains filtered or unexported fields
}

AudioRawData is the raw-audio payload shared by the audio frame types. It carries a chunk of PCM audio plus the metadata needed to interpret it; the audio frames embed it alongside a category base.

func (*AudioRawData) NumFrames 

func (a *AudioRawData) NumFrames() int

NumFrames is the number of audio frames (samples per channel) in the buffer: len(Audio) / (NumChannels * 2) for 16-bit PCM. It is 0 when NumChannels is unset.

type BaseControlFrame 

type BaseControlFrame struct{ BaseFrame }

BaseControlFrame is embedded by control frames. Construct with NewBaseControlFrame.

func NewBaseControlFrame 

func NewBaseControlFrame(typeName string) BaseControlFrame

NewBaseControlFrame initializes a BaseControlFrame for the named concrete type.

type BaseDataFrame 

type BaseDataFrame struct{ BaseFrame }

BaseDataFrame is embedded by data frames. Construct with NewBaseDataFrame.

func NewBaseDataFrame 

func NewBaseDataFrame(typeName string) BaseDataFrame

NewBaseDataFrame initializes a BaseDataFrame for the named concrete type.

type BaseFrame 

type BaseFrame struct {
	// contains filtered or unexported fields
}

BaseFrame is embedded by every concrete frame and implements Frame. Construct it with NewBaseFrame so the id, name and metadata map are initialized.

func NewBaseFrame 

func NewBaseFrame(typeName string) BaseFrame

NewBaseFrame initializes a BaseFrame for a concrete frame whose type is named typeName (e.g. "TextFrame"). It assigns a unique id; the "<typeName>#<id>" name is formatted on demand.

func (*BaseFrame) BroadcastSiblingID 

func (f *BaseFrame) BroadcastSiblingID() (uint64, bool)

BroadcastSiblingID implements Frame.

func (*BaseFrame) ID 

func (f *BaseFrame) ID() uint64

ID implements Frame.

func (*BaseFrame) Metadata 

func (f *BaseFrame) Metadata() map[string]any

Metadata implements Frame. It lazily initializes the map so a zero BaseFrame is still usable.

func (*BaseFrame) Name 

func (f *BaseFrame) Name() string

Name implements Frame. The label "<typeName>#<id>" is formatted on demand.

func (*BaseFrame) PTS 

func (f *BaseFrame) PTS() (int64, bool)

PTS implements Frame.

func (*BaseFrame) SetBroadcastSiblingID 

func (f *BaseFrame) SetBroadcastSiblingID(id uint64)

SetBroadcastSiblingID implements Frame.

func (*BaseFrame) SetPTS 

func (f *BaseFrame) SetPTS(pts int64)

SetPTS implements Frame.

func (*BaseFrame) SetTransportDestination 

func (f *BaseFrame) SetTransportDestination(dest string)

SetTransportDestination implements Frame.

func (*BaseFrame) SetTransportSource 

func (f *BaseFrame) SetTransportSource(source string)

SetTransportSource implements Frame.

func (*BaseFrame) String 

func (f *BaseFrame) String() string

String implements fmt.Stringer and returns Name.

func (*BaseFrame) TransportDestination 

func (f *BaseFrame) TransportDestination() string

TransportDestination implements Frame.

func (*BaseFrame) TransportSource 

func (f *BaseFrame) TransportSource() string

TransportSource implements Frame.

type BaseSystemFrame 

type BaseSystemFrame struct{ BaseFrame }

BaseSystemFrame is embedded by system frames. Construct with NewBaseSystemFrame.

func NewBaseSystemFrame 

func NewBaseSystemFrame(typeName string) BaseSystemFrame

NewBaseSystemFrame initializes a BaseSystemFrame for the named concrete type.

type BotSpeakingFrame 

type BotSpeakingFrame struct {
	BaseSystemFrame
}

BotSpeakingFrame is emitted periodically while the bot is speaking. It is a system frame.

func NewBotSpeakingFrame 

func NewBotSpeakingFrame() *BotSpeakingFrame

NewBotSpeakingFrame builds a BotSpeakingFrame.

type BotStartedSpeakingFrame 

type BotStartedSpeakingFrame struct {
	BaseSystemFrame
}

BotStartedSpeakingFrame indicates the bot started speaking. It is a system frame.

func NewBotStartedSpeakingFrame 

func NewBotStartedSpeakingFrame() *BotStartedSpeakingFrame

NewBotStartedSpeakingFrame builds a BotStartedSpeakingFrame.

type BotStoppedSpeakingFrame 

type BotStoppedSpeakingFrame struct {
	BaseSystemFrame
}

BotStoppedSpeakingFrame indicates the bot stopped speaking. It is a system frame.

func NewBotStoppedSpeakingFrame 

func NewBotStoppedSpeakingFrame() *BotStoppedSpeakingFrame

NewBotStoppedSpeakingFrame builds a BotStoppedSpeakingFrame.

type CancelFrame 

type CancelFrame struct {
	BaseSystemFrame
	// Reason is an optional reason for the cancellation.
	Reason any
}

CancelFrame indicates the pipeline must stop immediately, without processing any remaining queued frames. It is a system frame.

func NewCancelFrame 

func NewCancelFrame() *CancelFrame

NewCancelFrame builds a CancelFrame.

func (*CancelFrame) String 

func (f *CancelFrame) String() string

String implements fmt.Stringer.

type ControlFrame 

type ControlFrame interface {
	Frame
	// contains filtered or unexported methods
}

ControlFrame is processed in order like a DataFrame and is canceled by user interruptions; it carries control information such as settings updates or a request to end the pipeline once everything is flushed. Embed BaseControlFrame to define one.

type DataFrame 

type DataFrame interface {
	Frame
	// contains filtered or unexported methods
}

DataFrame is processed in order and is canceled by user interruptions. It usually carries data such as LLM context, text, audio or images. Embed BaseDataFrame to define one.

type EndFrame 

type EndFrame struct {
	BaseControlFrame
	UninterruptibleMixin
	// Reason is an optional reason for ending the pipeline.
	Reason any
}

EndFrame indicates the pipeline has ended and processors should shut down. As a control frame it is received in order, after preceding frames are flushed. It is uninterruptible so it survives an interruption and the pipeline always shuts down cleanly.

func NewEndFrame 

func NewEndFrame() *EndFrame

NewEndFrame builds an EndFrame.

func (*EndFrame) String 

func (f *EndFrame) String() string

String implements fmt.Stringer.

type ErrorFrame 

type ErrorFrame struct {
	BaseSystemFrame
	// Error describes the error that occurred.
	Error string
	// Fatal reports whether the error is unrecoverable and requires shutdown.
	Fatal bool
	// Source is the processor that raised the error, if known.
	Source ErrorSource
	// Err is the underlying error, if any.
	Err error
}

ErrorFrame notifies upstream that an error occurred downstream. A fatal error is unrecoverable and the bot should exit. It is a system frame.

func NewErrorFrame 

func NewErrorFrame(message string) *ErrorFrame

NewErrorFrame builds a non-fatal ErrorFrame describing message.

func (*ErrorFrame) String 

func (f *ErrorFrame) String() string

String implements fmt.Stringer.

type ErrorSource 

type ErrorSource interface {
	Name() string
}

ErrorSource identifies the component that raised an error — in practice the frame processor that produced it. It is declared here, rather than imported from the processor package, so the frames package keeps no dependency on it; a frame processor satisfies this interface by exposing its name.

type Frame 

type Frame interface {
	fmt.Stringer

	// ID is a process-unique identifier for this frame instance.
	ID() uint64
	// Name is a human-readable label, "<TypeName>#<n>".
	Name() string

	// PTS is the presentation timestamp in nanoseconds; ok is false if unset.
	PTS() (pts int64, ok bool)
	// SetPTS sets the presentation timestamp in nanoseconds.
	SetPTS(pts int64)

	// BroadcastSiblingID is the id of the paired frame when this frame was
	// broadcast in both directions; ok is false if unset.
	BroadcastSiblingID() (id uint64, ok bool)
	// SetBroadcastSiblingID records the paired frame id.
	SetBroadcastSiblingID(id uint64)

	// Metadata is an arbitrary, mutable per-frame metadata map.
	Metadata() map[string]any

	// TransportSource is the name of the transport that created this frame.
	TransportSource() string
	// SetTransportSource sets the transport source.
	SetTransportSource(source string)
	// TransportDestination is the name of the transport this frame targets.
	TransportDestination() string
	// SetTransportDestination sets the transport destination.
	SetTransportDestination(dest string)
	// contains filtered or unexported methods
}

Frame is implemented by every frame that flows through a pipeline. Concrete frames embed BaseFrame (directly or via BaseSystemFrame, BaseDataFrame or BaseControlFrame), which supplies all of these methods.

The unexported isFrame marker means a type must embed BaseFrame to satisfy Frame; this guarantees every Frame has a valid id, name and metadata map. Frames carry mutable state and are passed as pointers.

type FunctionCallCancelFrame 

type FunctionCallCancelFrame struct {
	BaseControlFrame
	// ToolCallID identifies the canceled call.
	ToolCallID string
	// ToolName is the tool's name.
	ToolName string
}

FunctionCallCancelFrame reports that a tool call was canceled (for example by a barge-in) so trackers can decrement their in-flight count. It is a control frame.

func NewFunctionCallCancelFrame 

func NewFunctionCallCancelFrame(toolCallID, name string) *FunctionCallCancelFrame

NewFunctionCallCancelFrame builds a FunctionCallCancelFrame.

func (*FunctionCallCancelFrame) String 

func (f *FunctionCallCancelFrame) String() string

String implements fmt.Stringer.

type FunctionCallInProgressFrame 

type FunctionCallInProgressFrame struct {
	BaseControlFrame
	// ToolCallID is the id of the call that is executing.
	ToolCallID string
	// ToolName is the tool's name.
	ToolName string
}

FunctionCallInProgressFrame reports that a specific tool call has started executing. It is useful for observability and for clients that surface an "in progress" indication. It is a control frame.

func NewFunctionCallInProgressFrame 

func NewFunctionCallInProgressFrame(toolCallID, name string) *FunctionCallInProgressFrame

NewFunctionCallInProgressFrame builds a FunctionCallInProgressFrame.

func (*FunctionCallInProgressFrame) String 

func (f *FunctionCallInProgressFrame) String() string

String implements fmt.Stringer.

type FunctionCallResultFrame 

type FunctionCallResultFrame struct {
	BaseControlFrame
	// ToolCallID pairs the result to its call.
	ToolCallID string
	// ToolName is the tool's name.
	ToolName string
	// Result is the tool-result content.
	Result string
	// IsError reports whether the tool failed.
	IsError bool
}

FunctionCallResultFrame carries the result of one tool call. The assistant context aggregator collects results into a single tool-result message. It is a control frame.

func NewFunctionCallResultFrame 

func NewFunctionCallResultFrame(toolCallID, name, result string, isError bool) *FunctionCallResultFrame

NewFunctionCallResultFrame builds a FunctionCallResultFrame.

func (*FunctionCallResultFrame) String 

func (f *FunctionCallResultFrame) String() string

String implements fmt.Stringer.

type FunctionCallsStartedFrame 

type FunctionCallsStartedFrame struct {
	BaseControlFrame
	// PreambleText is text the model produced alongside the calls, if any.
	PreambleText string
	// Calls are the tool calls the model requested this turn.
	Calls []ToolCall
}

FunctionCallsStartedFrame announces that the model requested one or more tool calls in the current assistant turn. PreambleText is any text the model spoke before the calls. The assistant context aggregator writes the assistant turn (preamble text plus the tool-use blocks) from this frame. It is a control frame so it stays ordered with the response's text frames.

func NewFunctionCallsStartedFrame 

func NewFunctionCallsStartedFrame(preamble string, calls []ToolCall) *FunctionCallsStartedFrame

NewFunctionCallsStartedFrame builds a FunctionCallsStartedFrame.

func (*FunctionCallsStartedFrame) String 

func (f *FunctionCallsStartedFrame) String() string

String implements fmt.Stringer.

type InputAudioRawFrame 

type InputAudioRawFrame struct {
	BaseSystemFrame
	AudioRawData
}

InputAudioRawFrame is a chunk of audio coming from an input transport. When a transport exposes multiple audio sources, the source name is carried in TransportSource. It is a system frame.

func NewInputAudioRawFrame 

func NewInputAudioRawFrame(audio []byte, sampleRate, numChannels int) *InputAudioRawFrame

NewInputAudioRawFrame builds an InputAudioRawFrame from PCM audio.

func (*InputAudioRawFrame) String 

func (f *InputAudioRawFrame) String() string

String implements fmt.Stringer.

type InputDTMFFrame added in v0.0.4 

type InputDTMFFrame struct {
	BaseSystemFrame
	// Button is the key that was pressed.
	Button KeypadEntry
}

InputDTMFFrame is a DTMF keypress received from the transport — for example a phone caller pressing a key. It is a system frame so it is delivered with priority and in order.

func NewInputDTMFFrame added in v0.0.4 

func NewInputDTMFFrame(button KeypadEntry) *InputDTMFFrame

NewInputDTMFFrame builds an InputDTMFFrame.

func (*InputDTMFFrame) String added in v0.0.4 

func (f *InputDTMFFrame) String() string

String implements fmt.Stringer.

type InputTransportMessageFrame 

type InputTransportMessageFrame struct {
	BaseSystemFrame
	// Message is the raw message payload as received (typically JSON).
	Message []byte
}

InputTransportMessageFrame carries an application message received by a transport from the client — for example an RTVI message off a WebRTC data channel. It is a system frame so it is handled with priority and in order.

func NewInputTransportMessageFrame 

func NewInputTransportMessageFrame(message []byte) *InputTransportMessageFrame

NewInputTransportMessageFrame builds an InputTransportMessageFrame.

func (*InputTransportMessageFrame) String 

func (f *InputTransportMessageFrame) String() string

String implements fmt.Stringer.

type InterimTranscriptionFrame 

type InterimTranscriptionFrame struct {
	TextFrame
	// UserID identifies the user who spoke.
	UserID string
	// Timestamp is when the interim transcription occurred.
	Timestamp string
	// Language is the detected or specified language as a BCP-47 tag; "" when
	// unset.
	Language string
	// Result is the raw result from the STT service, if available.
	Result any
}

InterimTranscriptionFrame carries a partial (non-final) speech transcription for a user.

func NewInterimTranscriptionFrame 

func NewInterimTranscriptionFrame(text, userID, timestamp string) *InterimTranscriptionFrame

NewInterimTranscriptionFrame builds an InterimTranscriptionFrame.

func (*InterimTranscriptionFrame) String 

func (f *InterimTranscriptionFrame) String() string

String implements fmt.Stringer.

type InterruptionFrame 

type InterruptionFrame struct {
	BaseSystemFrame
}

InterruptionFrame interrupts the pipeline — for example when the user starts speaking, to cancel in-progress bot output. It can be pushed by any processor. It is a system frame.

func NewInterruptionFrame 

func NewInterruptionFrame() *InterruptionFrame

NewInterruptionFrame builds an InterruptionFrame.

type KeypadEntry added in v0.0.4 

type KeypadEntry string

KeypadEntry is a single DTMF keypad key: the digits 0-9, the symbols * and #, and the letters A-D. 

const (
	KeypadZero  KeypadEntry = "0"
	KeypadOne   KeypadEntry = "1"
	KeypadTwo   KeypadEntry = "2"
	KeypadThree KeypadEntry = "3"
	KeypadFour  KeypadEntry = "4"
	KeypadFive  KeypadEntry = "5"
	KeypadSix   KeypadEntry = "6"
	KeypadSeven KeypadEntry = "7"
	KeypadEight KeypadEntry = "8"
	KeypadNine  KeypadEntry = "9"
	KeypadStar  KeypadEntry = "*"
	KeypadPound KeypadEntry = "#"
	KeypadA     KeypadEntry = "A"
	KeypadB     KeypadEntry = "B"
	KeypadC     KeypadEntry = "C"
	KeypadD     KeypadEntry = "D"
)

The DTMF keypad entries.

type LLMContext 

type LLMContext struct {
	// contains filtered or unexported fields
}

LLMContext holds the conversation so far: a system prompt plus the running list of user and assistant messages. The user and assistant aggregators append to a shared context as the conversation proceeds, and the LLM service reads it to generate each response. It is safe for concurrent use.

func NewLLMContext 

func NewLLMContext(system string) *LLMContext

NewLLMContext builds a context with the given system prompt.

func (*LLMContext) AddAssistantMessage 

func (c *LLMContext) AddAssistantMessage(text string)

AddAssistantMessage appends an assistant message.

func (*LLMContext) AddAssistantToolCalls 

func (c *LLMContext) AddAssistantToolCalls(text string, calls []ToolCall)

AddAssistantToolCalls appends an assistant message carrying optional preamble text and the tool calls the model requested in the same turn.

func (*LLMContext) AddToolResults 

func (c *LLMContext) AddToolResults(results []ToolResult)

AddToolResults appends a user message returning the outputs of one or more tool calls. The results of all calls in an assistant turn belong in a single message.

func (*LLMContext) AddUserMessage 

func (c *LLMContext) AddUserMessage(text string)

AddUserMessage appends a user message.

func (*LLMContext) Compact 

func (c *LLMContext) Compact(
	ctx context.Context,
	keepRecent int,
	summarize func(ctx context.Context, prior string, dropped []Message) (string, error),
) (bool, error)

Compact shrinks a long conversation: it drops the oldest messages beyond the keepRecent most recent — cutting on a clean user-turn boundary so the preserved tail stays a valid message list — and folds them into the rolling summary, which System then appends to the prompt. summarize turns the prior summary and the dropped messages into the new summary; it is invoked WITHOUT the context lock held, so it may call out to an LLM. Compact reports whether it compacted anything.

Compact only ever removes a prefix, and only the summary (not the messages) carries the dropped history forward, so messages appended at the tail while summarize runs are preserved. It must not be run concurrently with itself on the same context.

func (*LLMContext) EstimatedTokens 

func (c *LLMContext) EstimatedTokens() int

EstimatedTokens is a rough estimate of the context's size in tokens, used to decide when to compact. It approximates four characters per token across the system prompt, the rolling summary, and every message.

func (*LLMContext) Messages 

func (c *LLMContext) Messages() []Message

Messages returns a copy of the conversation messages.

func (*LLMContext) Recall 

func (c *LLMContext) Recall() string

Recall returns the transient retrieved context folded into the system prompt, or "" if none is set.

func (*LLMContext) SetRecall 

func (c *LLMContext) SetRecall(recall string)

SetRecall sets transient retrieved context — typically long-term memories surfaced by a memory service — that is folded into the system prompt for subsequent generations, replacing any previous value. The text is used verbatim, so include any framing (e.g. a "recalled memories" header) in it. A memory processor refreshes it each turn; pass "" to clear it.

func (*LLMContext) SetSystem 

func (c *LLMContext) SetSystem(system string)

SetSystem replaces the system prompt. Used to switch the assistant's behavior mid-session (the next generation picks up the new prompt).

func (*LLMContext) SetTools 

func (c *LLMContext) SetTools(tools []Tool)

SetTools replaces the set of tools the model may call. Used alongside SetSystem to switch the available toolset mid-session.

func (*LLMContext) Summary 

func (c *LLMContext) Summary() string

Summary returns the rolling summary of compacted older turns, or "" if the conversation has not been compacted yet.

func (*LLMContext) System 

func (c *LLMContext) System() string

System returns the system prompt the LLM should run with. Once older turns have been compacted (see Compact), the rolling summary is appended so the model retains that history even though the messages themselves are gone.

func (*LLMContext) Tools 

func (c *LLMContext) Tools() []Tool

Tools returns a copy of the tools the model may call.

type LLMContextFrame 

type LLMContextFrame struct {
	BaseDataFrame
	// Context is the conversation to generate a response from.
	Context *LLMContext
}

LLMContextFrame carries the conversation context to the LLM service to trigger a response. It is a data frame.

func NewLLMContextFrame 

func NewLLMContextFrame(ctx *LLMContext) *LLMContextFrame

NewLLMContextFrame builds an LLMContextFrame.

func (*LLMContextFrame) String 

func (f *LLMContextFrame) String() string

String implements fmt.Stringer.

type LLMFullResponseEndFrame 

type LLMFullResponseEndFrame struct {
	BaseControlFrame
	// SkipTTS, when set, reports whether the response should skip TTS. A nil
	// value means "unset".
	SkipTTS *bool
}

LLMFullResponseEndFrame marks the end of an LLM response. It is a control frame.

func NewLLMFullResponseEndFrame 

func NewLLMFullResponseEndFrame() *LLMFullResponseEndFrame

NewLLMFullResponseEndFrame builds an LLMFullResponseEndFrame.

type LLMFullResponseStartFrame 

type LLMFullResponseStartFrame struct {
	BaseControlFrame
	// SkipTTS, when set, reports whether the response should skip TTS. A nil
	// value means "unset".
	SkipTTS *bool
}

LLMFullResponseStartFrame marks the beginning of an LLM response, followed by one or more TextFrames and a final LLMFullResponseEndFrame. It is a control frame.

func NewLLMFullResponseStartFrame 

func NewLLMFullResponseStartFrame() *LLMFullResponseStartFrame

NewLLMFullResponseStartFrame builds an LLMFullResponseStartFrame.

type LLMMarkerFrame 

type LLMMarkerFrame struct {
	BaseDataFrame
	// Marker is the marker text.
	Marker string
}

LLMMarkerFrame carries a turn-completion marker the LLM emitted (e.g. "✓"). It is informational — downstream TTS ignores it — and lets observers see the model's completeness verdict. It is a data frame.

func NewLLMMarkerFrame 

func NewLLMMarkerFrame(marker string) *LLMMarkerFrame

NewLLMMarkerFrame builds an LLMMarkerFrame.

type LLMMessagesAppendFrame 

type LLMMessagesAppendFrame struct {
	BaseControlFrame
	// Messages are the messages to append.
	Messages []Message
}

LLMMessagesAppendFrame asks the context aggregator to append messages to the LLM context (used by the turn-completion re-prompt). It is a control frame.

func NewLLMMessagesAppendFrame 

func NewLLMMessagesAppendFrame(messages []Message) *LLMMessagesAppendFrame

NewLLMMessagesAppendFrame builds an LLMMessagesAppendFrame.

type LLMRunFrame 

type LLMRunFrame struct {
	BaseDataFrame
}

LLMRunFrame instructs the LLM service to process the current context and generate a response. Queue it to make the bot speak first at the start of a session, or to re-run after editing the context. It carries no data — the user aggregator runs its current shared context. It is a data frame.

func NewLLMRunFrame 

func NewLLMRunFrame() *LLMRunFrame

NewLLMRunFrame builds an LLMRunFrame.

type LLMServiceMetadataFrame added in v0.0.4 

type LLMServiceMetadataFrame struct {
	ServiceMetadataFrame
	// Realtime reports whether the broadcasting LLM is a realtime
	// (speech-to-speech) service.
	Realtime bool
}

LLMServiceMetadataFrame is the metadata an LLM service broadcasts. It reports whether the service is a realtime (speech-to-speech) LLM.

func NewLLMServiceMetadataFrame added in v0.0.4 

func NewLLMServiceMetadataFrame(service string) *LLMServiceMetadataFrame

NewLLMServiceMetadataFrame builds an LLMServiceMetadataFrame for the named service.

type LLMTextFrame 

type LLMTextFrame struct {
	TextFrame
}

LLMTextFrame is a TextFrame produced by an LLM service. LLM output already includes any necessary inter-frame spaces.

func NewLLMTextFrame 

func NewLLMTextFrame(text string) *LLMTextFrame

NewLLMTextFrame builds an LLMTextFrame.

type LLMTokenUsage 

type LLMTokenUsage struct {
	// PromptTokens is the number of input tokens.
	PromptTokens int64
	// CompletionTokens is the number of output tokens.
	CompletionTokens int64
	// CacheReadTokens is the number of input tokens read from the prompt cache.
	CacheReadTokens int64
	// CacheCreationTokens is the number of input tokens written to the prompt cache.
	CacheCreationTokens int64
	// TotalTokens is the sum of the prompt and completion tokens.
	TotalTokens int64
}

LLMTokenUsage reports the token counts billed for one LLM generation. The cache counts are a subset of the input tokens: CacheReadTokens were served from a prompt cache and CacheCreationTokens were written to one.

type Message 

type Message struct {
	Role Role
	Text string
	// ToolCalls is set on an assistant message that requested tool calls.
	ToolCalls []ToolCall
	// ToolResults is set on a message returning the outputs of tool calls.
	ToolResults []ToolResult
}

Message is a single conversation turn. A plain turn carries Text; an assistant turn that invoked tools also carries ToolCalls; a turn returning tool outputs carries ToolResults.

type MetricsFrame 

type MetricsFrame struct {
	BaseSystemFrame
	// Processor is the name of the processor that produced the metrics.
	Processor string
	// Model is the model that produced the metrics, when known.
	Model string
	// TTFB is the time to first byte (first token or audio), or nil when not measured.
	TTFB *time.Duration
	// TTFA is the time to the first audible sample (TTFB plus any leading
	// silence a TTS service padded onto its response), or nil when not measured.
	TTFA *time.Duration
	// LeadingSilence is the silence before the first audible sample (TTFA minus
	// TTFB), or nil when not measured.
	LeadingSilence *time.Duration
	// Processing is the wall-clock time the operation took, or nil when not measured.
	Processing *time.Duration
	// Tokens reports LLM token usage, or nil when not applicable.
	Tokens *LLMTokenUsage
	// Characters reports the number of characters synthesized by TTS, or nil.
	Characters *int
}

MetricsFrame reports metrics measured by a processor. It is a system frame, so it is delivered with priority and is not dropped by an interruption — usage is billed even when a turn is cut short. Each field is optional: a processor sets the kinds it measured (e.g. an LLM reports TTFB, processing time and tokens).

func NewMetricsFrame 

func NewMetricsFrame(processor string) *MetricsFrame

NewMetricsFrame builds a MetricsFrame attributed to the named processor.

func (*MetricsFrame) String 

func (f *MetricsFrame) String() string

String implements fmt.Stringer.

type MixerControlFrame added in v0.0.4 

type MixerControlFrame struct {
	BaseControlFrame
	// Settings are the mixer settings to apply (e.g. "volume", "enabled").
	Settings map[string]any
}

MixerControlFrame updates an output transport's audio mixer at runtime — for example to change the background track, adjust its volume, or mute it. The Settings are interpreted by the mixer implementation. It is a control frame.

func NewMixerControlFrame added in v0.0.4 

func NewMixerControlFrame(settings map[string]any) *MixerControlFrame

NewMixerControlFrame builds a MixerControlFrame carrying settings.

type OutputAudioRawFrame 

type OutputAudioRawFrame struct {
	BaseDataFrame
	AudioRawData
}

OutputAudioRawFrame is a chunk of audio to be played by an output transport. When a transport exposes multiple audio destinations, the destination name is carried in TransportDestination. It is a data frame.

func NewOutputAudioRawFrame 

func NewOutputAudioRawFrame(audio []byte, sampleRate, numChannels int) *OutputAudioRawFrame

NewOutputAudioRawFrame builds an OutputAudioRawFrame from PCM audio.

func (*OutputAudioRawFrame) String 

func (f *OutputAudioRawFrame) String() string

String implements fmt.Stringer.

type OutputDTMFFrame added in v0.0.4 

type OutputDTMFFrame struct {
	BaseControlFrame
	// Button is the key to emit.
	Button KeypadEntry
}

OutputDTMFFrame requests the transport play a DTMF tone for Button — for example to navigate an IVR menu. It is a control frame, delivered in order after preceding audio.

func NewOutputDTMFFrame added in v0.0.4 

func NewOutputDTMFFrame(button KeypadEntry) *OutputDTMFFrame

NewOutputDTMFFrame builds an OutputDTMFFrame.

func (*OutputDTMFFrame) String added in v0.0.4 

func (f *OutputDTMFFrame) String() string

String implements fmt.Stringer.

type OutputTransportMessageFrame 

type OutputTransportMessageFrame struct {
	BaseSystemFrame
	// Message is the message payload to send; the transport serializes it.
	Message any
}

OutputTransportMessageFrame carries an application message to send to the client over the transport — for example an RTVI message onto a WebRTC data channel. Message is serialized by the output transport. It is a system frame.

func NewOutputTransportMessageFrame 

func NewOutputTransportMessageFrame(message any) *OutputTransportMessageFrame

NewOutputTransportMessageFrame builds an OutputTransportMessageFrame.

type Role 

type Role string

Role identifies who authored a conversation message. 

const (
	// RoleSystem is the system prompt that frames the assistant's behavior.
	RoleSystem Role = "system"
	// RoleUser is a message from the user.
	RoleUser Role = "user"
	// RoleAssistant is a message from the assistant.
	RoleAssistant Role = "assistant"
)

type STTMetadataFrame 

type STTMetadataFrame struct {
	BaseSystemFrame
	// ServiceName names the STT service that broadcast the metadata.
	ServiceName string
	// UserTurns is the turn strategy the service recommends; the user aggregator
	// adopts it unless the application configured its own.
	UserTurns UserTurnRecommendation
	// TTFSP99Latency is the p99 latency from end of speech to a finalized
	// transcript. Zero means unknown; strategies fall back to a default.
	TTFSP99Latency time.Duration
}

STTMetadataFrame carries metadata an STT service broadcasts at pipeline start. Turn-stop strategies use the p99 time-to-final-speech latency to size their safety-net timeouts, and a service that does its own server-side endpointing recommends external turn strategies through UserTurns. It is a system frame and satisfies ServiceMetadata.

func NewSTTMetadataFrame 

func NewSTTMetadataFrame(ttfsP99 time.Duration) *STTMetadataFrame

NewSTTMetadataFrame builds an STTMetadataFrame reporting the p99 time-to-final-speech latency. Set ServiceName and UserTurns on the returned frame to describe the service further.

func (*STTMetadataFrame) RecommendedUserTurns added in v0.0.4 

func (f *STTMetadataFrame) RecommendedUserTurns() UserTurnRecommendation

RecommendedUserTurns implements ServiceMetadata.

func (*STTMetadataFrame) Service added in v0.0.4 

func (f *STTMetadataFrame) Service() string

Service implements ServiceMetadata.

func (*STTMetadataFrame) String 

func (f *STTMetadataFrame) String() string

String implements fmt.Stringer.

type ServiceMetadata added in v0.0.4 

type ServiceMetadata interface {
	SystemFrame
	// Service is the name of the service that broadcast the metadata.
	Service() string
	// RecommendedUserTurns is the turn strategy the service recommends, or
	// UserTurnUnspecified.
	RecommendedUserTurns() UserTurnRecommendation
}

ServiceMetadata is implemented by every metadata frame a service broadcasts at pipeline start. Downstream processors assert this interface to read the common fields; a concrete STTMetadataFrame or LLMServiceMetadataFrame carries more.

type ServiceMetadataFrame added in v0.0.4 

type ServiceMetadataFrame struct {
	BaseSystemFrame
	// ServiceName names the broadcasting service.
	ServiceName string
	// UserTurns is the turn strategy the service recommends; the user aggregator
	// adopts it unless the application configured its own.
	UserTurns UserTurnRecommendation
}

ServiceMetadataFrame is broadcast by a service at pipeline start to share its configuration and performance characteristics with downstream processors. It is a system frame. LLMServiceMetadataFrame embeds it; STTMetadataFrame (see turns.go) carries the same common fields to satisfy ServiceMetadata.

func NewServiceMetadataFrame added in v0.0.4 

func NewServiceMetadataFrame(service string) *ServiceMetadataFrame

NewServiceMetadataFrame builds a ServiceMetadataFrame for the named service.

func (*ServiceMetadataFrame) RecommendedUserTurns added in v0.0.4 

func (f *ServiceMetadataFrame) RecommendedUserTurns() UserTurnRecommendation

RecommendedUserTurns implements ServiceMetadata.

func (*ServiceMetadataFrame) Service added in v0.0.4 

func (f *ServiceMetadataFrame) Service() string

Service implements ServiceMetadata.

func (*ServiceMetadataFrame) String added in v0.0.4 

func (f *ServiceMetadataFrame) String() string

String implements fmt.Stringer.

type SpeechControlParamsFrame 

type SpeechControlParamsFrame struct {
	BaseSystemFrame
	// StopSecs, PreSpeechMs and MaxDurationSecs mirror the turn analyzer's
	// timing parameters.
	StopSecs        float64
	PreSpeechMs     float64
	MaxDurationSecs float64
}

SpeechControlParamsFrame broadcasts the active end-of-turn timing parameters so clients and observers can mirror them. It is a system frame.

func NewSpeechControlParamsFrame 

func NewSpeechControlParamsFrame(stopSecs, preSpeechMs, maxDurationSecs float64) *SpeechControlParamsFrame

NewSpeechControlParamsFrame builds a SpeechControlParamsFrame.

type StartFrame 

type StartFrame struct {
	BaseSystemFrame
	// AudioInSampleRate is the input audio sample rate in Hz.
	AudioInSampleRate int
	// AudioOutSampleRate is the output audio sample rate in Hz.
	AudioOutSampleRate int
	// EnableMetrics enables performance metrics collection.
	EnableMetrics bool
	// EnableUsageMetrics enables usage metrics collection.
	EnableUsageMetrics bool
	// ReportOnlyInitialTTFB reports only the initial time-to-first-byte.
	ReportOnlyInitialTTFB bool
}

StartFrame is the first frame pushed down a pipeline. It initializes every processor with the pipeline-wide configuration. It is a system frame.

func NewStartFrame 

func NewStartFrame() *StartFrame

NewStartFrame builds a StartFrame with the default sample rates (16 kHz in, 24 kHz out). Override any field before pushing it.

type SystemFrame 

type SystemFrame interface {
	Frame
	// contains filtered or unexported methods
}

SystemFrame takes priority over other frames and is not affected by user interruptions; system frames are handled in order. Assert a Frame to SystemFrame to test its category. Embed BaseSystemFrame to define one.

type TTSAudioRawFrame 

type TTSAudioRawFrame struct {
	OutputAudioRawFrame
	// ContextID identifies the TTS context that generated this audio; "" when
	// unset.
	ContextID string
}

TTSAudioRawFrame is a chunk of output audio generated by a TTS service, ready for playback.

func NewTTSAudioRawFrame 

func NewTTSAudioRawFrame(audio []byte, sampleRate, numChannels int) *TTSAudioRawFrame

NewTTSAudioRawFrame builds a TTSAudioRawFrame from PCM audio.

type TTSSpeakFrame 

type TTSSpeakFrame struct {
	BaseDataFrame
	// Text is the exact text to speak.
	Text string
	// AppendToContext reports whether the spoken text is appended to the LLM
	// context as an assistant message. Defaults to true; set it false for
	// utterances that should not become part of the conversation (e.g. a wake
	// acknowledgement, which would otherwise start the context on an assistant
	// turn).
	AppendToContext bool
}

TTSSpeakFrame carries fixed text for the TTS service to speak directly, bypassing the LLM and the TTS sentence aggregator — the way to make the bot say a set phrase (a greeting, an acknowledgement). It is a data frame.

func NewTTSSpeakFrame 

func NewTTSSpeakFrame(text string) *TTSSpeakFrame

NewTTSSpeakFrame builds a TTSSpeakFrame that speaks text, appending it to the LLM context by default.

func (*TTSSpeakFrame) String 

func (f *TTSSpeakFrame) String() string

String implements fmt.Stringer.

type TTSStartedFrame 

type TTSStartedFrame struct {
	BaseControlFrame
	// ContextID identifies this TTS context; "" when unset.
	ContextID string
	// AppendToContext reports whether the spoken text for this response will be
	// appended to the LLM context. Defaults to true.
	AppendToContext bool
}

TTSStartedFrame marks the beginning of a TTS response. The following TTSAudioRawFrames are part of the response until a TTSStoppedFrame. It is a control frame.

func NewTTSStartedFrame 

func NewTTSStartedFrame() *TTSStartedFrame

NewTTSStartedFrame builds a TTSStartedFrame.

type TTSStoppedFrame 

type TTSStoppedFrame struct {
	BaseControlFrame
	// ContextID identifies this TTS context; "" when unset.
	ContextID string
}

TTSStoppedFrame marks the end of a TTS response. It is a control frame.

func NewTTSStoppedFrame 

func NewTTSStoppedFrame() *TTSStoppedFrame

NewTTSStoppedFrame builds a TTSStoppedFrame.

type TextFrame 

type TextFrame struct {
	BaseDataFrame
	// Text is the text content.
	Text string
	// SkipTTS reports whether a TTS service should skip this text. A nil value
	// means "unset": the decision is left to the frame flow.
	SkipTTS *bool
	// IncludesInterFrameSpaces reports whether any leading/trailing spaces
	// needed between adjacent frames are already part of Text.
	IncludesInterFrameSpaces bool
	// AppendToContext reports whether this text should be appended to the LLM
	// context. Defaults to true.
	AppendToContext bool
}

TextFrame is a chunk of text flowing through the pipeline — emitted by LLM services and consumed by aggregators, TTS services and more. It is a data frame.

func NewTextFrame 

func NewTextFrame(text string) *TextFrame

NewTextFrame builds a TextFrame with the default field values.

func (*TextFrame) String 

func (f *TextFrame) String() string

String implements fmt.Stringer.

type Tool 

type Tool struct {
	Name        string
	Description string
	Parameters  json.RawMessage
}

Tool is a function the model may call. Parameters is a JSON-Schema object (`{"type":"object","properties":{…},"required":[…]}`) describing the arguments the tool accepts.

type ToolCall 

type ToolCall struct {
	ID   string
	Name string
	Args json.RawMessage
}

ToolCall is a request from the model to invoke a tool. Args is the raw JSON arguments the model produced.

type ToolResult 

type ToolResult struct {
	ID      string
	Name    string
	Content string
	IsError bool
}

ToolResult is the outcome of a tool invocation, paired to a ToolCall by ID.

type TranscriptionFrame 

type TranscriptionFrame struct {
	TextFrame
	// UserID identifies the user who spoke.
	UserID string
	// Timestamp is when the transcription occurred.
	Timestamp string
	// Language is the detected or specified language as a BCP-47 tag; "" when
	// unset.
	Language string
	// Result is the raw result from the STT service, if available.
	Result any
	// Finalized reports whether this is the final transcription for an
	// utterance, for STT services that signal commit/finalize.
	Finalized bool
}

TranscriptionFrame carries a finalized speech transcription for a user.

func NewTranscriptionFrame 

func NewTranscriptionFrame(text, userID, timestamp string) *TranscriptionFrame

NewTranscriptionFrame builds a TranscriptionFrame.

func (*TranscriptionFrame) String 

func (f *TranscriptionFrame) String() string

String implements fmt.Stringer.

type Uninterruptible 

type Uninterruptible interface {
	// contains filtered or unexported methods
}

Uninterruptible marks a data or control frame that must survive interruptions: it stays queued and any task processing it is never canceled, guaranteeing delivery and completion. Embed UninterruptibleMixin (alongside a category base) and assert with this interface.

type UninterruptibleMixin 

type UninterruptibleMixin struct{}

UninterruptibleMixin is embedded to mark a frame Uninterruptible.

type UserIdleTimeoutUpdateFrame 

type UserIdleTimeoutUpdateFrame struct {
	BaseSystemFrame
	// Timeout is the new idle timeout.
	Timeout time.Duration
}

UserIdleTimeoutUpdateFrame updates the user-idle timeout at runtime. A value <= 0 disables idle detection. It is a system frame.

func NewUserIdleTimeoutUpdateFrame 

func NewUserIdleTimeoutUpdateFrame(timeout time.Duration) *UserIdleTimeoutUpdateFrame

NewUserIdleTimeoutUpdateFrame builds a UserIdleTimeoutUpdateFrame.

func (*UserIdleTimeoutUpdateFrame) String 

func (f *UserIdleTimeoutUpdateFrame) String() string

String implements fmt.Stringer.

type UserMuteStartedFrame 

type UserMuteStartedFrame struct {
	BaseSystemFrame
}

UserMuteStartedFrame reports that user input is now being suppressed (a mute strategy engaged). It is a system frame.

func NewUserMuteStartedFrame 

func NewUserMuteStartedFrame() *UserMuteStartedFrame

NewUserMuteStartedFrame builds a UserMuteStartedFrame.

type UserMuteStoppedFrame 

type UserMuteStoppedFrame struct {
	BaseSystemFrame
}

UserMuteStoppedFrame reports that user input is no longer suppressed. It is a system frame.

func NewUserMuteStoppedFrame 

func NewUserMuteStoppedFrame() *UserMuteStoppedFrame

NewUserMuteStoppedFrame builds a UserMuteStoppedFrame.

type UserSpeakingFrame 

type UserSpeakingFrame struct {
	BaseSystemFrame
}

UserSpeakingFrame is emitted periodically while the user is speaking, a keepalive that lets strategies and idle logic know audio is still arriving. It is a system frame.

func NewUserSpeakingFrame 

func NewUserSpeakingFrame() *UserSpeakingFrame

NewUserSpeakingFrame builds a UserSpeakingFrame.

type UserStartedSpeakingFrame 

type UserStartedSpeakingFrame struct {
	BaseSystemFrame
}

UserStartedSpeakingFrame indicates the user turn has started. It is a system frame.

func NewUserStartedSpeakingFrame 

func NewUserStartedSpeakingFrame() *UserStartedSpeakingFrame

NewUserStartedSpeakingFrame builds a UserStartedSpeakingFrame.

type UserStoppedSpeakingFrame 

type UserStoppedSpeakingFrame struct {
	BaseSystemFrame
}

UserStoppedSpeakingFrame indicates the user turn has ended. It is a system frame.

func NewUserStoppedSpeakingFrame 

func NewUserStoppedSpeakingFrame() *UserStoppedSpeakingFrame

NewUserStoppedSpeakingFrame builds a UserStoppedSpeakingFrame.

type UserTurnInferenceCompletedFrame 

type UserTurnInferenceCompletedFrame struct {
	BaseControlFrame
}

UserTurnInferenceCompletedFrame signals that an external judge (an LLM completion gate, an EOT classifier) decided the user's turn is semantically complete. A turn-stop strategy waits for it to finalize the turn. It is a control frame.

func NewUserTurnInferenceCompletedFrame 

func NewUserTurnInferenceCompletedFrame() *UserTurnInferenceCompletedFrame

NewUserTurnInferenceCompletedFrame builds a UserTurnInferenceCompletedFrame.

type UserTurnRecommendation added in v0.0.4 

type UserTurnRecommendation int

UserTurnRecommendation is a turn-taking strategy a service recommends to the user-turn aggregator through a metadata frame. The aggregator adopts the recommendation only when the application did not configure its own turn strategies, which always win. 

const (
	// UserTurnUnspecified makes no recommendation; the configured or default
	// strategies stay in place.
	UserTurnUnspecified UserTurnRecommendation = iota
	// UserTurnExternal recommends external turn strategies: the service performs
	// its own server-side end-of-turn detection and emits the user speaking
	// frames, so the pipeline relays them rather than running VAD-based turns.
	UserTurnExternal
)

func (UserTurnRecommendation) String added in v0.0.4 

func (r UserTurnRecommendation) String() string

String implements fmt.Stringer.

type VADUserStartedSpeakingFrame 

type VADUserStartedSpeakingFrame struct {
	BaseSystemFrame
	// StartSecs is the VAD's confirmation delay (how long speech persisted
	// before onset was confirmed), in seconds.
	StartSecs float64
}

VADUserStartedSpeakingFrame reports that a voice-activity detector heard the user start speaking. It is the raw VAD signal the turn subsystem consumes to decide a user turn; it is distinct from UserStartedSpeakingFrame, which is the turn decision. It is a system frame.

func NewVADUserStartedSpeakingFrame 

func NewVADUserStartedSpeakingFrame(startSecs float64) *VADUserStartedSpeakingFrame

NewVADUserStartedSpeakingFrame builds a VADUserStartedSpeakingFrame.

func (*VADUserStartedSpeakingFrame) String 

func (f *VADUserStartedSpeakingFrame) String() string

String implements fmt.Stringer.

type VADUserStoppedSpeakingFrame 

type VADUserStoppedSpeakingFrame struct {
	BaseSystemFrame
	// StopSecs is the silence duration the VAD required before confirming the
	// stop, in seconds.
	StopSecs float64
	// Timestamp is when the stop was detected, as an RFC3339 string; "" when
	// unset.
	Timestamp string
}

VADUserStoppedSpeakingFrame reports that the VAD heard the user stop speaking. It is a system frame.

func NewVADUserStoppedSpeakingFrame 

func NewVADUserStoppedSpeakingFrame(stopSecs float64, timestamp string) *VADUserStoppedSpeakingFrame

NewVADUserStoppedSpeakingFrame builds a VADUserStoppedSpeakingFrame.

func (*VADUserStoppedSpeakingFrame) String 

func (f *VADUserStoppedSpeakingFrame) String() string

String implements fmt.Stringer.

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.