cadence

README

cadence

A provider-agnostic Go engine for low-latency STT→LLM→TTS voice pipelines. Streaming engines behind small uniform interfaces, so your voice application never knows — or cares — which vendor is talking. Swap providers with a config change, run several side by side, or plug in your own in-house engine.

cadence owns the hot path — the latency-critical loop between a committed user turn and the first synthesized audio — and deliberately nothing else:

             your voice application
        tools · prompts · business logic
                     │
                     ▼
         ╔═══════════════════════╗
         ║        cadence        ║
 audio → ║  endpoint → STT → LLM ║
         ║               ↓       ║
 audio ← ║      TTS ← buffer     ║
         ╚═══════════════════════╝
                     │
               instrumentation

What cadence is not: a transport (bring your own telephony or WebRTC — xphone if you want a phone line in Go), an agent framework (turn policy, tools and state stay in your application), or a deployment platform. Small, embeddable, instrumented — the engine underneath a voice agent, not the agent.

Battle-tested in a production telephony platform handling real call traffic.

go get github.com/amenophis1er/cadence

Engines

Role	Engines	Streaming
STT	Deepgram (Nova), Cartesia, Whisper / WhisperLive	partials, finals, server-VAD speech events
TTS	ElevenLabs, Deepgram, Cartesia, Inworld, Kokoro	warm WebSocket per call, sentence-level flush, HTTP fallback
LLM	OpenAI-compatible (chat completions), Anthropic, Gemini	event stream with tool-call support

Every role is an interface (engines/types.go); anything satisfying it is a first-class engine — including local/self-hosted models (Ollama and other OpenAI-compatible servers work out of the box via NewOpenAICompat).

Quick start

Streaming speech-to-text

stt := engines.NewDeepgramSTT(engines.DeepgramSTTConfig{
    APIKey: os.Getenv("DEEPGRAM_API_KEY"),
    Model:  "nova-3",
})

events := make(chan engines.STTEvent, 64)
if err := stt.Start(ctx, events); err != nil {
    log.Fatal(err)
}
defer stt.Stop()

go func() {
    for frame := range telephonyAudio { // mu-law 8 kHz, 160-byte / 20 ms frames
        stt.PushAudio(frame)
    }
}()

for ev := range events {
    switch ev.Type {
    case engines.STTSpeechStarted:   // caller began talking — barge-in signal
    case engines.STTTranscriptDelta: // live partial: ev.Text
    case engines.STTTranscriptFinal: // committed utterance: ev.Text
        // ev.CommittedBy says which rule ended the turn (provider
        // speech_final vs fallback debounce); ev.HeldMs is how long the
        // engine held it locally; ev.MaxSegmentGapMs is the longest
        // is_final inter-arrival gap merged into this turn — its tail
        // across calls is the smallest safe FlushTimeoutMs. Zero on all
        // other event types.
    }
}

Streaming text-to-speech

tts := engines.NewElevenTTSWS(engines.ElevenTTSConfig{
    APIKey: os.Getenv("ELEVENLABS_API_KEY"),
    Model:  "eleven_flash_v2_5",
    Voice:  "JBFqnCBsd6RMkjVDRZzb",
})

textCh := make(chan engines.TextChunk)
audioCh := make(chan engines.AudioChunk, 64)

go tts.Stream(ctx, textCh, audioCh) // one warm connection for the whole call

textCh <- engines.TextChunk{Text: "Hello! How can I help?", IsSentenceEnd: true}
// … keep sending sentences as the LLM produces them; close(textCh) when done.

for chunk := range audioCh {
    playToCaller(chunk.Data)
}

Barge-in — from the goroutine that detects the caller speaking (e.g. on STTSpeechStarted), while the stream above is live. Engines implementing InterruptibleTTSEngine (Deepgram WS today) abandon queued speech and keep the connection warm — no teardown, no re-handshake before the agent speaks again:

if it, ok := tts.(engines.InterruptibleTTSEngine); ok {
    it.Clear() // drop queued audio; the session stays usable
} else {
    cancel() // engines without it: cancel Stream's ctx, ending the session
}

Alongside Clear, stop feeding textCh from the interrupted response (cancel the LLM turn): chunks already buffered in textCh are caller-owned and will still be spoken — see the InterruptibleTTSEngine contract.

Streaming LLM, sentence-aligned for TTS

RunSentenceBuffer sits between the LLM's token stream and TTS, flushing on sentence terminators, clause boundaries, idle timeouts or a hard word cap — so synthesis starts before generation finishes:

llm := engines.NewOpenAICompat(engines.ChatConfig{
    URL:    "https://api.openai.com/v1/chat/completions",
    APIKey: os.Getenv("OPENAI_API_KEY"),
    Model:  "gpt-4.1",
})

llmEvents := make(chan engines.LLMEvent, 64)
sentences := make(chan string, 8)

go llm.Stream(ctx, messages, nil, llmEvents)
go func() {
    for s := range sentences {
        textCh <- engines.TextChunk{Text: s, IsSentenceEnd: true} // → TTS above
    }
}()

result := cadence.RunSentenceBuffer(ctx, cadence.DefaultSentenceBufferConfig(), llmEvents, sentences)
fmt.Println("assistant said:", result.FullText)

Cost attribution

Engines that implement UsageReporter expose per-session consumption — audio seconds, tokens in/out, and cached-token counts (the effective prompt-cache hit ratio):

if ur, ok := llm.(engines.UsageReporter); ok {
    u := ur.Usage()
    log.Printf("tokens in=%d (cached=%d) out=%d", u.TokensIn, u.CachedTokens, u.TokensOut)
}

Extras

• audio/ — dependency-free G.711 μ-law ↔ PCM16 codec and 8k/16k resampling helpers.
• EnergyVAD — local energy-based voice activity detection for STT engines without server-side endpointing (onset debounce, exactly-once utterance-end signaling).
• Observability hooks — cadence records nothing itself; install engines.OnTTSStreamEnd or engines.OnSTTCommit (which attributes each utterance commit to provider endpointing vs the fallback debounce) to feed Prometheus, OpenTelemetry, or plain logs. Hooks run on engine hot paths — record and return, don't block.

Measured, not claimed

The hot path is characterized, not guessed at: "The anatomy of time-to-first-audio in a voice agent" decomposes a live LLM→buffer→TTS chain (median post-commit first audio 1.06 s: ~59 % LLM, ~28 % TTS, ~14 % cadence's buffering) and sweeps the sentence-buffer and VAD trade-offs. The harnesses live in bench/ — reproduce the numbers before trusting them:

Harness	Measures	Needs
bench/sbbench	sentence-buffer time-to-first-chunk vs word rate × config	nothing
bench/vadbench	VAD hesitation-cut rate + commit latency vs offset	nothing
bench/e2ebench	live LLM → buffer → TTS stage decomposition	provider keys

Design notes

• Interfaces over adapters. The application depends on STTEngine / TTSEngine / LLMEngine; vendors are implementation details. The streaming variants (StreamingSTTEngine, StreamingTTSEngine) are optional richer contracts — consumers prefer them when available.
• Telephony-native. Audio contracts are μ-law 8 kHz, 20 ms frames — what real phone networks speak. PushAudio is non-blocking and drops on overrun rather than stalling the audio loop.
• Barge-in without teardown where the vendor allows it. Engines implementing InterruptibleTTSEngine expose Clear(): queued speech is abandoned, the session stays warm. Everywhere else, cancel the context — the engine sends the vendor's EOS, drains, and returns.
• Orchestration stays yours. Cadence deliberately ships engines, not a conversation loop — turn-taking, interruption policy and state belong to the application.

License

MIT — see LICENSE.

Documentation

Overview

Package cadence provides provider-agnostic voice AI engines: streaming speech-to-text, text-to-speech and LLM chat behind small uniform interfaces, plus the glue a real-time voice pipeline needs between them (sentence-aligned buffering for TTS, local energy VAD).

The engine implementations and their contracts live in the engines subpackage; G.711/PCM audio helpers live in audio. This root package carries the pieces that sit between engines: RunSentenceBuffer turns an LLM token stream into TTS-sized sentences, and EnergyVAD detects speech onset/offset for STT engines without server-side endpointing.

Cadence deliberately does not include a conversation loop — turn-taking, barge-in policy and dialogue state belong to the application.

Index

• func MuLawRMS(mu []byte) float64
• type EnergyVAD
  • func NewEnergyVAD() *EnergyVAD
  • func (v *EnergyVAD) Observe(mu []byte) (speakingNow bool, justEnded bool)
  • func (v *EnergyVAD) Reset()
• type SentenceBufferConfig
  • func DefaultSentenceBufferConfig() SentenceBufferConfig
• type SentenceBufferResult
  • func RunSentenceBuffer(ctx context.Context, cfg SentenceBufferConfig, in <-chan engines.LLMEvent, ...) SentenceBufferResult

Functions

func MuLawRMS

func MuLawRMS(mu []byte) float64

Types

type EnergyVAD

type EnergyVAD struct {
	// contains filtered or unexported fields
}

EnergyVAD is a stateful, RMS-based voice activity detector. It consumes 20 ms frames of mu-law 8 kHz audio (160 bytes each) and reports whether the user has just stopped speaking.

The detector is intentionally simple — speech-vs-silence by short-time RMS against a fixed threshold, with frame-count debouncing for both onset and offset. Replace with Silero (CGo onnxruntime) when noise robustness matters.

func NewEnergyVAD

func NewEnergyVAD() *EnergyVAD

func (*EnergyVAD) Observe

func (v *EnergyVAD) Observe(mu []byte) (speakingNow bool, justEnded bool)

observe ingests one mu-law frame and returns (speakingNow, justEnded). "justEnded" fires exactly once per utterance, on the frame that crosses the silence threshold after speech.

func (*EnergyVAD) Reset

func (v *EnergyVAD) Reset()

reset clears all state, e.g. after committing a turn.

type SentenceBufferConfig

type SentenceBufferConfig struct {
	ClauseMinWords int           // flush at "," ";" ":" "—" only after this many words
	MaxWords       int           // hard cap regardless of punctuation
	FlushTimeout   time.Duration // flush after this much idle time since last delta
}

SentenceBufferConfig tunes the LLM-to-TTS framing pipeline. Defaults match the original Rust prototype: flush on hard sentence terminators, on clause boundaries once enough words have accumulated, on inactivity, or on a hard word cap. The numbers trade first-token latency against TTS prosody.

func DefaultSentenceBufferConfig

func DefaultSentenceBufferConfig() SentenceBufferConfig

type SentenceBufferResult

type SentenceBufferResult struct {
	FullText  string
	ToolCalls []engines.ToolCall
}

SentenceBufferResult is what RunSentenceBuffer hands back when the upstream LLM event channel closes.

func RunSentenceBuffer

func RunSentenceBuffer(
	ctx context.Context,
	cfg SentenceBufferConfig,
	in <-chan engines.LLMEvent,
	out chan<- string,
) SentenceBufferResult

RunSentenceBuffer consumes a stream of LLMEvents (text deltas + tool calls) and emits sentence-aligned text chunks downstream so TTS can begin synthesising before the LLM has finished generating. Returns once `in` closes or `ctx` cancels.

`out` is closed before return so the consuming TTS goroutine unblocks.