cadence

package module
v0.1.0 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Aug 6, 2026 License: MIT Imports: 8 Imported by: 0

README

cadence

CI Go Reference

Provider-agnostic voice AI engines for Go. Streaming STT, TTS and LLM behind small uniform interfaces, so your voice application never knows — or cares — which vendor is talking. Swap providers with a config change, run several side by side, or plug in your own in-house engine.

Battle-tested in a production telephony platform handling real call traffic.

go get github.com/amenophis1er/cadence

Engines

Role Engines Streaming
STT Deepgram (Nova), Cartesia, Whisper / WhisperLive partials, finals, server-VAD speech events
TTS ElevenLabs, Deepgram, Cartesia, Inworld, Kokoro warm WebSocket per call, sentence-level flush, HTTP fallback
LLM OpenAI-compatible (chat completions), Anthropic, Gemini event stream with tool-call support

Every role is an interface (engines/types.go); anything satisfying it is a first-class engine — including local/self-hosted models (Ollama and other OpenAI-compatible servers work out of the box via NewOpenAICompat).

Quick start

Streaming speech-to-text
stt := engines.NewDeepgramSTT(engines.DeepgramSTTConfig{
    APIKey: os.Getenv("DEEPGRAM_API_KEY"),
    Model:  "nova-3",
})

events := make(chan engines.STTEvent, 64)
if err := stt.Start(ctx, events); err != nil {
    log.Fatal(err)
}
defer stt.Stop()

go func() {
    for frame := range telephonyAudio { // mu-law 8 kHz, 160-byte / 20 ms frames
        stt.PushAudio(frame)
    }
}()

for ev := range events {
    switch ev.Type {
    case engines.STTSpeechStarted:   // caller began talking — barge-in signal
    case engines.STTTranscriptDelta: // live partial: ev.Text
    case engines.STTTranscriptFinal: // committed utterance: ev.Text
    }
}
Streaming text-to-speech
tts := engines.NewElevenTTSWS(engines.ElevenTTSConfig{
    APIKey: os.Getenv("ELEVENLABS_API_KEY"),
    Model:  "eleven_flash_v2_5",
    Voice:  "JBFqnCBsd6RMkjVDRZzb",
})

textCh := make(chan engines.TextChunk)
audioCh := make(chan engines.AudioChunk, 64)

go tts.Stream(ctx, textCh, audioCh) // one warm connection for the whole call

textCh <- engines.TextChunk{Text: "Hello! How can I help?", IsSentenceEnd: true}
// … keep sending sentences as the LLM produces them; close(textCh) when done.
// Cancel ctx to barge-in: the engine drains and exits.

for chunk := range audioCh {
    playToCaller(chunk.Data)
}
Streaming LLM, sentence-aligned for TTS

RunSentenceBuffer sits between the LLM's token stream and TTS, flushing on sentence terminators, clause boundaries, idle timeouts or a hard word cap — so synthesis starts before generation finishes:

llm := engines.NewOpenAICompat(engines.ChatConfig{
    URL:    "https://api.openai.com/v1/chat/completions",
    APIKey: os.Getenv("OPENAI_API_KEY"),
    Model:  "gpt-4.1",
})

llmEvents := make(chan engines.LLMEvent, 64)
sentences := make(chan string, 8)

go llm.Stream(ctx, messages, nil, llmEvents)
go func() {
    for s := range sentences {
        textCh <- engines.TextChunk{Text: s, IsSentenceEnd: true} // → TTS above
    }
}()

result := cadence.RunSentenceBuffer(ctx, cadence.DefaultSentenceBufferConfig(), llmEvents, sentences)
fmt.Println("assistant said:", result.FullText)
Cost attribution

Engines that implement UsageReporter expose per-session consumption — audio seconds, tokens in/out, and cached-token counts (the effective prompt-cache hit ratio):

if ur, ok := llm.(engines.UsageReporter); ok {
    u := ur.Usage()
    log.Printf("tokens in=%d (cached=%d) out=%d", u.TokensIn, u.CachedTokens, u.TokensOut)
}

Extras

  • audio/ — dependency-free G.711 μ-law ↔ PCM16 codec and 8k/16k resampling helpers.
  • EnergyVAD — local energy-based voice activity detection for STT engines without server-side endpointing (onset debounce, exactly-once utterance-end signaling).
  • Observability hooks — cadence records nothing itself; install engines.OnTTSStreamEnd (and friends as they grow) to feed Prometheus, OpenTelemetry, or plain logs.

Design notes

  • Interfaces over adapters. The application depends on STTEngine / TTSEngine / LLMEngine; vendors are implementation details. The streaming variants (StreamingSTTEngine, StreamingTTSEngine) are optional richer contracts — consumers prefer them when available.
  • Telephony-native. Audio contracts are μ-law 8 kHz, 20 ms frames — what real phone networks speak. PushAudio is non-blocking and drops on overrun rather than stalling the audio loop.
  • Barge-in is context cancellation. No custom stop protocol: cancel the context, the engine sends the vendor's EOS, drains, and returns.
  • Orchestration stays yours. Cadence deliberately ships engines, not a conversation loop — turn-taking, interruption policy and state belong to the application.

License

MIT — see LICENSE.

Documentation

Overview

Package cadence provides provider-agnostic voice AI engines: streaming speech-to-text, text-to-speech and LLM chat behind small uniform interfaces, plus the glue a real-time voice pipeline needs between them (sentence-aligned buffering for TTS, local energy VAD).

The engine implementations and their contracts live in the engines subpackage; G.711/PCM audio helpers live in audio. This root package carries the pieces that sit between engines: RunSentenceBuffer turns an LLM token stream into TTS-sized sentences, and EnergyVAD detects speech onset/offset for STT engines without server-side endpointing.

Cadence deliberately does not include a conversation loop — turn-taking, barge-in policy and dialogue state belong to the application.

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

func MuLawRMS

func MuLawRMS(mu []byte) float64

Types

type EnergyVAD

type EnergyVAD struct {
	// contains filtered or unexported fields
}

EnergyVAD is a stateful, RMS-based voice activity detector. It consumes 20 ms frames of mu-law 8 kHz audio (160 bytes each) and reports whether the user has just stopped speaking.

The detector is intentionally simple — speech-vs-silence by short-time RMS against a fixed threshold, with frame-count debouncing for both onset and offset. Replace with Silero (CGo onnxruntime) when noise robustness matters.

func NewEnergyVAD

func NewEnergyVAD() *EnergyVAD

func (*EnergyVAD) Observe

func (v *EnergyVAD) Observe(mu []byte) (speakingNow bool, justEnded bool)

observe ingests one mu-law frame and returns (speakingNow, justEnded). "justEnded" fires exactly once per utterance, on the frame that crosses the silence threshold after speech.

func (*EnergyVAD) Reset

func (v *EnergyVAD) Reset()

reset clears all state, e.g. after committing a turn.

type SentenceBufferConfig

type SentenceBufferConfig struct {
	ClauseMinWords int           // flush at "," ";" ":" "—" only after this many words
	MaxWords       int           // hard cap regardless of punctuation
	FlushTimeout   time.Duration // flush after this much idle time since last delta
}

SentenceBufferConfig tunes the LLM-to-TTS framing pipeline. Defaults match the original Rust prototype: flush on hard sentence terminators, on clause boundaries once enough words have accumulated, on inactivity, or on a hard word cap. The numbers trade first-token latency against TTS prosody.

func DefaultSentenceBufferConfig

func DefaultSentenceBufferConfig() SentenceBufferConfig

type SentenceBufferResult

type SentenceBufferResult struct {
	FullText  string
	ToolCalls []engines.ToolCall
}

SentenceBufferResult is what RunSentenceBuffer hands back when the upstream LLM event channel closes.

func RunSentenceBuffer

func RunSentenceBuffer(
	ctx context.Context,
	cfg SentenceBufferConfig,
	in <-chan engines.LLMEvent,
	out chan<- string,
) SentenceBufferResult

RunSentenceBuffer consumes a stream of LLMEvents (text deltas + tool calls) and emits sentence-aligned text chunks downstream so TTS can begin synthesising before the LLM has finished generating. Returns once `in` closes or `ctx` cancels.

`out` is closed before return so the consuming TTS goroutine unblocks.

Directories

Path Synopsis
Package engines defines the pluggable STT, LLM, and TTS interfaces that the cadence orchestrator composes into a voice pipeline.
Package engines defines the pluggable STT, LLM, and TTS interfaces that the cadence orchestrator composes into a voice pipeline.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL