synthesizer

Documentation

Overview

Package synthesizer defines the core types and interfaces for text-to-speech synthesis.

This is the provider-agnostic core package. Vendor-specific implementations live in submodules (e.g. synthesizer/aliyun, synthesizer/openai, synthesizer/volcengine).

Index

• Constants
• func ComputeSampleByteCount(sampleRate, bitDepth, channels int) int
• func EmitPCMChunks(ctx context.Context, handler Handler, pcm []byte, cfg PCMEmitConfig) error
• func FrameBytes(cfg PCMEmitConfig) int
• func HashText(text string) string
• func NormalizeFramePeriod(d string) time.Duration
• func RegisterAllProviders(f *DefaultFactory, registrations map[Provider]Creator)
• func SetGlobalFactory(factory *DefaultFactory)
• func StripEmoji(text string) string
• type Capabilities
  • func DefaultCapabilities() Capabilities
  • func StreamingCapabilities() Capabilities
• type CapableEngine
• type Config
• type Creator
• type DefaultFactory
  • func GetGlobalFactory() *DefaultFactory
  • func NewFactory() *DefaultFactory
  • func (f *DefaultFactory) CreateEngine(config Config) (Engine, error)
  • func (f *DefaultFactory) GetSupportedProviders() []Provider
  • func (f *DefaultFactory) IsProviderSupported(provider Provider) bool
  • func (f *DefaultFactory) RegisterCreator(provider Provider, creator Creator)
• type Engine
  • func Create(config Config) (Engine, error)
  • func MustCreate(config Config) Engine
• type Factory
• type Handler
• type HandlerFunc
  • func (h HandlerFunc) OnMessage(data []byte)
  • func (h HandlerFunc) OnTimestamp(ts SentenceTimestamp)
• type PCMEmitConfig
  • func PCMEmitConfigFromFormat(f StreamFormat) PCMEmitConfig
• type Provider
  • func AllProviders() []Provider
  • func (p Provider) ToString() string
• type SentenceTimestamp
• type StreamFormat
  • func DefaultFormat() StreamFormat
• type SynthesisBuffer
  • func (s *SynthesisBuffer) OnMessage(data []byte)
  • func (s *SynthesisBuffer) OnTimestamp(ts SentenceTimestamp)
• type Word

Constants

const (
	TTS_QCLOUD            = "tts.qcloud"
	TTS_XUNFEI            = "tts.xunfei"
	TTS_QINIU             = "tts.qiniu"
	TTS_BAIDU             = "tts.baidu"
	TTS_GOOGLE            = "tts.google"
	TTS_AWS               = "tts.aws"
	TTS_AZURE             = "tts.azure"
	TTS_OPENAI            = "tts.openai"
	TTS_ELEVENLABS        = "tts.elevenlabs"
	TTS_LOCAL             = "tts.local"
	TTS_LOCAL_GOSPEECH    = "tts.local_gospeech"
	TTS_FISHSPEECH        = "tts.fishspeech"
	TTS_FISHAUDIO         = "tts.fishaudio"
	TTS_COQUI             = "tts.coqui"
	TTS_VOLCENGINE        = "tts.volcengine"
	TTS_VOLCENGINE_CLONE  = "tts.volcengine_clone"
	TTS_VOLCENGINE_LLM    = "tts.volcengine_llm"
	TTS_VOLCENGINE_STREAM = "tts.volcengine_stream"
	TTS_MINIMAX           = "tts.minimax"
	TTS_ALIYUN            = "tts.aliyun"
)

TTS provider string constants (matching LingEchoX naming for compatibility).

Functions

func ComputeSampleByteCount

func ComputeSampleByteCount(sampleRate, bitDepth, channels int) int

ComputeSampleByteCount computes the number of bytes for audio samples based on sample rate, bit depth, and number of channels. Formula: (sampleRate * bitDepth * channels) / 8

func EmitPCMChunks

func EmitPCMChunks(ctx context.Context, handler Handler, pcm []byte, cfg PCMEmitConfig) error

EmitPCMChunks delivers batch PCM to handler as fixed-size frames. Batch vendors should use this so all engines share the same push-chunk contract.

func FrameBytes

func FrameBytes(cfg PCMEmitConfig) int

FrameBytes returns PCM bytes for one frame from cfg.

func HashText

func HashText(text string) string

HashText returns a short hex digest of the input text, suitable for cache keys.

func NormalizeFramePeriod

func NormalizeFramePeriod(d string) time.Duration

NormalizeFramePeriod parses and validates a duration string, clamping to 10-300ms range with 20ms default.

func RegisterAllProviders

func RegisterAllProviders(f *DefaultFactory, registrations map[Provider]Creator)

RegisterAllProviders registers all known provider creators into the given factory. Provider submodules call RegisterCreator individually; this helper provides a single entry point for consumers that want to register all providers at once.

func SetGlobalFactory

func SetGlobalFactory(factory *DefaultFactory)

SetGlobalFactory sets the global factory instance.

func StripEmoji

func StripEmoji(text string) string

StripEmoji removes emoji characters from text.

Types

type Capabilities

type Capabilities struct {
	// StreamingTTFB is true when the first audio chunk can arrive before synthesis completes.
	StreamingTTFB bool
	// SuggestedFirstMaxRunes is a segmenter hint for the first LLM→TTS chunk.
	SuggestedFirstMaxRunes int
}

Capabilities describes vendor-specific synthesis behavior for pipeline tuning.

func DefaultCapabilities

func DefaultCapabilities() Capabilities

DefaultCapabilities returns conservative defaults for batch-oriented vendors.

func StreamingCapabilities

func StreamingCapabilities() Capabilities

StreamingCapabilities returns capabilities for streaming vendors.

type CapableEngine

type CapableEngine interface {
	Engine
	Capabilities() Capabilities
}

CapableEngine optionally exposes vendor capabilities.

type Config

type Config interface {
	GetProvider() Provider
}

Config is the unified TTS configuration interface.

type Creator

type Creator func(Config) (Engine, error)

Creator is a function that creates an Engine from a Config.

type DefaultFactory

type DefaultFactory struct {
	// contains filtered or unexported fields
}

DefaultFactory is the thread-safe default implementation of Factory.

func GetGlobalFactory

func GetGlobalFactory() *DefaultFactory

GetGlobalFactory returns the global factory instance.

func NewFactory

func NewFactory() *DefaultFactory

NewFactory creates a new empty factory.

func (*DefaultFactory) CreateEngine

func (f *DefaultFactory) CreateEngine(config Config) (Engine, error)

CreateEngine looks up the creator for the config's provider and invokes it.

func (*DefaultFactory) GetSupportedProviders

func (f *DefaultFactory) GetSupportedProviders() []Provider

GetSupportedProviders returns all registered providers.

func (*DefaultFactory) IsProviderSupported

func (f *DefaultFactory) IsProviderSupported(provider Provider) bool

IsProviderSupported checks if a provider is registered.

func (*DefaultFactory) RegisterCreator

func (f *DefaultFactory) RegisterCreator(provider Provider, creator Creator)

RegisterCreator registers a creator function for a provider.

type Engine

type Engine interface {
	// Provider returns the vendor identifier.
	Provider() Provider
	// Format returns the audio output format.
	Format() StreamFormat
	// CacheKey returns a unique cache key for the given text.
	CacheKey(text string) string
	// Synthesize converts text to speech and delivers audio via the handler.
	Synthesize(ctx context.Context, handler Handler, text string) error
	// Close releases resources.
	Close() error
}

Engine is the core TTS engine interface that all vendors implement.

func Create

func Create(config Config) (Engine, error)

Create is a convenience function using the global factory.

func MustCreate

func MustCreate(config Config) Engine

MustCreate is like Create but panics on error.

type Factory

type Factory interface {
	CreateEngine(config Config) (Engine, error)
	GetSupportedProviders() []Provider
	IsProviderSupported(provider Provider) bool
	RegisterCreator(provider Provider, creator Creator)
}

Factory creates TTS engines by provider.

type Handler

type Handler interface {
	// OnMessage is called for each audio chunk (PCM or encoded).
	OnMessage(data []byte)
	// OnTimestamp is called when word-level timestamps are available.
	OnTimestamp(ts SentenceTimestamp)
}

Handler is the callback interface for receiving TTS events.

type HandlerFunc

type HandlerFunc struct {
	OnMessageFn   func(data []byte)
	OnTimestampFn func(ts SentenceTimestamp)
}

HandlerFunc is a convenience type for implementing Handler with functions.

func (HandlerFunc) OnMessage

func (h HandlerFunc) OnMessage(data []byte)

func (HandlerFunc) OnTimestamp

func (h HandlerFunc) OnTimestamp(ts SentenceTimestamp)

type PCMEmitConfig

type PCMEmitConfig struct {
	SampleRate int
	BitDepth   int
	Channels   int
	FrameMS    int
}

PCMEmitConfig controls fixed-frame PCM delivery to handlers.

func PCMEmitConfigFromFormat

func PCMEmitConfigFromFormat(f StreamFormat) PCMEmitConfig

PCMEmitConfigFromFormat builds emit config from a stream format.

type Provider

type Provider string

Provider identifies a TTS service vendor.

const (
	ProviderQiniu           Provider = "qiniu"
	ProviderXunfei          Provider = "xunfei"
	ProviderAliyun          Provider = "aliyun"
	ProviderTencent         Provider = "qcloud"
	ProviderBaidu           Provider = "baidu"
	ProviderAzure           Provider = "azure"
	ProviderGoogle          Provider = "google"
	ProviderAWS             Provider = "aws"
	ProviderOpenAI          Provider = "openai"
	ProviderElevenLabs      Provider = "elevenlabs"
	ProviderLocal           Provider = "local"
	ProviderLocalGoSpeech   Provider = "local_gospeech"
	ProviderFishSpeech      Provider = "fishspeech"
	ProviderFishAudio       Provider = "fishaudio"
	ProviderCoqui           Provider = "coqui"
	ProviderVolcengine      Provider = "volcengine"
	ProviderVolcengineClone Provider = "volcengine_clone"
	ProviderVolcengineLLM   Provider = "volcengine_llm"
	ProviderMinimax         Provider = "minimax"
)

func AllProviders

func AllProviders() []Provider

AllProviders returns all known provider constants in a deterministic order.

func (Provider) ToString

func (p Provider) ToString() string

ToString returns the string representation of the provider.

type SentenceTimestamp

type SentenceTimestamp struct {
	Words []Word `json:"words"`
}

SentenceTimestamp holds word-level timestamps for a synthesized sentence.

type StreamFormat

type StreamFormat struct {
	SampleRate    int           // e.g. 16000, 24000
	BitDepth      int           // 8, 16, 24, 32
	Channels      int           // 1 = mono, 2 = stereo
	Codec         string        // "pcm", "mp3", "wav", "opus"
	FrameDuration time.Duration // e.g. 20ms
}

StreamFormat describes the audio output format.

func DefaultFormat

func DefaultFormat() StreamFormat

DefaultFormat returns a sensible default audio format (16kHz, 16-bit, mono, PCM).

type SynthesisBuffer

type SynthesisBuffer struct {
	Data      []byte
	Timestamp SentenceTimestamp
}

SynthesisBuffer is a simple Handler that accumulates all audio chunks and the last timestamp. Useful for batch synthesis where callers want the full audio buffer at once.

func (*SynthesisBuffer) OnMessage

func (s *SynthesisBuffer) OnMessage(data []byte)

func (*SynthesisBuffer) OnTimestamp

func (s *SynthesisBuffer) OnTimestamp(ts SentenceTimestamp)

type Word

type Word struct {
	Confidence float64 `json:"confidence"`
	EndTime    int     `json:"end_time"`   // milliseconds
	StartTime  int     `json:"start_time"` // milliseconds
	Word       string  `json:"word"`
}

Word represents a single word with timing information.