audiocpp

README

audiocpp-go

Go bindings for audio.cpp — text-to-speech and audio inference on ggml — via purego (no cgo).

audio.cpp exposes no C API of its own, so this project provides one: a thin extern "C" shim over its C++ engine::runtime facade, compiled into a self-contained shared library that Go loads at runtime.

How it works

• shim/audiocpp_c.{h,cpp} — the extern "C" surface (audiocpp_load / audiocpp_synthesize / audiocpp_free_samples / audiocpp_free / audiocpp_version, plus the device registry: audiocpp_device_count / audiocpp_device_info / audiocpp_device_memory / audiocpp_resolve_device). It is pointer-only, so it binds cleanly with purego. The shim wraps make_default_registry → load → create_task_session → prepare → run → AudioBuffer; the session is created per synthesis and keyed on task (tts / voice_design / voice_clone), so preset voices, voice design and cloning share one loaded model. C++ exceptions never cross the boundary — failures return a status code plus a message string.
• CMakeLists.txt — builds audio.cpp's static engine_runtime and the shim into a single SHARED libaudiocpp with global PIC and hidden visibility, exporting only the nine audiocpp_* symbols (-exported_symbols_list on macOS, --exclude-libs,ALL + a version script on Linux) so nothing from ggml/sentencepiece/etc. leaks.
• audiocpp.go + binding.go + library_{unix,windows}.go — the Go binding (package audiocpp): Load(libDir), New(ModelParams), (*Model).Synthesize(SynthParams) → *Audio, Close(), Version(), EncodeWAV(), the device registry (Devices(), DeviceMemory(), DeviceID(), ResolveDevice(), (*Model).Device()), and a typed Error{Code, Msg}. purego binds the nine symbols (the four device ones are optional, so an older library still loads); native PCM is copied into Go memory and then freed, so callers never hold C-owned buffers.

Usage

import audiocpp "github.com/pendra-ai/audiocpp-go"

// Point Load at a directory containing the prebuilt libaudiocpp for your platform.
if err := audiocpp.Load(libDir); err != nil { log.Fatal(err) }

m, err := audiocpp.New(audiocpp.ModelParams{
    ModelPath:  "/path/to/model.gguf",
    FamilyHint: "qwen3_tts",
    Backend:    "metal", // "cpu" | "cuda" | "vulkan" | "hip" | "metal" | "best"
})
if err != nil { log.Fatal(err) }
defer m.Close()

audio, err := m.Synthesize(audiocpp.SynthParams{
    Text:    "Hello from audio dot cpp, bound to Go.",
    VoiceID: "ryan",
})
if err != nil { log.Fatal(err) }

os.WriteFile("out.wav", audiocpp.EncodeWAV(audio), 0o644)

A Model is not safe for concurrent use — synthesise serially.

See examples/tts for a complete runnable example.

Devices and device memory

The library links ggml statically and hides it, so nothing outside can see which devices it uses or how it numbers them. The device registry exposes that: the library's own device list, each device's free and total memory, and the device a given model runs on. It is what lets a caller attribute a loaded model's memory to the right GPU on a host with several, without nvidia-smi (whose GPU indices are not CUDA's).

devs, err := audiocpp.Devices() // every device, in the library's own order
if errors.Is(err, audiocpp.ErrDeviceAPIUnavailable) {
    // A libaudiocpp built before the registry: it loads and synthesises without it.
}

dev, _ := m.Device()                              // the device this model runs on
before, _ := audiocpp.DeviceMemory(dev.Index)
audio, _ := m.Synthesize(params)                  // the first synthesis allocates on the device
after, _ := audiocpp.DeviceMemory(dev.Index)
footprint := before.FreeBytes - after.FreeBytes   // guard against after > before

Call	Returns
HasDeviceAPI() bool	Whether the loaded library has the registry (shim 0.2.0 or later, as Version() reports).
Devices() ([]Device, error)	Every device the library can run on. Fixed for the life of the process.
DeviceMemory(index int) (Memory, error)	FreeBytes and TotalBytes of Devices()[index], read now.
DeviceID(index int) (string, error)	The device's PCI bus id (0000:01:00.0) on CUDA, ROCm and Vulkan, else "".
ResolveDevice(backend string, device int) (Device, error)	The device ModelParams{Backend, Device} resolves to, without loading a model. cpu and best ignore device.
(*Model).Device() (Device, error)	The same, for a loaded model.

A Device carries Index (its position in Devices()), Backend (ggml's registry name: CUDA, Vulkan, MTL, ROCm, CPU), BackendIndex (the index within that backend, which is what ModelParams.Device takes), Name (CUDA0), Description (the hardware name) and Type (DeviceTypeCPU / GPU / IGPU / Accel / Meta).

Things to know when measuring with it:

• Numbering. For CUDA, BackendIndex is CUDA's own device ordinal (fastest card first unless CUDA_DEVICE_ORDER is set; CUDA_VISIBLE_DEVICES renumbers), not nvidia-smi's index. Use DeviceID to match a device against another tool's list.

• It is a device reading, not a per-model sum. A model's share is the drop in FreeBytes across its load and its first synthesis (the engine allocates on the device at the first synthesis, not at load), so it covers what the model keeps afterwards, not the peak during synthesis. audio.cpp does not account its buffers per model, so the library cannot report an exact per-model byte count. (*Model).Device() is the device the model was loaded for; a model may still keep some parts on the CPU.

• What each backend reports.

  Backend	DeviceMemory
  CUDA	The whole card, so other processes move it too. On a shared-memory host (DGX Spark, Jetson) free is the system's available memory. Every device is typed GPU, shared-memory ones included.
  Metal	This process's recommended working set less what this process has allocated.
  Vulkan	This process's heap budget less its use; FreeBytes == TotalBytes on a driver without VK_EXT_memory_budget.
  CPU (in every build)	System memory, with FreeBytes == TotalBytes except on Windows, so it cannot show a drop. Measure a CPU model by process RSS.

• Cost. Devices, ResolveDevice and (*Model).Device() only read the registry (the first call starts the backends, as the first synthesis would). DeviceMemory and DeviceID open the device they are asked about: on CUDA that creates this process's context on that GPU if it has none (a few hundred MB of that card), so read the model's device, not every device. Take the first reading before the window you want to measure, so the context is not counted as the model. A CUDA driver error during a read aborts the process, as it does inside the engine.

• Threading. All of these are stateless and safe to call from any goroutine, including while a Synthesize is running.

Building the shared library

# Clone the pinned upstream (into upstream/audio.cpp):
scripts/clone-upstream.sh

cmake -S . -B build -DCMAKE_BUILD_TYPE=Release \
  -DAUDIOCPP_SRC=upstream/audio.cpp -DAUDIOCPP_BACKEND=metal
cmake --build build --target audiocpp --parallel

# Verify only the shim's symbols are exported (fail-closed):
scripts/check-symbols.sh build/libaudiocpp.dylib

# Read the built library's device registry from Go (no model needed):
AUDIOCPP_GO_DEVICES=1 AUDIOCPP_GO_EXPECT_BACKEND=MTL AUDIOCPP_GO_RESOLVE_BACKEND=metal \
  go test -run TestDeviceRegistryIntegration .

# Go end-to-end against the local build (needs a model GGUF):
AUDIOCPP_GO_E2E=1 go test ./...

AUDIOCPP_BACKEND selects the ggml backend (cpu / cuda / vulkan / hip / metal). AUDIOCPP_NATIVE_CPU (default ON) tunes the CPU kernels for the build machine; pass -DAUDIOCPP_NATIVE_CPU=OFF for a library that must run on other CPUs. AUDIOCPP_MODEL_SET defaults to a small set for fast local builds; release builds pass -DAUDIOCPP_MODEL_SET=full.

Prebuilt libraries & releases

.github/workflows/build-libs.yml builds a self-contained libaudiocpp for each variant — linux amd64 (cpu / cuda / vulkan), linux arm64 (cpu / cuda), darwin arm64 (metal), windows amd64 (cpu / vulkan / cuda) — runs the fail-closed symbol gate on each, and publishes one vX.Y.Z release carrying the module tag (so go get github.com/pendra-ai/audiocpp-go@vX.Y.Z resolves) alongside per-variant audiocpp-libs-<os>-<arch>-<backend>.tar.gz archives and checksums.txt. The Windows CPU archive keeps its older name, audiocpp-libs-windows-amd64.tar.gz; the Windows GPU ones are audiocpp-libs-windows-amd64-vulkan.tar.gz and audiocpp-libs-windows-amd64-cuda.tar.gz. Each leg whose library can be opened on its runner (CPU, Vulkan, Metal, Windows CPU and Vulkan) also reads the device registry through the Go binding; the CUDA legs are build/link/symbol-check only (no GPU CI runners). Pull requests that touch the native build run the same matrix as a check, without publishing. The amd64 CUDA and Windows legs are best-effort, so a toolchain mismatch never blocks a release; the arm64 CUDA leg gates the release. The linux arm64 archives are built with generic CPU flags (-DAUDIOCPP_NATIVE_CPU=OFF) so they run on Arm cores other than the CI runner's. The Linux CUDA archives need libcudart.so.13, libcublas.so.13 and libcufft.so.12 from the host's CUDA 13 runtime. The Windows CUDA library needs cublas64_13.dll and cublasLt64_13.dll on the DLL search path and an NVIDIA driver (R580 or newer); cufft64_12.dll is delay-loaded, so it is needed only by a model that runs the CUDA inverse STFT (qwen3-tts does not). The Windows Vulkan library needs only the GPU driver's vulkan-1.dll. All Windows archives are built with generic CPU flags. Consumers extract an archive and pass its directory to Load(libDir).

Pin

audio.cpp is pinned in lib/version.txt. The shim compiles against audio.cpp's internal C++ headers, which move quickly, so the pin is intentionally strict.

Licensing

This project (the Go code and the extern "C" shim) is MIT-licensed — see LICENSE.

The prebuilt shared libraries statically link audio.cpp (Apache-2.0), ggml (MIT), and sentencepiece (Apache-2.0). Their licence and NOTICE files are bundled inside each released audiocpp-libs-*.tar.gz.

Documentation

Overview

Package audiocpp is a Go binding for audio.cpp's TTS engine, loaded in-process via purego (no cgo) over the extern "C" shim in shim/audiocpp_c.h. It mirrors a common purego binding approach.

The surface is deliberately small (nine C symbols), so — unlike sd-go, which splits a ~50-symbol API into pkg/sd + a root layer — the raw bindings and the ergonomic API live in one package here.

Index

• Variables
• func DeviceID(index int) (id string, err error)
• func EncodeWAV(a *Audio) []byte
• func HasDeviceAPI() bool
• func Load(libDir string) (err error)
• func Version() string
• type Audio
• type Device
  • func Devices() (devs []Device, err error)
  • func ResolveDevice(backend string, device int) (dev Device, err error)
• type DeviceType
  • func (t DeviceType) String() string
• type Error
  • func (e *Error) Error() string
• type Memory
  • func DeviceMemory(index int) (mem Memory, err error)
• type Model
  • func New(p ModelParams) (m *Model, err error)
  • func (m *Model) Close() error
  • func (m *Model) Device() (Device, error)
  • func (m *Model) Synthesize(p SynthParams) (a *Audio, err error)
• type ModelParams
• type SynthParams

Variables

var ErrDeviceAPIUnavailable = errors.New("audiocpp: the loaded library has no device registry (needs shim 0.2.0 or later)")

ErrDeviceAPIUnavailable is returned by the device functions when the loaded library predates the device registry (shim 0.2.0). Such a library still loads and synthesises; only these readings are missing.

Functions

func DeviceID

func DeviceID(index int) (id string, err error)

DeviceID returns the lower-case PCI bus id ("0000:01:00.0") of the device at index where the backend knows it (CUDA, ROCm, Vulkan), else "". It names the same card under any tool's numbering. Like DeviceMemory it opens the device.

func EncodeWAV

func EncodeWAV(a *Audio) []byte

EncodeWAV encodes an Audio result as a 16-bit PCM WAV file (little-endian). A convenience for callers/examples; the worker ships base64 WAV to the API.

func HasDeviceAPI

func HasDeviceAPI() bool

HasDeviceAPI reports whether the loaded library has the device registry. False before Load, and for a library older than shim 0.2.0.

func Load

func Load(libDir string) (err error)

Load resolves and dlopens the audiocpp shared library from libDir (an empty libDir falls back to the OS default search path). It is idempotent, safe for concurrent use, and returns an error — never panics — when the library is absent or a symbol is missing, so importing this package or calling Load with no library present keeps the caller healthy.

func Version

func Version() string

Version returns the shim's version string. Requires Load to have succeeded.

Types

type Audio

type Audio struct {
	Samples    []float32
	SampleRate int
	Channels   int
}

Audio is a synthesis result: interleaved f32 PCM plus format.

type Device

type Device struct {
	// Index is the device's position in the library's device list, as returned
	// by Devices. It is what DeviceMemory takes and is fixed for the process.
	Index int
	// Backend is the ggml registry name: "CUDA", "Vulkan", "MTL", "ROCm", "CPU".
	Backend string
	// BackendIndex is the index within Backend — the value ModelParams.Device
	// takes. For CUDA it is CUDA's own device ordinal, which is not nvidia-smi's
	// index (CUDA orders fastest first unless CUDA_DEVICE_ORDER says otherwise,
	// and CUDA_VISIBLE_DEVICES renumbers). DeviceID names the card itself.
	BackendIndex int
	// Name is ggml's device handle, e.g. "CUDA0".
	Name string
	// Description is the hardware name, e.g. "NVIDIA GeForce RTX 4090".
	Description string
	// Type is the device kind. CUDA reports DeviceTypeGPU for every device,
	// shared-memory ones (DGX Spark, Jetson) included.
	Type DeviceType
}

Device describes one device in the library's own ggml registry. The library links ggml statically and hides it, so this is the only view of the devices — and the device numbering — a loaded speech model actually uses.

func Devices

func Devices() (devs []Device, err error)

Devices lists the devices the library can run a model on, in the library's own order (Devices()[i].Index == i). The list is fixed for the life of the process, so callers may cache it. Safe for concurrent use, including during a Synthesize. Listing runs the backend start-up the first synthesis would run anyway, and does not read from the devices.

func ResolveDevice

func ResolveDevice(backend string, device int) (dev Device, err error)

ResolveDevice returns the device a model loaded with ModelParams.Backend backend and ModelParams.Device device runs on, resolved the way the engine resolves them ("" is "cpu"; "best" is the first GPU, else the first integrated GPU, else the CPU; both ignore device). It needs no loaded model, so a caller can pick the device to watch before New. An unknown backend, or a device this library does not have, is an *Error with the bad-argument code.

type DeviceType

type DeviceType int

DeviceType is the kind of a device, mirroring ggml's ggml_backend_dev_type.

const (
	DeviceTypeCPU   DeviceType = 0 // host CPU, system memory
	DeviceTypeGPU   DeviceType = 1 // GPU with its own memory
	DeviceTypeIGPU  DeviceType = 2 // integrated GPU sharing host memory
	DeviceTypeAccel DeviceType = 3 // accelerator used alongside the CPU backend
	DeviceTypeMeta  DeviceType = 4 // wrapper over several devices
)

func (DeviceType) String

func (t DeviceType) String() string

type Error

type Error struct {
	Code int
	Msg  string
}

Error is a synthesis or load failure from the engine. Code is the coarse shim status (see the shim's audiocpp_status); Msg is the engine's free-text message. The whole point of the in-process binding over the old HTTP path is that this arrives synchronously — no status code to re-parse off a response.

func (*Error) Error

func (e *Error) Error() string

type Memory

type Memory struct {
	FreeBytes  uint64
	TotalBytes uint64
}

Memory is a device's free and total memory in bytes, as the backend reports it. CUDA reports the whole card, so every process's use counts; Metal and Vulkan report this process's budget less this process's use (and Vulkan reports FreeBytes == TotalBytes on a driver without VK_EXT_memory_budget). Both are zero when the backend has no reading.

func DeviceMemory

func DeviceMemory(index int) (mem Memory, err error)

DeviceMemory reads the free and total memory of the device at index (a Device.Index) now. It is not a per-model figure: a model's share is the drop in FreeBytes across its load and its first synthesis — the engine allocates on the device at the first synthesis, not at load. The CPU device reports FreeBytes == TotalBytes except on Windows, so it cannot show a drop. Safe for concurrent use, including during a Synthesize.

Reading opens the device: on CUDA it creates this process's context on that GPU if there is none yet (a few hundred MB of its memory), so read the model's device rather than every device. A CUDA driver error during the read aborts the process, as it does inside the engine.

type Model

type Model struct {
	// contains filtered or unexported fields
}

Model is a loaded TTS model. It is NOT safe for concurrent use — synthesise serially — hold one model behind a single-slot mutex. An internal mutex is kept only to prevent a use-after-free race between Synthesize and Close, not to enable parallelism.

func New

func New(p ModelParams) (m *Model, err error)

New loads a model. The returned Model must be Close()d to free native memory.

func (*Model) Close

func (m *Model) Close() error

Close unloads the model and frees native memory. Idempotent.

func (*Model) Device

func (m *Model) Device() (Device, error)

Device returns the device the model was loaded for: ResolveDevice for its backend and device. (A model may still keep some of its parts on the CPU.) It does not touch the native model, so it is safe during a Synthesize and after Close.

func (*Model) Synthesize

func (m *Model) Synthesize(p SynthParams) (a *Audio, err error)

Synthesize turns text into audio. Safe against a concurrent Close (returns an error if the model is closed); concurrent Synthesize calls are serialized.

type ModelParams

type ModelParams struct {
	ModelPath   string            // path to the GGUF (required)
	FamilyHint  string            // audio.cpp family, e.g. "qwen3_tts"
	Backend     string            // "cpu"|"metal"|"cuda"|"vulkan"|"hip"|"best" ("" == cpu)
	Device      int               // GPU device index
	Threads     int               // CPU threads (<=0 -> engine default)
	LoadOptions map[string]string // extra load-time options
}

ModelParams configures a model load.

type SynthParams

type SynthParams struct {
	Task          string            // "tts" (default) | "voice_design" | "voice_clone"
	Text          string            // text to speak (required)
	VoiceID       string            // preset speaker id, e.g. "ryan"
	RefPCM        []float32         // reference-audio voice (mono f32); alternative to VoiceID
	RefSampleRate int               // sample rate of RefPCM
	Options       map[string]string // per-request options (instruct, seed, temperature, ...)
}

SynthParams configures one synthesis call.