chunkachino

README

chunkachino

A tiny, fast, pure-Go streaming chunker for the LLM-token → TTS-chunk pipeline.

The hard rule: Add never returns a chunk that ends mid-word. LLM providers love to split a single logical token like ". ho" / "pe you..." across the wire. A naive chunker flushes the "." immediately and ships ". ho" to the TTS engine, which dutifully reads "dot ho". chunkachino waits for the next whitespace (or end-of-stream Flush) before emitting, so every chunk is guaranteed word-complete.

Install

go get github.com/coffyg/chunkachino

Zero runtime dependencies. Go 1.23+.

Quick example

import "github.com/coffyg/chunkachino"

c := chunkachino.New(chunkachino.Config{
    Mode:     chunkachino.ModeSentence,
    MaxWords: 30,
    Language: "en",
})

for tok := range llmStream {
    for _, chunk := range c.Add(tok) {
        tts.Speak(chunk)
    }
}
if tail := c.Flush(); tail != "" {
    tts.Speak(tail)
}

Add returns a slice (usually empty, sometimes one, occasionally several) because a single large token can carry multiple complete sentences.

Modes

Mode	Flushes on	Abbreviation/decimal aware	Safety valve
ModeWord	every MinWords complete words	n/a	n/a
ModeSentence	. ! ? … at a real sentence boundary	yes	MaxWords
ModePhrase	. ! ? , ; : at a word boundary	yes (for . ! ?)	MaxWords

ModeSentence is the default recommendation for long-form TTS. ModePhrase gives you commas-and-colons granularity for prosody at the cost of more, shorter chunks. ModeWord is for very low-latency "talk as tokens arrive" pipelines.

Supported language keys for the abbreviation catalog:

Key	Catalog	Notes
en-US	English
es-ES	Spanish	Peninsular Spanish (Sr., EE.UU., p.ej., ...)
fr-FR	French	M., Mme., p.ex., c.-à-d., ...
pt-PT	Portuguese	European Portuguese (Lda., V. Exa., séc., ...)
de-DE	German	z.B., d.h., u.a., v.Chr., Str., ...
en	English	legacy short alias for en-US
fr	French	legacy short alias for fr-FR

Anything else falls back to English. There is intentionally no BCP 47 primary-subtag fallback tree — en-GB, es, pt, de, ja-JP, etc. all resolve to English.

What counts as a "real" sentence end

The following do not trigger a flush:

• Mr. Smith went home. — Mr, Mrs, Ms, Dr, Prof, Sr, Jr, St, Rev, Hon, Capt, etc.
• Eat fruit, e.g. apples. — e.g, i.e, etc, vs, viz, cf, al, Ph.D, M.D, a.m, p.m, Jan, Feb, ...
• Le Dr. Martin est là. — M, Mm, Mme, Mmes, Mlle, Mgr, Me, Dr, Pr, Ste, p.ex, c.-à-d, ...
• El Sr. García vive aquí. — Sr, Sra, Srta, Dr, Dra, Ud, Uds, p.ej, EE.UU, S.A, pág, ...
• O Prof. Dr. Santos chegou. — Sr, Dr, Prof, Eng, Exmo, V. Exa, Lda, séc, pág, p.ex, ...
• z.B. der Dr. Müller ist hier. — z.B, d.h, u.a, bzw, usw, ggf, ca, Nr, v.Chr, Str, Bd, ...
• The price is $3.5 today. — digit-on-both-sides decimals, version strings like 1.2.0.
• Pi is ٣.١٤ approximately. — Arabic-Indic and other Unicode digits count too.
• President J. F. Kennedy spoke. — single-letter initial runs detected with one-rune lookahead.
• Cumprimentos a V. Exa. hoje. — spaced compound abbreviations (letter. + Word.) checked as a squashed key.
• She said "hi." — terminator inside closing quote/bracket still flushes at the quote.
• Visit www.example.com today. — periods inside URLs, emails, phone numbers have no adjacent whitespace so never trigger a flush.
• Earned 1,000 today. — commas inside numbers don't flush in phrase mode.
• Wait... Really?! Okay. — stacked terminators (..., ?!, !!!) collapse to one chunk each.
• Hi 👨‍👩‍👧 friend. — multi-rune emoji sequences (ZWJ, skin tones, regional indicators) are never split mid-cluster.

Safety valve

In ModeSentence and ModePhrase, if the buffered word count exceeds MaxWords without a real terminator, chunkachino flushes at the next whitespace so a rambling LLM response never stalls the TTS pipeline. Default MaxWords is 30. Set to 0 to disable (the chunker will then wait forever for a terminator — not recommended in production).

Runt gate (MinChunkWords, v0.0.2)

A lone interjection — haha..., heh..., Ah. — is a legal sentence, so sentence mode flushes it as its own chunk, and a TTS engine then gives one word full sentence prosody. It sounds weird every time.

Set MinChunkWords to refuse terminator flushes below N complete words: the runt stays buffered and merges into the sentence that follows ("haha... You should have seen his face." emits as ONE chunk). Consecutive runts accumulate until they jointly clear the bar. The MaxWords valve still applies, and an end-of-stream runt still ships via Flush — there is nothing after it to merge into, and merging backward would cost every chunk one sentence of latency.

0 (default) disables the gate — existing pipelines are byte-exact. Ignored in ModeWord.

c := chunkachino.New(chunkachino.Config{
    Mode:          chunkachino.ModeSentence,
    MaxWords:      30,
    MinChunkWords: 3, // "haha..." rides with its neighbor instead of speaking alone
    Language:      "en",
})

Performance

Measured on a 13th Gen Intel i7-13700:

BenchmarkChunkerAdd_Sentence      62_897_626    48.43 ns/op    5 B/op    0 allocs/op
BenchmarkChunkerAdd_Phrase        69_535_870    43.84 ns/op    6 B/op    0 allocs/op
BenchmarkChunkerAdd_Word          93_981_572    38.17 ns/op    5 B/op    0 allocs/op
BenchmarkChunkerAdd_Spanish       48_046_845    49.34 ns/op    5 B/op    0 allocs/op
BenchmarkChunkerAdd_Portuguese    44_895_519    53.27 ns/op    6 B/op    0 allocs/op
BenchmarkChunkerAdd_German        36_095_850    60.81 ns/op    7 B/op    0 allocs/op
BenchmarkChunkerFullTurn             592_878  6_203    ns/op  656 B/op   19 allocs/op
BenchmarkChunkerOneBigBlob           584_074  5_677    ns/op  768 B/op   15 allocs/op

Sub-100 ns per Add on every locale's common path and zero heap allocations. A full LLM turn (~30 tokens of text) measures around 6 microseconds end-to-end.

Run tests

./test.sh     # verbose + race detector
./bench.sh    # benchmarks

Coverage: 95.6% of statements.

Design notes

• Chunker is a []rune buffer with a single forward-moving scan cursor. Abbreviation/decimal/initial checks are rune-indexed so unicode (French accents, smart quotes, ellipsis) works without special-casing.
• When disambiguation requires lookahead that hasn't arrived yet (the classic "J. — is this the end of a sentence or the start of J. F. K.?"), the scanner rewinds the cursor to the whitespace and waits for the next Add instead of emitting a wrong boundary.
• Not safe for concurrent use from multiple goroutines. Give each stream its own Chunker (or call Reset between messages).

License

MIT — see LICENSE

Documentation

Overview

Package chunkachino is a tiny, fast, pure-Go streaming chunker built for the LLM-token → TTS-chunk pipeline.

The hard invariant: Add never returns a chunk that ends mid-word. LLM providers love to split a token like ". ho" / "pe you..." across the wire; a naive chunker would flush the "." immediately and ship ". ho" to the TTS engine, producing "dot ho" / "pe you" garbage. Chunkachino waits for the next whitespace (or end-of-stream Flush) before emitting, so every chunk is guaranteed word-complete.

Three modes cover the common TTS cases:

• ModeWord: flush every N complete words at a whitespace boundary. Useful for very low-latency "talk as tokens arrive" pipelines.
• ModeSentence: flush on . ! ? at a real sentence boundary (abbreviation and decimal aware). Safety valve flushes at MaxWords if a terminator never arrives.
• ModePhrase: flush on . ! ? , ; : at a word boundary. Finer granularity for prosody in long-form TTS. Same safety valve.

Sentence/phrase mode also takes an optional MinChunkWords runt gate: a terminator on a chunk shorter than N complete words does not flush, so a lone "haha..." merges into the sentence that follows it instead of reaching the TTS engine as its own weird-prosody utterance.

The API is intentionally minimal:

c := chunkachino.New(chunkachino.Config{
    Mode:     chunkachino.ModeSentence,
    MinWords: 3,
    MaxWords: 30,
    Language: "en",
})
for tok := range llmStream {
    for _, chunk := range c.Add(tok) {
        tts.Speak(chunk)
    }
}
if tail := c.Flush(); tail != "" {
    tts.Speak(tail)
}

Add returns a slice (usually empty, sometimes length 1, occasionally more) so a single large token carrying several sentences drains cleanly. This is the same shape stream2sentence and other production streaming chunkers converged on; the "one chunk per call" API reads nicer in docs but forces ugly re-entrancy in callers the first time a batched token arrives.

Index

• type Chunker
  • func New(cfg Config) *Chunker
  • func (c *Chunker) Add(token string) []string
  • func (c *Chunker) Flush() string
  • func (c *Chunker) Reset()
• type Config
• type Mode

Types

type Chunker

type Chunker struct {
	// contains filtered or unexported fields
}

Chunker is a stateful streaming text chunker. NOT safe for concurrent use from multiple goroutines; each stream should own its own Chunker.

func New

func New(cfg Config) *Chunker

New returns a ready-to-use Chunker with the given config. Missing fields are filled with defaults: MinWords=5, MaxWords=30, Language="en".

func (*Chunker) Add

func (c *Chunker) Add(token string) []string

Add feeds a token from the LLM stream into the chunker and returns any chunks that just became ready. Most tokens yield zero chunks; a token that completes a sentence/phrase/word-count boundary yields one; a single big token containing multiple complete sentences can yield several at once.

Empty tokens are a no-op and return a nil slice. Whitespace-only tokens are accepted — they often carry the boundary signal (the space after ". ").

func (*Chunker) Flush

func (c *Chunker) Flush() string

Flush returns any remaining buffered text at end-of-stream. The returned string may be empty. After Flush the chunker is empty but NOT reset — config and language are preserved; call Reset to reuse for a new message.

func (*Chunker) Reset

func (c *Chunker) Reset()

Reset clears all buffered state but keeps the config. Use this to reuse a Chunker across independent messages.

type Config

type Config struct {
	// Mode selects the flush strategy.
	Mode Mode

	// MinWords is the minimum number of complete words a word-mode chunk
	// must contain before it can flush. Ignored in sentence/phrase mode.
	// Default: 5.
	MinWords int

	// MaxWords is the safety-valve word count in sentence/phrase mode. If
	// the buffer hits this count without seeing a terminator, the chunker
	// flushes at the next whitespace instead of stalling. Default: 30.
	// Set to 0 to disable the safety valve (chunker will wait forever).
	MaxWords int

	// Language selects the abbreviation catalog. Accepted keys are the
	// five Soulkyn audio locales exactly — "en-US", "es-ES", "fr-FR",
	// "pt-PT", "de-DE" — plus the legacy short aliases "en" and "fr".
	// Anything else falls back to English. Default: "en".
	Language string

	// MinChunkWords is the sentence/phrase-mode RUNT GATE: a chunk that
	// reaches a terminator with fewer than this many complete words does
	// NOT flush — it stays buffered and merges into the following
	// sentence, so "haha..." + "You should have seen it." emit as ONE
	// chunk. Built for TTS: a lone interjection ("haha...", "heh...",
	// "Ah.") synthesized as its own utterance gets full sentence prosody
	// and sounds wrong; grouped with its neighbor it reads naturally.
	// Consecutive runts accumulate until they jointly clear the bar.
	//
	// The MaxWords safety valve still applies. An END-OF-STREAM runt
	// still ships via Flush — there is no following sentence to merge
	// into, and merging BACKWARD would mean delaying every chunk by one
	// sentence, which a low-latency pipeline can't afford.
	//
	// 0 (default) disables the gate — pre-v0.0.2 behavior, byte-exact.
	// Ignored in ModeWord (MinWords governs there).
	MinChunkWords int
}

Config configures a Chunker. Zero values are filled with sensible defaults (see New).

type Mode

type Mode int

Mode controls how Chunker decides when a buffered chunk is ready.

const (
	// ModeWord emits at a whitespace boundary once MinWords complete words
	// have accumulated. Punctuation is carried along but never triggers a
	// flush on its own.
	ModeWord Mode = iota

	// ModeSentence emits at a sentence-ending terminator (. ! ? and unicode
	// equivalents) followed by whitespace, after passing abbreviation and
	// decimal filters. A safety valve flushes at the next whitespace once
	// MaxWords is exceeded so very long terminator-less runs never stall.
	ModeSentence

	// ModePhrase emits at any phrase terminator (. ! ? , ; : and unicode
	// equivalents) followed by whitespace. Same abbreviation, decimal, and
	// safety-valve rules as ModeSentence.
	ModePhrase
)