litertlmgo

README

litertlm-go

A purego-backed, cgo-free Go wrapper for Google's LiteRT-LM C API. Run local LLM inference from Go without a C toolchain — the C library is loaded dynamically at runtime via ebitengine/purego.

Inspired by Hybridgroup's Yzma.

📖 Full documentation: vladimirvivien.github.io/litertlm-go

Install

go get github.com/vladimirvivien/litertlm-go@latest

Quickstart

package main

import (
    "context"
    "fmt"
    "os"

    "github.com/vladimirvivien/litertlm-go/pkg/litertlm"
)

func main() {
    ctx := context.Background()
    client, err := litertlm.New(ctx,
        litertlm.WithLib(os.Getenv("LITERTLM_LIB")),
        litertlm.WithModel(os.Getenv("LITERTLM_MODEL")),
    )
    if err != nil {
        panic(err)
    }
    defer client.Close()

    text, err := client.Generate(ctx, "Write a haiku about the sea.")
    if err != nil {
        panic(err)
    }
    fmt.Println(text)
}

LITERTLM_LIB=/abs/path/to/dist/lib \
LITERTLM_MODEL=/abs/path/to/gemma-4-E2B-it.litertlm \
    go run main.go

Full walkthrough → docs/getting-started.

Building the C library

LiteRT-LM doesn't ship a prebuilt C API. Build it yourself:

• Linux / macOS — LITERTLM-BUILD.md
• Windows — LITERTLM-BUILD-WINDOWS.md

Examples

Path	What it shows
examples/hello/	Minimal Generate
examples/stream/	GenerateStream with range-over-func
examples/chat/	Multi-turn Chat with a system prompt
examples/conversation/	Chat + tools + structured tool_calls
examples/structured/	GenerateData[T] (typed JSON output via reflection + retries)
examples/cancel/	Cancelling a streaming generation via context.WithCancel
examples/prefill-decode/	Explicit two-phase generation (low-level)
examples/score/	ScoreTexts + Score / TokenLength (low-level)
examples/tokenize/	Engine.Tokenize / Detokenize + start/stop tokens (low-level)
examples/gpu/	GPU-backed generation + benchmark metrics

Documentation

The high-level API (recommended for most use cases) is documented at vladimirvivien.github.io/litertlm-go:

• Getting started
• Client — New, Generate, GenerateStream, GenerateResponse
• Chat — multi-turn, system prompts, tool calling
• Structured output — GenerateData[T]
• Low-level API — when to drop down
• Troubleshooting

The site is built with MkDocs Material and published from the docs/ directory on tagged pushes (see .github/workflows/docs.yml). Preview locally:

pip install -r docs/requirements.txt
mkdocs serve  # http://localhost:8000/

License

Apache-2.0, same as LiteRT-LM itself.

Documentation

Overview

Package litertlmgo is the module root for litertlm-go.

Consumers import the actual Go API from the subpackages:

import "github.com/vladimirvivien/litertlm-go/pkg/litertlm"
import "github.com/vladimirvivien/litertlm-go/pkg/loader"

See README.md for the full usage guide and LITERTLM-BUILD.md for the native-library build recipe.