ds4

README

ds4go

ds4go is a zero-CGO Go wrapper for the ds4 inference engine. Applications using ds4go loads a pre-built libds4 shared library at runtime with github.com/ebitengine/purego. The shared library owns hardware acceleration. Use a Metal, CUDA, or CPU build of ds4 that matches your machine and model.

ds4 itself is an inference engine focused on the DeepSeek v4 Flash model targeting machines with 96G or more of GPU-accessible RAM.

We try to maintain parity with the upstream ds4 library, wrapping its C API. We build slightly-opinionated tools to facilitate using ds4.

Motivation

C is a wonderful language for low-level, high-performance, portable code; a clean C API can be wrapped and used by other laguages. Golang is a wonderful language for systems and tools development, and generally more friendly for developers, esepecially when creating networked applications. LLMs are great at programming both. We take the high-performance C engine of ds4 and allow Golang to directly utilize it, simplifying local LLM application development.

Install

Install the ds4go CLI with the quick-install script, Homebrew, or the Go toolchain:

# Quick install script (Linux/macOS)
curl -fsSL https://nimblemarkets.github.io/ds4go/install.sh | sh

# Homebrew (macOS/Linux)
brew install --cask nimblemarkets/tap/ds4go

# or with the Go toolchain
go install github.com/NimbleMarkets/ds4go/cmd/ds4go@latest

To use ds4go as a library:

go get github.com/NimbleMarkets/ds4go

Once the CLI is installed, fetch a prebuilt native libds4 from GitHub Releases:

ds4go install --backend auto

The installer downloads from github.com/NimbleMarkets/ds4 by default. Use --repo, --version, --backend, or --url to select a fork, release, build, or direct archive. It installs into $DS4_DIR/lib, defaulting to ~/.ds4/lib. --backend auto selects metal on macOS arm64, cuda on Linux, and cpu elsewhere. If the library is already installed and up-to-date, the installer exits successfully without re-downloading. If a different version is present, it will prompt to replace it (or require --force in non-interactive environments).

DS4_DIR is the ds4 home directory used by ds4go tooling:

$DS4_DIR/lib/      native shared libraries
$DS4_DIR/models/   GGUF model files

Manage curated DeepSeek V4 Flash models with:

ds4go model list
ds4go model download q2-imatrix
ds4go model set q2-imatrix

The default model path for commands and examples is $DS4_DIR/models/ds4flash.gguf.

Place the shared library in ~/.ds4/lib/, $DS4_DIR/lib/, next to your executable, or in a lib/ directory next to your executable. You can also point at it explicitly. The current working directory and the repository root are not searched, to avoid loading a planted library:

export DS4_LIB=/absolute/path/to/libds4.dylib
# or
export DS4_DIR=/opt/ds4

Platform defaults are:

Platform	Library
macOS	libds4.dylib
Linux	libds4.so
Windows	libds4.dll

Usage

import ds4 "github.com/NimbleMarkets/ds4go"

engine, err := ds4.NewEngine(ds4.EngineOptions{
    ModelPath: "/models/ds4flash.gguf",
    Backend:   ds4.BackendMetal,
})
if err != nil {
    panic(err)
}
defer engine.Close()

session, err := engine.NewSession(32768)
if err != nil {
    panic(err)
}
defer session.Close()

prompt, err := engine.EncodeChatPrompt("", "Explain Redis streams briefly.", ds4.ThinkHigh)
if err != nil {
    panic(err)
}
defer prompt.Free()

_, err = ds4.Generator{Engine: engine, Session: session}.GenerateTokens(prompt, ds4.GenerateOptions{
    MaxTokens: 128,
    StopOnEOS: true,
    OnToken: func(token int) {
        text, _ := engine.TokenText(token)
        fmt.Print(text)
    },
})

CLI

go run ./cmd/ds4go prompt --model ./ds4flash.gguf -p "Explain Redis streams in one paragraph."
go run ./cmd/ds4go prompt --model ./ds4flash.gguf

cmd/ds4go prompt and the examples accept the same arguments as the upstream ds4 C programs, parsed with pflag so options take the --option form. cmd/ds4go prompt, examples/simple, and examples/chat mirror the ds4 CLI (ds4_cli.c); examples/openai-compatible mirrors ds4-server (ds4_server.c). Run any of them with --help for the full list.

The only addition with no C equivalent is --lib, which points at the libds4 shared library the pure-Go wrapper loads at runtime. When empty, ds4go searches DS4_LIB, $DS4_DIR/lib (or ~/.ds4/lib), executable-local paths, and then the platform loader path.

$ ds4go help cheat
ds4go — command cheat sheet

  ├── completion      Generate the autocompletion script for the specified shell
  │   ├── bash        Generate the autocompletion script for bash
  │   ├── fish        Generate the autocompletion script for fish
  │   ├── powershell  Generate the autocompletion script for powershell
  │   └── zsh         Generate the autocompletion script for zsh
  │
  ├── install  Download a prebuilt libds4 shared library
  │
  ├── model         Browse, download, and manage curated ds4 models
  │   ├── delete    Delete a downloaded model from disk
  │   ├── download  Download a curated model from Hugging Face
  │   ├── info      Show details for a curated model
  │   ├── list      List installed and available models
  │   └── set       Set the default chat model
  │
  ├── prompt  Run prompt or interactive chat inference
  │
  ├── status  Find processes holding or using the libds4 shared library
  │
  ├── uninstall  Uninstall the installed libds4 shared library
  │
  ├── validate  Validate the installed libds4 shared library
  │
  └── web         Test browser-backed web tools
      ├── search  Execute Google search and print Markdown links
      └── visit   Visit a web page and print extracted Markdown

Run 'ds4go help <command>' for detailed usage.

Examples

go run ./examples/simple --model ./ds4flash.gguf
go run ./examples/chat --model ./ds4flash.gguf
go run ./examples/toolloop --mock
go run ./examples/toolloop --model ./ds4flash.gguf --nothink --tokens 512
go run ./examples/openai-compatible --model ./ds4flash.gguf --host 127.0.0.1 --port 8000

The toolloop example registers a Go add tool and exercises DSML tool-call parsing, tool dispatch, tool-result rendering, and exact replay. Use --mock for a no-model smoke test. The OpenAI-compatible example exposes POST /v1/chat/completions for a minimal local test server.

API Coverage

Most users should import the root package ds4 from github.com/NimbleMarkets/ds4go. It provides Go-native runtime policy and convenience helpers on top of the raw API. This includes DetectDefaultBackend(libPath), which queries backend preferences from installation metadata (ds4go-install.json) or falls back to system checks (probes for /dev/nvidia0 or nvidia-smi on Linux to select CUDA; defaults to Metal on macOS arm64, and CPU reference otherwise).

The strict binding layer lives in package ds4api, imported as github.com/NimbleMarkets/ds4go/ds4api. It mirrors the public ds4.h API: engines, sessions, token vectors, chat prompt rendering, tokenization, logprob helpers, MTP metadata, directional steering options, snapshot/payload save-load, and DS4 context-memory helpers. APIs that take FILE * use the package's opaque ds4api.File wrapper around a C FILE*.

ds4_log is exposed as LogString, which safely calls it with a fixed "%s" format. Arbitrary C varargs are intentionally not surfaced as a Go variadic API. SetStderr/SetStderrFd redirect libds4's diagnostic stream to a file or descriptor (see below). SetAbortFunc exposes libds4's fatal-invariant hook, which fires immediately before libds4 aborts the process.

Native stderr

libds4 writes its diagnostics — including Metal/CUDA backend messages — to its own stderr stream. ds4go redirects that stream to a file or descriptor with SetStderr, SetStderrFd, and DiscardLogs:

f, _ := os.Create("ds4.log")
err := ds4.SetStderr(f)   // redirect libds4 diagnostics to f
err = ds4.DiscardLogs()   // or send them to the null device
err = ds4.SetStderr(nil)  // restore the native stderr

libds4 dups the descriptor internally and writes unbuffered, so you may close your file once it is no longer the active target. The redirect target is process-global inside libds4, not per engine, so install it once during startup, before NewEngine. It is targeted at libds4's own output — not a process-wide dup2 — so anything other libraries write directly to file descriptor 2 is unaffected. Diagnostics are redirected as plain text; libds4 uses log levels only to colorize TTY output, so no per-message level is surfaced to Go.

To capture diagnostics into an in-process io.Writer — a TUI log overlay, a ring buffer, or an slog adapter — use CaptureStderr, which bridges the descriptor redirect to a writer with an internal pipe and pump goroutine:

cap, err := ds4.CaptureStderr(myWriter) // libds4 diagnostics stream into myWriter
defer cap.Close()                       // restore native stderr and drain on exit

Redirection is not supported on Windows: os.File.Fd returns a Win32 HANDLE, which libds4's CRT-based ds4_set_stderr_fd cannot accept, so these calls return ErrStderrUnsupportedOnWindows there.

For CLI use, you can also redirect stderr with your shell:

ds4go prompt ... 2>ds4.log
ds4go prompt ... 2>/dev/null

Fatal abort hook

Recent libds4 builds expose ds4_abort_set, and ds4go wraps it as SetAbortFunc. This is a last-chance fatal-invariant hook: libds4 calls it after logging the fatal message at LogError and immediately before native abort().

err := ds4.SetAbortFunc(func(msg string) {
    crashReporter.Record("libds4 fatal invariant", msg)
})

Returning from the callback does not recover the engine. The native library still calls abort() because the invariant is already broken. Use the hook for crash telemetry, flushing logs, or deliberate process termination. Do not call back into ds4go/libds4 from the callback; it can run from native worker threads while an FFI call is active.

Signal Safety

Do not use signal.NotifyContext around C FFI calls. SIGINT (Ctrl+C) can be delivered to any OS thread, including C worker threads inside libds4 (Metal, CUDA, or CPU). When that happens the C runtime aborts and the process segfaults.

Safe cancellation is programmatic only — pass a context.Context to GenerateOptions.Context and cancel it from Go code. The generator checks ctx.Done() between tokens, so cancellation never interrupts an active FFI call:

ctx, cancel := context.WithTimeout(context.Background(), 30*time.Second)
defer cancel()

_, err = ds4.Generator{Engine: engine, Session: session}.GenerateTokens(prompt, ds4.GenerateOptions{
    MaxTokens: 128,
    Context:   ctx,
    OnToken: func(token int) {
        text, _ := engine.TokenText(token)
        fmt.Print(text)
    },
})

This is exactly how examples/openai-compatible handles client disconnects — it wires r.Context() into generation so the engine stops cleanly when the HTTP connection drops.

Notes

Bindings are generated by hand against the public ds4 header at https://github.com/antirez/ds4/blob/main/ds4.h.

Inference runs in-process. The Golang wrapper adds FFI calls but does not proxy tokens through a server or copy model weights. Prefill, generation, Metal/CUDA/CPU execution, MTP, KV reuse, and disk KV payload serialization are all handled by the loaded ds4 shared library.

Open Collaboration

We welcome contributions and feedback. Please adhere to our Code of Conduct when engaging our community.

• GitHub Issues
• GitHub Pull Requests

Acknowledgements

Thanks to @antirez for his work on ds4 and for his local-LLM advocacy. Thanks to DeepSeek for their public contributions.

License

Released under the MIT License, see LICENSE.txt.

Copyright (c) 2026 Neomantra Corp.

---

Made with ❤ and 🔥 by the team behind Nimble.Markets.

Documentation

Overview

Package ds4 provides Go-native conveniences for the ds4 inference engine.

The lower-level github.com/NimbleMarkets/ds4go/ds4api package is the strict purego wrapper around ds4.h. This package owns runtime policy such as default paths, friendly diagnostics, and small convenience entry points.

Index

• Constants
• Variables
• func ApplyMTPDefaults(opts *EngineOptions)
• func DefaultDir() string
• func DefaultLibraryDir() string
• func DefaultLibraryPath() string
• func DefaultMTPPath() string
• func DefaultModelPath() string
• func DefaultModelsDir() string
• func DiscardLogs() error
• func EngineHolders(modelsDir string) (map[int][]string, error)
• func EnrichEngineOpenError(err error) error
• func Load(path string) (*ds4api.Library, error)
• func NewEngine(opts ds4api.EngineOptions) (*ds4api.Engine, error)
• func SetAbortFunc(fn AbortFunc) error
• func SetDefaultLibrary(lib *ds4api.Library)
• func SetStderr(f *os.File) error
• func SetStderrFd(fd int) error
• type AbortFunc
• type ArgmaxGenerateOptions
• type Backend
  • func DetectDefaultBackend(libPath string) Backend
• type ChatMessage
• type ContextMemory
• type Engine
• type EngineOptions
• type GenerateOptions
• type GenerationDoneFunc
• type Generator
  • func (g Generator) Continue(opts GenerateOptions) ([]int, error)
  • func (g Generator) Generate(prompt []int, opts GenerateOptions) ([]int, error)
  • func (g Generator) GenerateString(prompt string, opts GenerateOptions) (string, error)
  • func (g Generator) GenerateTokens(prompt *ds4api.Tokens, opts GenerateOptions) ([]int, error)
• type Library
• type LogType
• type ProcessInfo
  • func LibraryHolders(libPath string) ([]ProcessInfo, error)
• type ProgressFunc
• type Session
• type SessionRewriteResult
• type StderrCapture
  • func CaptureStderr(dst io.Writer) (*StderrCapture, error)
  • func (c *StderrCapture) Close() error
• type ThinkMode
• type TokenEmitFunc
• type TokenScore
• type Tokens
  • func BuildChatPrompt(engine *Engine, system string, tools []dsml.Tool, history []ChatMessage, ...) (*Tokens, error)
• type Tool
  • func (t Tool) Invoke(ctx context.Context, args json.RawMessage) (string, error)
  • func (t Tool) Schema() ToolSchema
• type ToolCall
• type ToolFunc
• type ToolHandler
• type ToolLoop
  • func (l ToolLoop) Run(opts ToolLoopOptions) (ToolLoopResult, error)
• type ToolLoopOptions
• type ToolLoopResult
• type ToolRegistry
  • func NewToolRegistry() *ToolRegistry
  • func (r *ToolRegistry) BuildPrompt(engine *Engine, system string, history []ChatMessage, think ThinkMode) (*Tokens, error)
  • func (r *ToolRegistry) ExecuteToolCalls(ctx context.Context, calls []ToolCall) ([]ChatMessage, error)
  • func (r *ToolRegistry) MustRegister(handler ToolHandler)
  • func (r *ToolRegistry) ParseAssistant(text string, thinking bool) (ChatMessage, error)
  • func (r *ToolRegistry) Register(handler ToolHandler) error
  • func (r *ToolRegistry) RegisterFunc(schema ToolSchema, fn ToolFunc) error
  • func (r *ToolRegistry) RenderToolsSection() (string, error)
  • func (r *ToolRegistry) ReplayStore() *dsml.ReplayStore
  • func (r *ToolRegistry) Schemas() []ToolSchema
  • func (r *ToolRegistry) SetReplayStore(store *dsml.ReplayStore)
• type ToolSchema

Constants

const (
	// DefaultTemperature is ds4's default sampling temperature.
	DefaultTemperature = ds4api.DefaultTemperature
	// DefaultTopP is ds4's default nucleus sampling probability.
	DefaultTopP = ds4api.DefaultTopP
	// DefaultMinP is ds4's default minimum relative-probability filter.
	DefaultMinP = ds4api.DefaultMinP

	// BackendMetal selects the Metal backend.
	BackendMetal = ds4api.BackendMetal
	// BackendCUDA selects the CUDA backend.
	BackendCUDA = ds4api.BackendCUDA
	// BackendCPU selects the CPU reference backend.
	BackendCPU = ds4api.BackendCPU

	// ThinkNone disables thinking markers in chat prompts.
	ThinkNone = ds4api.ThinkNone
	// ThinkHigh enables ordinary high-effort thinking.
	ThinkHigh = ds4api.ThinkHigh
	// ThinkMax requests maximum-effort thinking. ds4 may downgrade it to
	// ThinkHigh when the context is below ThinkMaxMinContext.
	ThinkMax = ds4api.ThinkMax

	// SessionRewriteError means the rewrite failed.
	SessionRewriteError = ds4api.SessionRewriteError
	// SessionRewriteOK means the rewrite completed in place.
	SessionRewriteOK = ds4api.SessionRewriteOK
	// SessionRewriteRebuildNeeded means the caller should restore or rebuild
	// the session state.
	SessionRewriteRebuildNeeded = ds4api.SessionRewriteRebuildNeeded

	// LogDefault is the default ds4 log style.
	LogDefault = ds4api.LogDefault
	// LogPrefill marks prefill messages.
	LogPrefill = ds4api.LogPrefill
	// LogGeneration marks generation messages.
	LogGeneration = ds4api.LogGeneration
	// LogKVCache marks KV-cache messages.
	LogKVCache = ds4api.LogKVCache
	// LogTool marks tool-calling messages.
	LogTool = ds4api.LogTool
	// LogWarning marks warnings.
	LogWarning = ds4api.LogWarning
	// LogTiming marks timing messages.
	LogTiming = ds4api.LogTiming
	// LogOK marks successful status messages.
	LogOK = ds4api.LogOK
	// LogError marks errors.
	LogError = ds4api.LogError

	// DefaultMTPDraftTokens is the default number of draft tokens speculative
	// decoding generates per step when MTP is enabled. A value of 0 disables
	// speculative decoding; set it explicitly to enable MTP.
	DefaultMTPDraftTokens = 0
	// DefaultMTPMargin is the default minimum margin (in tokens) between the
	// draft model's accepted sequence and the full target model output.
	DefaultMTPMargin = 3
)

Variables

var ErrContextFull = errors.New("ds4go: session context full")

ErrContextFull is returned when a session has no room left in its context window. When generation is capped by the remaining room, it is returned alongside the tokens produced before the limit was reached.

Functions

func ApplyMTPDefaults

func ApplyMTPDefaults(opts *EngineOptions)

ApplyMTPDefaults populates MTPPath, MTPDraftTokens, and MTPMargin with sensible defaults when an MTP model is installed. It only fills fields that are currently empty or zero, so explicit caller settings are respected.

func DefaultDir

func DefaultDir() string

DefaultDir returns the ds4go data directory.

DS4_DIR overrides the default. When DS4_DIR is unset, DefaultDir returns "$HOME/.ds4" when the user home directory can be determined, otherwise ".ds4".

func DefaultLibraryDir

func DefaultLibraryDir() string

DefaultLibraryDir returns the directory where libds4 is installed by default: the "lib" subdirectory of DefaultDir.

func DefaultLibraryPath

func DefaultLibraryPath() string

DefaultLibraryPath returns the preferred libds4 shared-library path.

Search order is DS4_LIB, DS4_DIR/lib, executable-local paths, and finally the platform library name for system loader lookup.

The current working directory is deliberately NOT searched: loading a shared library from the CWD would let an attacker who can write a file into a directory the user happens to run ds4go from plant a malicious libds4 and gain code execution (binary planting). Use DS4_LIB or DS4_DIR to load a library from a non-default location.

func DefaultMTPPath

func DefaultMTPPath() string

DefaultMTPPath returns the path to the installed MTP companion model, or empty string if it is not present.

func DefaultModelPath

func DefaultModelPath() string

DefaultModelPath returns the path to the default model symlink.

The default model is a symlink at $DS4_DIR/models/<DefaultModelSymlink> that points to the active downloaded model. Use ds4go model set to switch it.

func DefaultModelsDir

func DefaultModelsDir() string

DefaultModelsDir returns the directory where downloaded models are stored: the "models" subdirectory of DefaultDir.

func DiscardLogs

func DiscardLogs() error

DiscardLogs redirects libds4's diagnostic output to the null device for the default library. The native stderr is restored by SetStderr(nil).

Not supported on Windows; see SetStderrFd.

func EngineHolders

func EngineHolders(modelsDir string) (map[int][]string, error)

EngineHolders returns a map of process PIDs to the list of model files they are currently running. If modelsDir is empty, it uses the default models directory. To ensure security, queries are restricted to paths within authorized directories (DefaultDir or executable directory).

func EnrichEngineOpenError

func EnrichEngineOpenError(err error) error

EnrichEngineOpenError adds process names to ds4 engine-open errors that mention lock-holder PIDs.

func Load

func Load(path string) (*ds4api.Library, error)

Load loads libds4 using ds4go's runtime path policy.

Passing an empty path searches DS4_LIB, DS4_DIR/lib, executable-local library locations, and finally the platform loader path. The current working directory is not searched; see DefaultLibraryPath.

func NewEngine

func NewEngine(opts ds4api.EngineOptions) (*ds4api.Engine, error)

NewEngine loads the default libds4 shared library and opens a ds4 engine.

func SetAbortFunc

func SetAbortFunc(fn AbortFunc) error

SetAbortFunc installs a last-chance libds4 fatal-invariant callback.

libds4 invokes the callback after logging the fatal message and immediately before native abort(). Passing nil restores the default behavior. This hook is process-global inside libds4 and is intended for crash telemetry, flushing logs, or deliberate process termination; returning from the callback does not recover the engine.

func SetDefaultLibrary

func SetDefaultLibrary(lib *ds4api.Library)

SetDefaultLibrary makes lib the low-level package default library.

func SetStderr

func SetStderr(f *os.File) error

SetStderr redirects libds4's diagnostic output to f for the default library. Passing nil restores the native stderr.

libds4 dups the descriptor internally and writes its diagnostics there unbuffered, so f may be closed once it is no longer the active target. The redirect target is process-global inside libds4; install it once at startup, before generation is active. Calling Fd on f detaches it from the Go runtime poller and puts it in blocking mode, so do not pass a file you also use for asynchronous I/O.

Not supported on Windows; see SetStderrFd.

func SetStderrFd

func SetStderrFd(fd int) error

SetStderrFd redirects libds4's diagnostic output to the file descriptor fd for the default library. Pass -1 to restore the native stderr.

libds4 dups fd internally and writes its diagnostics there unbuffered, so the caller may close its own descriptor after this call. The redirect target is process-global inside libds4; install it once at startup, before generation.

Types

type AbortFunc

type AbortFunc = ds4api.AbortFunc

AbortFunc receives a libds4 fatal-invariant message immediately before abort.

type ArgmaxGenerateOptions

type ArgmaxGenerateOptions = ds4api.ArgmaxGenerateOptions

ArgmaxGenerateOptions controls ds4_engine_generate_argmax.

type Backend

type Backend = ds4api.Backend

Backend selects the accelerator implementation compiled into libds4.

func DetectDefaultBackend

func DetectDefaultBackend(libPath string) Backend

DetectDefaultBackend probes the environment and installation metadata to determine the preferred backend for the shared library at libPath.

Passing an empty string probes using the default library path.

type ChatMessage

type ChatMessage struct {
	// Role is "system", "user", "assistant", or "tool".
	Role string
	// Content is the plain text content for the message.
	Content string
	// ReasoningContent is the assistant reasoning block when thinking mode is enabled.
	ReasoningContent string
	// ToolCalls is the assistant's requested tool calls for this turn.
	ToolCalls []ToolCall
	// ToolCallID associates a tool result message with the call it answers.
	// DSML does not render this ID into <tool_result>; prompt builders expect
	// tool result messages to be ordered to match the assistant's ToolCalls.
	ToolCallID string
}

ChatMessage is one tool-aware chat turn.

type ContextMemory

type ContextMemory = ds4api.ContextMemory

ContextMemory is ds4_context_memory.

type Engine

type Engine = ds4api.Engine

Engine wraps a ds4_engine.

type EngineOptions

type EngineOptions = ds4api.EngineOptions

EngineOptions configures ds4_engine_open.

type GenerateOptions

type GenerateOptions struct {
	// MaxTokens is the maximum number of tokens to generate.
	MaxTokens int
	// Temperature controls sampling. Values <= 0 use argmax.
	Temperature float32
	// TopK limits sampling to the best k tokens when Temperature > 0.
	TopK int
	// TopP applies nucleus sampling when Temperature > 0.
	TopP float32
	// MinP applies minimum probability sampling when Temperature > 0.
	MinP float32
	// Seed seeds ds4's sampler. A zero seed is valid and deterministic.
	Seed uint64
	// StopOnEOS stops generation when ds4 emits the engine EOS token.
	StopOnEOS bool
	// ExcludeToken asks argmax generation to skip a specific token id.
	ExcludeToken int
	// OnToken streams generated tokens. Returning normally continues generation.
	OnToken ds4api.TokenEmitFunc
	// Context, when non-nil, can be cancelled to stop generation gracefully
	// before the next token is sampled.
	Context context.Context
}

GenerateOptions controls Go-native session generation helpers.

type GenerationDoneFunc

type GenerationDoneFunc = ds4api.GenerationDoneFunc

GenerationDoneFunc is called after ds4 completes generation.

type Generator

type Generator struct {
	Engine  *ds4api.Engine
	Session *ds4api.Session
}

Generator binds a ds4 engine and session for Go-native generation helpers.

func (Generator) Continue

func (g Generator) Continue(opts GenerateOptions) ([]int, error)

Continue generates tokens from the current session logits.

func (Generator) Generate

func (g Generator) Generate(prompt []int, opts GenerateOptions) ([]int, error)

Generate synchronizes to prompt and generates tokens from the session.

func (Generator) GenerateString

func (g Generator) GenerateString(prompt string, opts GenerateOptions) (string, error)

GenerateString tokenizes prompt, generates, and decodes the generated text.

func (Generator) GenerateTokens

func (g Generator) GenerateTokens(prompt *ds4api.Tokens, opts GenerateOptions) ([]int, error)

GenerateTokens synchronizes to prompt and generates tokens from the session.

type Library

type Library = ds4api.Library

Library is a loaded libds4 shared library.

type LogType

type LogType = ds4api.LogType

LogType is the category used by libds4 diagnostics.

type ProcessInfo

type ProcessInfo struct {
	PID  int
	Name string
}

ProcessInfo represents a process holding or using a ds4 resource.

func LibraryHolders

func LibraryHolders(libPath string) ([]ProcessInfo, error)

LibraryHolders returns a list of processes currently holding onto the libds4 shared library. If libPath is empty, it uses the default library path. To ensure security, queries are restricted to paths within authorized directories (DefaultDir, executable directory, or explicit DS4_LIB).

type ProgressFunc

type ProgressFunc = ds4api.ProgressFunc

ProgressFunc receives ds4 progress events.

type Session

type Session = ds4api.Session

Session wraps a ds4_session.

type SessionRewriteResult

type SessionRewriteResult = ds4api.SessionRewriteResult

SessionRewriteResult is returned by session rewrite helpers.

type StderrCapture

type StderrCapture struct {
	// contains filtered or unexported fields
}

StderrCapture pumps libds4's redirected diagnostic stream into an io.Writer until Close restores the native stderr. It is created by CaptureStderr.

func CaptureStderr

func CaptureStderr(dst io.Writer) (*StderrCapture, error)

CaptureStderr redirects libds4's diagnostic output into dst and returns a handle that restores the native stderr when closed.

dst receives the raw bytes libds4 writes — line splitting and any leveling are the caller's concern. This is the io.Writer counterpart to SetStderr, implemented with an os.Pipe and a background pump; use SetStderr directly when the sink is already a file or the null device. The redirect target is process-global inside libds4, so only one capture (or SetStderr target) is active at a time; install it once during startup, before generation.

Not supported on Windows; see SetStderrFd. On failure no redirect is installed and the pipe is released.

func (*StderrCapture) Close

func (c *StderrCapture) Close() error

Close restores the native stderr and waits for the pump to drain. Diagnostics libds4 already wrote are flushed to dst before Close returns. Close is idempotent only in the sense that the underlying files tolerate a double close; call it exactly once per CaptureStderr.

type ThinkMode

type ThinkMode = ds4api.ThinkMode

ThinkMode controls ds4's rendered chat thinking mode.

type TokenEmitFunc

type TokenEmitFunc = ds4api.TokenEmitFunc

TokenEmitFunc is called when ds4 emits a generated token.

type TokenScore

type TokenScore = ds4api.TokenScore

TokenScore is ds4_token_score.

type Tokens

type Tokens = ds4api.Tokens

Tokens owns a ds4_tokens value allocated by libds4.

func BuildChatPrompt

func BuildChatPrompt(engine *Engine, system string, tools []dsml.Tool, history []ChatMessage, think ThinkMode) (*Tokens, error)

BuildChatPrompt renders a tool-aware chat prompt using ds4's chat helpers.

The tools section is prepended to the system message. If system is empty and tools is non-empty, BuildChatPrompt creates a system turn containing only the rendered tools section. Tool result messages are rendered as user turns containing DSML <tool_result> blocks.

type Tool

type Tool struct {
	ToolSchema
	Handler ToolFunc
}

Tool binds a schema to a Go function.

func (Tool) Invoke

func (t Tool) Invoke(ctx context.Context, args json.RawMessage) (string, error)

Invoke executes the bound handler.

func (Tool) Schema

func (t Tool) Schema() ToolSchema

Schema returns the tool schema.

type ToolCall

type ToolCall struct {
	// ID is the stable tool-call identifier used for exact replay.
	ID string
	// Name is the called tool name.
	Name string
	// Arguments is the JSON argument object string.
	Arguments string
}

ToolCall is one tool request emitted by the assistant.

type ToolFunc

type ToolFunc func(ctx context.Context, args json.RawMessage) (string, error)

ToolFunc is a function-backed tool implementation.

type ToolHandler

type ToolHandler interface {
	// Schema returns the public tool schema shown to the model.
	Schema() ToolSchema
	// Invoke executes the tool with the JSON arguments requested by the model.
	Invoke(ctx context.Context, args json.RawMessage) (string, error)
}

ToolHandler exposes a Go tool to the model.

type ToolLoop

type ToolLoop struct {
	// Engine owns token decoding and prompt rendering.
	Engine *Engine
	// Session is the live ds4 session used for generation.
	Session *Session
	// Tools stores the tool schemas, handlers, and replay state.
	Tools *ToolRegistry
	// ThinkMode controls ds4's assistant prefix rendering.
	ThinkMode ThinkMode
	// Thinking tells ParseAssistant whether to require and extract a reasoning block.
	Thinking bool
	// CompleteFunc overrides the default generator-backed completion path.
	// When nil, Run uses Generator.GenerateTokens and Engine.TokenText.
	CompleteFunc func(prompt *Tokens, opts GenerateOptions) (string, error)
}

ToolLoop drives multi-turn tool calling on top of Generator and ToolRegistry.

func (ToolLoop) Run

func (l ToolLoop) Run(opts ToolLoopOptions) (ToolLoopResult, error)

Run executes assistant generation, dispatches requested tools, and continues until the assistant returns a plain answer or MaxRounds is reached. A cancelled GenerateOptions.Context, a tool handler error, or a call to an unregistered tool aborts the run and is returned as an error.

type ToolLoopOptions

type ToolLoopOptions struct {
	// System is the system prompt content.
	System string
	// History is the existing chat transcript excluding the generated assistant turn.
	History []ChatMessage
	// Generate controls model generation for each assistant turn.
	Generate GenerateOptions
	// MaxRounds bounds the total number of assistant turns — tool-calling
	// rounds plus the final answer. Values <= 0 default to 8.
	MaxRounds int
}

ToolLoopOptions configures one tool loop run.

type ToolLoopResult

type ToolLoopResult struct {
	// History is the full updated transcript, including the final assistant turn.
	History []ChatMessage
	// Assistant is the final assistant message with no further tool calls.
	Assistant ChatMessage
	// ToolRounds is the number of assistant turns that requested tools.
	ToolRounds int
}

ToolLoopResult is the final result of one tool loop run.

type ToolRegistry

type ToolRegistry struct {
	// contains filtered or unexported fields
}

ToolRegistry stores Go-exposed tools and exact sampled DSML replay state.

func NewToolRegistry

func NewToolRegistry() *ToolRegistry

NewToolRegistry creates an empty tool registry with exact DSML replay enabled.

func (*ToolRegistry) BuildPrompt

func (r *ToolRegistry) BuildPrompt(engine *Engine, system string, history []ChatMessage, think ThinkMode) (*Tokens, error)

BuildPrompt renders a tool-aware chat prompt using ds4's chat helpers.

func (*ToolRegistry) ExecuteToolCalls

func (r *ToolRegistry) ExecuteToolCalls(ctx context.Context, calls []ToolCall) ([]ChatMessage, error)

ExecuteToolCalls invokes the registered Go handlers for the given tool calls. It stops and returns an error if ctx is cancelled, if a call names an unregistered tool, or if a handler returns an error.

func (*ToolRegistry) MustRegister

func (r *ToolRegistry) MustRegister(handler ToolHandler)

MustRegister adds one tool handler and panics on error.

func (*ToolRegistry) ParseAssistant

func (r *ToolRegistry) ParseAssistant(text string, thinking bool) (ChatMessage, error)

ParseAssistant parses one assistant completion and assigns stable tool-call IDs.

func (*ToolRegistry) Register

func (r *ToolRegistry) Register(handler ToolHandler) error

Register adds one tool handler to the registry.

func (*ToolRegistry) RegisterFunc

func (r *ToolRegistry) RegisterFunc(schema ToolSchema, fn ToolFunc) error

RegisterFunc adds one function-backed tool to the registry.

func (*ToolRegistry) RenderToolsSection

func (r *ToolRegistry) RenderToolsSection() (string, error)

RenderToolsSection renders the DSML tools section for the registered tools.

func (*ToolRegistry) ReplayStore

func (r *ToolRegistry) ReplayStore() *dsml.ReplayStore

ReplayStore returns the exact sampled DSML replay store.

func (*ToolRegistry) Schemas

func (r *ToolRegistry) Schemas() []ToolSchema

Schemas returns the registered tool schemas in registration order.

func (*ToolRegistry) SetReplayStore

func (r *ToolRegistry) SetReplayStore(store *dsml.ReplayStore)

SetReplayStore replaces the exact sampled DSML replay store. Passing nil disables replay.

type ToolSchema

type ToolSchema struct {
	// Name is the tool's callable name.
	Name string
	// Description explains what the tool does.
	Description string
	// Parameters is the JSON Schema object for the tool's arguments.
	Parameters json.RawMessage
}

ToolSchema describes one Go-exposed tool.