ds4api

package
v0.2.2 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: May 18, 2026 License: MIT Imports: 13 Imported by: 0

Documentation

Overview

Package ds4api test infrastructure: a pure-Go mock of libds4.

NewMockLibrary creates a Library whose raw function pointers are Go implementations backed by in-memory state. This lets tests exercise the ds4go binding and generator layers without loading a real shared library.

Package ds4api provides pure-Go bindings for the ds4 inference engine.

The package does not use cgo. It loads a user-provided libds4 shared library at runtime through purego and wraps the public API from ds4.h.

Index

Constants

View Source 
const (
	// DefaultTemperature is ds4's default sampling temperature.
	DefaultTemperature float32 = 1.0
	// DefaultTopP is ds4's default nucleus sampling probability.
	DefaultTopP float32 = 1.0
	// DefaultMinP is ds4's default minimum relative-probability filter.
	DefaultMinP float32 = 0.05
)

Sampling defaults mirror the DS4_DEFAULT_* macros in ds4.h. The default sampler keeps top-p at 1.0 and uses min-p as the active filter.

Variables

This section is empty.

Functions

func BackendName 

func BackendName(backend Backend) string

BackendName returns ds4's printable name for backend.

func DumpTextTokenization 

func DumpTextTokenization(modelPath, text string, fp File) error

DumpTextTokenization calls ds4_dump_text_tokenization.

func LogIsTTY 

func LogIsTTY(fp File) bool

LogIsTTY calls ds4_log_is_tty for a C FILE*.

func LogString 

func LogString(fp File, typ LogType, msg string)

LogString writes a plain string through ds4_log using a "%s" format.

func RewriteRequiresRebuild 

func RewriteRequiresRebuild(liveLen, canonicalLen, common int) bool

RewriteRequiresRebuild calls ds4_session_rewrite_requires_rebuild.

func SetDefaultLibrary 

func SetDefaultLibrary(lib *Library)

SetDefaultLibrary makes lib the package default library.

func ThinkMaxMinContext 

func ThinkMaxMinContext() uint32

ThinkMaxMinContext returns the minimum context size ds4 recommends for ThinkMax.

func ThinkMaxPrefix 

func ThinkMaxPrefix() string

ThinkMaxPrefix returns ds4's maximum-effort thinking prompt prefix.

func ThinkModeEnabled 

func ThinkModeEnabled(mode ThinkMode) bool

ThinkModeEnabled reports whether mode emits thinking markers.

func ThinkModeName 

func ThinkModeName(mode ThinkMode) string

ThinkModeName returns ds4's printable name for mode.

Types

type ArgmaxGenerateOptions 

type ArgmaxGenerateOptions struct {
	// NPredict is the number of tokens to generate.
	NPredict int
	// CtxSize is the context size used for this generation.
	CtxSize int
	// OnToken streams generated tokens.
	OnToken TokenEmitFunc
	// OnDone is called by ds4 when generation is complete.
	OnDone GenerationDoneFunc
	// OnProgress receives ds4 progress events.
	OnProgress ProgressFunc
}

ArgmaxGenerateOptions controls ds4_engine_generate_argmax.

type Backend 

type Backend int32

Backend selects the accelerator implementation compiled into libds4. 

const (
	// BackendMetal selects the Metal backend.
	BackendMetal Backend = iota
	// BackendCUDA selects the CUDA backend.
	BackendCUDA
	// BackendCPU selects the CPU reference backend.
	BackendCPU
)

type ContextMemory 

type ContextMemory struct {
	// TotalBytes is the estimated total context memory.
	TotalBytes uint64
	// RawBytes is the raw KV-cache memory estimate.
	RawBytes uint64
	// CompressedBytes is the compressed KV-cache memory estimate.
	CompressedBytes uint64
	// ScratchBytes is the temporary scratch memory estimate.
	ScratchBytes uint64
	// PrefillCap is the prefill capacity.
	PrefillCap uint32
	// RawCap is the raw KV-cache row capacity.
	RawCap uint32
	// CompCap is the compressed KV-cache row capacity.
	CompCap uint32
}

ContextMemory is ds4_context_memory.

func ContextMemoryEstimate 

func ContextMemoryEstimate(backend Backend, ctxSize int) ContextMemory

ContextMemoryEstimate estimates ds4 context memory for a backend and context size.

type Engine 

type Engine struct {
	// contains filtered or unexported fields
}

Engine wraps a ds4_engine.

func NewEngine 

func NewEngine(opts EngineOptions) (*Engine, error)

NewEngine opens a ds4 engine using the default shared library.

func (*Engine) ChatAppendAssistantPrefix 

func (e *Engine) ChatAppendAssistantPrefix(tokens *Tokens, thinkMode ThinkMode) error

ChatAppendAssistantPrefix appends the assistant prefix for generation.

func (*Engine) ChatAppendMaxEffortPrefix 

func (e *Engine) ChatAppendMaxEffortPrefix(tokens *Tokens) error

ChatAppendMaxEffortPrefix appends ds4's maximum-effort thinking prefix.

func (*Engine) ChatAppendMessage 

func (e *Engine) ChatAppendMessage(tokens *Tokens, role, content string) error

ChatAppendMessage appends a rendered role/content chat message.

func (*Engine) ChatBegin 

func (e *Engine) ChatBegin(tokens *Tokens) error

ChatBegin appends ds4's chat preamble to tokens.

func (*Engine) Close 

func (e *Engine) Close()

Close releases the underlying ds4_engine.

func (*Engine) CollectIMatrix 

func (e *Engine) CollectIMatrix(datasetPath, outputPath string, ctxSize, maxPrompts, maxTokens int) error

CollectIMatrix calls ds4_engine_collect_imatrix.

func (*Engine) DumpTokens 

func (e *Engine) DumpTokens(tokens *Tokens) error

DumpTokens calls ds4_engine_dump_tokens.

func (*Engine) EncodeChatPrompt 

func (e *Engine) EncodeChatPrompt(system, prompt string, thinkMode ThinkMode) (*Tokens, error)

EncodeChatPrompt encodes a system and user prompt with ds4's chat template.

func (*Engine) FirstTokenTest 

func (e *Engine) FirstTokenTest(prompt *Tokens) error

FirstTokenTest calls ds4_engine_first_token_test.

func (*Engine) GenerateArgmax 

func (e *Engine) GenerateArgmax(prompt *Tokens, opts ArgmaxGenerateOptions) ([]int, error)

GenerateArgmax calls ds4_engine_generate_argmax.

func (*Engine) HasMTP 

func (e *Engine) HasMTP() bool

HasMTP reports whether this engine has an MTP draft model.

func (*Engine) HeadTest 

func (e *Engine) HeadTest(prompt *Tokens) error

HeadTest calls ds4_engine_head_test.

func (*Engine) MTPDraftTokens 

func (e *Engine) MTPDraftTokens() int

MTPDraftTokens returns the configured MTP draft length.

func (*Engine) MetalGraphFullTest 

func (e *Engine) MetalGraphFullTest(prompt *Tokens) error

MetalGraphFullTest calls ds4_engine_metal_graph_full_test.

func (*Engine) MetalGraphPromptTest 

func (e *Engine) MetalGraphPromptTest(prompt *Tokens, ctxSize int) error

MetalGraphPromptTest calls ds4_engine_metal_graph_prompt_test.

func (*Engine) MetalGraphTest 

func (e *Engine) MetalGraphTest(prompt *Tokens) error

MetalGraphTest calls ds4_engine_metal_graph_test.

func (*Engine) NewSession 

func (e *Engine) NewSession(ctxSize int) (*Session, error)

NewSession creates a ds4 session for this engine and context size.

func (*Engine) NewTokens 

func (e *Engine) NewTokens(ids []int) (*Tokens, error)

NewTokens creates a libds4-owned token vector associated with this engine's library.

func (*Engine) RoutedQuantBits 

func (e *Engine) RoutedQuantBits() int

RoutedQuantBits returns the routed expert quantization bits used by the engine.

func (*Engine) Summary 

func (e *Engine) Summary() error

Summary prints ds4's engine summary to its configured output.

func (*Engine) TokenAssistant 

func (e *Engine) TokenAssistant() int

TokenAssistant returns ds4's assistant-role token id.

func (*Engine) TokenEOS 

func (e *Engine) TokenEOS() int

TokenEOS returns ds4's end-of-sequence token id.

func (*Engine) TokenText 

func (e *Engine) TokenText(token int) (string, error)

TokenText decodes one token to text and frees the C allocation returned by ds4.

func (*Engine) TokenUser 

func (e *Engine) TokenUser() int

TokenUser returns ds4's user-role token id.

func (*Engine) TokenizeRenderedChat 

func (e *Engine) TokenizeRenderedChat(text string) (*Tokens, error)

TokenizeRenderedChat tokenizes a rendered chat prompt.

func (*Engine) TokenizeText 

func (e *Engine) TokenizeText(text string) (*Tokens, error)

TokenizeText tokenizes plain text with ds4_tokenize_text.

type EngineOptions 

type EngineOptions struct {
	// ModelPath is the path to the DeepSeek V4 Flash GGUF model.
	ModelPath string
	// MTPPath is the optional MTP draft model path.
	MTPPath string
	// Backend selects Metal, CUDA, or CPU according to the libds4 build.
	Backend Backend
	// NThreads controls CPU worker threads when the backend uses them.
	NThreads int
	// MTPDraftTokens controls speculative draft length.
	MTPDraftTokens int
	// MTPMargin controls speculative acceptance confidence.
	MTPMargin float32
	// DirectionalSteeringFile points at an optional directional steering file.
	DirectionalSteeringFile string
	// DirectionalSteeringAttn scales directional steering in attention blocks.
	DirectionalSteeringAttn float32
	// DirectionalSteeringFFN scales directional steering in FFN blocks.
	DirectionalSteeringFFN float32
	// WarmWeights asks ds4 to warm model weights after load.
	WarmWeights bool
	// Quality requests ds4's quality-oriented execution path where supported.
	Quality bool
}

EngineOptions configures ds4_engine_open.

type File 

type File uintptr

File is an opaque C FILE* used by ds4 APIs that accept FILE pointers.

func OpenFile 

func OpenFile(path, mode string) (File, error)

OpenFile opens a C FILE* with fopen for ds4 FILE*-based APIs.

func (File) Close 

func (f File) Close() error

Close closes a C FILE* opened by OpenFile.

type GenerationDoneFunc 

type GenerationDoneFunc func()

GenerationDoneFunc is called after ds4 completes generation.

type Library 

type Library struct {
	// contains filtered or unexported fields
}

Library is a loaded libds4 shared library.

func DefaultLibrary 

func DefaultLibrary() (*Library, error)

DefaultLibrary returns the lazily loaded default library.

func Load 

func Load(path string) (*Library, error)

Load loads libds4 from path and registers all ds4.h symbols.

Passing an empty path uses DS4_LIB, then searches common local library locations. Higher-level ds4go runtime path policy lives in the module root.

func NewMockLibrary added in v0.2.2 

func NewMockLibrary() *Library

NewMockLibrary returns a Library whose C symbols are backed by trivial in-memory state. The mock supports engine/session lifecycle, tokenization, deterministic generation, and optional MTP metadata.

func (*Library) NewEngine 

func (l *Library) NewEngine(opts EngineOptions) (*Engine, error)

NewEngine opens a ds4 engine using this shared library.

func (*Library) Path 

func (l *Library) Path() string

Path returns the filesystem path used to load this library.

type LogType 

type LogType int32

LogType is the category used by ds4_log. 

const (
	// LogDefault is the default ds4 log style.
	LogDefault LogType = iota
	// LogPrefill marks prefill messages.
	LogPrefill
	// LogGeneration marks generation messages.
	LogGeneration
	// LogKVCache marks KV-cache messages.
	LogKVCache
	// LogTool marks tool-calling messages.
	LogTool
	// LogWarning marks warnings.
	LogWarning
	// LogTiming marks timing messages.
	LogTiming
	// LogOK marks successful status messages.
	LogOK
	// LogError marks errors.
	LogError
)

type ProgressFunc 

type ProgressFunc func(event string, current, total int)

ProgressFunc receives ds4 progress events.

type Session 

type Session struct {
	// contains filtered or unexported fields
}

Session wraps a ds4_session.

func (*Session) Argmax 

func (s *Session) Argmax() int

Argmax returns the argmax token id for the current logits.

func (*Session) ArgmaxExcluding 

func (s *Session) ArgmaxExcluding(excludedID int) int

ArgmaxExcluding returns the argmax token id excluding one token.

func (*Session) Close 

func (s *Session) Close()

Close releases the underlying ds4_session.

func (*Session) CommonPrefix 

func (s *Session) CommonPrefix(prompt *Tokens) int

CommonPrefix returns the common prefix length between the live session and prompt.

func (*Session) Ctx 

func (s *Session) Ctx() int

Ctx returns the session context size.

func (*Session) Eval 

func (s *Session) Eval(token int) error

Eval evaluates one token and advances the session.

func (*Session) EvalSpeculativeArgmax 

func (s *Session) EvalSpeculativeArgmax(firstToken, maxTokens, eosToken int) ([]int, error)

EvalSpeculativeArgmax calls ds4_session_eval_speculative_argmax.

func (*Session) Invalidate 

func (s *Session) Invalidate()

Invalidate invalidates the live session state.

func (*Session) LoadPayload 

func (s *Session) LoadPayload(fp File, payloadBytes uint64) error

LoadPayload reads a DS4-specific session payload from fp.

func (*Session) LoadPayloadFile 

func (s *Session) LoadPayloadFile(path string, payloadBytes uint64) error

LoadPayloadFile reads a DS4-specific session payload from path.

func (*Session) LoadSnapshot 

func (s *Session) LoadSnapshot(data []byte) error

LoadSnapshot restores a session snapshot previously returned by SaveSnapshot.

func (*Session) PayloadBytes 

func (s *Session) PayloadBytes() uint64

PayloadBytes returns ds4_session_payload_bytes.

func (*Session) Pos 

func (s *Session) Pos() int

Pos returns the current session token position.

func (*Session) Rewind 

func (s *Session) Rewind(pos int)

Rewind rewinds the session to token position pos.

func (*Session) RewriteFromCommon 

func (s *Session) RewriteFromCommon(prompt *Tokens, common int) (SessionRewriteResult, error)

RewriteFromCommon rewrites a session from a known common prefix length.

func (*Session) Sample 

func (s *Session) Sample(temperature float32, topK int, topP, minP float32, rng *uint64) int

Sample samples the next token from current logits.

func (*Session) SavePayload 

func (s *Session) SavePayload(fp File) error

SavePayload writes the DS4-specific session payload to fp.

func (*Session) SavePayloadFile 

func (s *Session) SavePayloadFile(path string) error

SavePayloadFile writes the DS4-specific session payload to path.

func (*Session) SaveSnapshot 

func (s *Session) SaveSnapshot() ([]byte, error)

SaveSnapshot serializes a session snapshot to a Go byte slice.

func (*Session) SetProgress 

func (s *Session) SetProgress(fn ProgressFunc) error

SetProgress sets a persistent progress callback for ds4_session_set_progress.

func (*Session) Sync 

func (s *Session) Sync(prompt []int) error

Sync synchronizes the live session to a full prompt token prefix.

func (*Session) SyncTokens 

func (s *Session) SyncTokens(prompt *Tokens) error

SyncTokens synchronizes the live session to a full prompt token prefix.

func (*Session) TokenLogprob 

func (s *Session) TokenLogprob(token int) (TokenScore, error)

TokenLogprob returns the score for a specific token.

func (*Session) Tokens 

func (s *Session) Tokens() *Tokens

Tokens returns a borrowed snapshot of ds4_session_tokens.

func (*Session) TopLogprobs 

func (s *Session) TopLogprobs(k int) ([]TokenScore, error)

TopLogprobs returns the top k token scores for the current logits.

type SessionRewriteResult 

type SessionRewriteResult int32

SessionRewriteResult is returned by ds4 session rewrite helpers. 

const (
	// SessionRewriteError means the rewrite failed.
	SessionRewriteError SessionRewriteResult = -1
	// SessionRewriteOK means the rewrite completed in place.
	SessionRewriteOK SessionRewriteResult = 0
	// SessionRewriteRebuildNeeded means the caller should restore or rebuild the session state.
	SessionRewriteRebuildNeeded SessionRewriteResult = 1
)

type ThinkMode 

type ThinkMode int32

ThinkMode controls ds4's rendered chat thinking mode. 

const (
	// ThinkNone disables thinking markers in chat prompts.
	ThinkNone ThinkMode = iota
	// ThinkHigh enables ordinary high-effort thinking.
	ThinkHigh
	// ThinkMax enables the maximum-effort thinking prefix when the context is large enough.
	ThinkMax
)

func ThinkModeForContext 

func ThinkModeForContext(mode ThinkMode, ctxSize int) ThinkMode

ThinkModeForContext returns the effective thinking mode for a context size.

type TokenEmitFunc 

type TokenEmitFunc func(token int)

TokenEmitFunc is called when ds4 emits a generated token.

type TokenScore 

type TokenScore struct {
	// ID is the token id.
	ID int
	// Logit is the raw model logit.
	Logit float32
	// Logprob is the log probability for the token.
	Logprob float32
}

TokenScore is ds4_token_score.

type Tokens 

type Tokens struct {
	// contains filtered or unexported fields
}

Tokens owns a ds4_tokens value allocated by libds4.

func NewTokens 

func NewTokens(ids []int) (*Tokens, error)

NewTokens creates a libds4-owned token vector from ids.

func (*Tokens) Cap 

func (t *Tokens) Cap() int

Cap returns the token vector capacity.

func (*Tokens) Copy 

func (t *Tokens) Copy() *Tokens

Copy returns a deep copy of this token vector.

func (*Tokens) Free 

func (t *Tokens) Free()

Free releases memory owned by this token vector.

func (*Tokens) Len 

func (t *Tokens) Len() int

Len returns the number of tokens.

func (*Tokens) Push 

func (t *Tokens) Push(token int)

Push appends one token id to the vector.

func (*Tokens) Slice 

func (t *Tokens) Slice() []int

Slice returns a copy of the token ids.

func (*Tokens) StartsWith 

func (t *Tokens) StartsWith(prefix *Tokens) bool

StartsWith reports whether t begins with prefix.

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.