Help us shape the future of Go! Take the 2026 Go Developer Survey

needle

package module
v0.0.0-...-4dbd0fe Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Oct 2, 2026 License: MIT Imports: 22 Imported by: 0

README

needle-go

needle-go is a Go library for running the Cactus Needle on-device tool-calling model.

It provides:

  • CGO-free native engine loading
  • Needle 2 and Needle 3, with Needle 3 as the default
  • automatic, checksum-verified engine and base-weight downloads
  • high-level and manual completion loops
  • typed Go tool handlers
  • structured response extraction
  • Needle 3 text embeddings
  • opt-in Whistle speech transcription and audio tool calling

This project is in early development and its API may change.

Install

Requires Go 1.25 or newer.

go get github.com/zbiljic/needle-go

Engine builds are available for macOS, Linux (glibc and musl), and Windows on amd64 and arm64.

Quick Start

package main

import (
	"context"
	"fmt"
	"log"

	needle "github.com/zbiljic/needle-go"
)

type weatherArguments struct {
	City string `json:"city" jsonschema:"description=City whose weather should be returned."`
}

func main() {
	ctx := context.Background()
	weather := needle.NewTool(
		"get_weather",
		"Get the current weather for a city.",
		func(_ context.Context, arguments weatherArguments) (string, error) {
			return "Clear in " + arguments.City, nil
		},
	)

	agent, err := needle.New(ctx, needle.Config{Tools: []needle.Tool{weather}})
	if err != nil {
		log.Fatal(err)
	}
	response, err := agent.Run(
		ctx,
		"What is the weather in Lagos?",
		needle.DefaultMaxSteps,
		needle.DefaultMaxNewTokens,
	)
	if err != nil {
		log.Fatal(err)
	}
	fmt.Println(response.Results)
}

By default, needle.New downloads and caches the pinned Needle 3 engine (3.1.0) and its needle3.cact base weights (~35 MB) from the Needle 3 model repository. Subsequent runs use the cached files. No Python installation is required.

Tool triggers

Opt into regex routing by assigning triggers after NewTool, using the weather tool above:

weather.Schema.Triggers = []string{`\bweather\b`, `\bforecast\b`}

The native engine interprets the patterns and matches them case-insensitively. Matching triggers can bypass the engine's confidence floor; application confidence policies and response validation still apply.

Independent requests

Agents retain conversation context by default. Set Stateless: true for independent requests:

agent, err := needle.New(ctx, needle.Config{Tools: tools, Stateless: true})

Each Complete or Run starts with a fresh conversation; Run retains context between its tool rounds. Keep the default stateful mode for manual tool loops that feed results back through Complete, and call Reset between independent queries.

Text embeddings

Needle 3 agents expose text features as []float32:

vector, err := agent.Embed(ctx, "turn on the kitchen lights")

Embed does not reset the conversation, including in stateless mode. Switching agents still follows the conversation rules below. Needle 2 and engines without needle_embed return an error; both legacy and audio-capable Needle 3 ABIs are supported.

The base model uses confidence-head probe features rather than a trained contrastive retrieval head. Evaluate similarity on your own data; this API does not enable automatic tool retrieval. See the cosine-ranking example and upstream embedding notes.

To check finite, repeatable vectors and a tool-result continuation with an embedding between turns against the real engine:

NEEDLE_TEST_NATIVE=1 go test -run '^TestNativeEmbeddings$' -v .

Speech

NewWhistle adds optional speech transcription and audio tool calling with Needle 3. It accepts 16 kHz mono []float32 PCM, up to 30 seconds. Speech weights are downloaded only when requested.

See the speech guide for the API, supported languages and offline setup, or try the PCM example.

Model generations

Use Generation: 2 for the embedded Needle 2 model (engine 2.0.4):

agent, err := needle.New(ctx, needle.Config{
	Generation: 2,
	Tools:      tools,
})

Generation: 0 and Generation: 3 select Needle 3. A custom WeightsPath selects the compatible engine from the .cact header, taking precedence over Generation. Custom Needle 3 archives retain the engine's confidence scores when their metadata carries a recognized confidence head, including platform fine-tunes and the published base archive supplied through WeightsPath. Custom Needle 2 weights and recognized text archives without a valid confidence head report Confidence == nil. Unrecognized model-kind metadata is rejected before loading. Automatically loaded base weights retain the engine's confidence scores.

Both generations can run in the same process, with separate native runtimes. Within one generation, switching agents reinitializes the active conversation. After loading custom Needle 2 weights, returning to its embedded base model requires a separate process. Needle 3 can reload its base archive.

Use LibraryPath for an existing, trusted engine matching the selected generation. When LibraryPath is empty, environment overrides are NEEDLE2_LIB_PATH and NEEDLE3_LIB_PATH; the legacy NEEDLE_LIB_PATH is a fallback for Needle 2 only. A Needle 3 library override still downloads missing base weights unless WeightsPath is supplied.

Native loading detects the audio-capable ABI from its exported capability symbols. Trusted older Needle 2 and Needle 3 libraries retain their legacy completion ABI; incomplete capability sets are rejected before inference. Engines exposing needle_last_error provide detailed native failure messages.

FetchEngine prepares the library and required base weights for offline use. FetchOptions.Generation selects the generation. Default caches are under ~/.cache/cactus-needle/<engine-version>/<platform>/; CacheDir selects an exact directory. V2 uses libneedle.*, while v3 uses libneedle3.*, so the generations can share a custom cache directory for the same platform. For offline use, fetch for the target platform and generation, preserve the .sha256 marker files, and use the same cache directory at runtime. Markers contain the downloaded artifact hash and installed file hash; cached downloads verify both. CachedEngine checks only for the library and trusts the caller's file.

Native engine diagnostics

The optional needlez command can fetch the engine and verify that it loads:

go run github.com/zbiljic/needle-go/cmd/needlez@latest fetch
go run github.com/zbiljic/needle-go/cmd/needlez@latest doctor --smoke
go run github.com/zbiljic/needle-go/cmd/needlez@latest test

To run the checks against Needle 2 from a repository checkout:

go run ./cmd/needlez fetch --generation 2
go run ./cmd/needlez doctor --generation 2 --smoke
go run ./cmd/needlez test --generation 2

Responses preserve suppressed_calls for inspection. Run executes only function_calls; it never executes suppressed calls automatically.

Examples

License

Licensed under the MIT License.

Documentation

Overview

Package needle provides Go access to the Needle tool-calling model.

Index

Constants

View Source 
const (
	// EngineVersion is the default (Needle 3) native engine version.
	EngineVersion = "3.1.0"
	// EnvLibraryPath overrides Needle 2 engine discovery for legacy clients.
	EnvLibraryPath = "NEEDLE_LIB_PATH"
)
View Source 
const (
	// DefaultGeneration is used when no generation or custom weights are selected.
	DefaultGeneration = 3
	// DefaultMaxSteps is the default maximum number of tool-calling rounds.
	DefaultMaxSteps = 8
	// DefaultMaxNewTokens is the default generation limit for each completion.
	DefaultMaxNewTokens = 512
	// DefaultBufferSize is the default native response buffer size.
	DefaultBufferSize = 64 * 1024
)
View Source 
const (
	// SpeechSampleRate is the required rate for mono float32 PCM.
	SpeechSampleRate = 16000
	// MaxSpeechSamples limits a clip to 30 seconds.
	MaxSpeechSamples = 30 * SpeechSampleRate
	// DefaultSpeechBufferSize accommodates transcripts with word timestamps.
	DefaultSpeechBufferSize = 1 << 18
)

Variables

View Source 
var ErrEngineNotFound = errors.New("needle: engine library not found")

ErrEngineNotFound indicates that a cached engine library is unavailable.

View Source 
var ErrUnsupportedPlatform = errors.New("needle: unsupported platform")

ErrUnsupportedPlatform indicates that no PureGo shared-library backend is available for the requested target.

Functions

func CachedEngine 

func CachedEngine(options FetchOptions) (string, error)

CachedEngine returns the expected cached library path without downloading anything. It accepts libraries installed by needle-go or another client.

func EngineVersionFor 

func EngineVersionFor(generation int) (string, error)

EngineVersionFor returns the pinned engine version for a generation (zero means 3).

func Extract 

func Extract[T any](response Response) (T, error)

Extract decodes a structured response containing exactly one function call into a value of type T. It does not validate engine warnings; callers that act on the value should call ValidateResponse first.

func FetchEngine 

func FetchEngine(ctx context.Context, options FetchOptions) (string, error)

FetchEngine downloads, verifies, and caches the shared library and required base weights for one desktop platform. It returns the library path.

func FetchWhistle 

func FetchWhistle(ctx context.Context, options FetchOptions) (string, error)

FetchWhistle caches the Needle 3 engine and Whistle speech weights for offline use. It returns the library path and does not download text weights. Generation must be zero or 3; Platform and CacheDir follow FetchEngine.

func ValidateResponse 

func ValidateResponse(response Response) error

ValidateResponse checks only engine validation warnings, accepting absent metadata. It rejects negation or any ungrounded field path, including possible false positives; a matching number elsewhere in the prompt does not exempt a warning. Unflagged output still needs application validation.

func WeightsGeneration 

func WeightsGeneration(path string) (int, error)

WeightsGeneration reads a .cact header and returns its model generation. It does not validate the rest of the archive.

Types

type Agent 

type Agent interface {
	// Complete performs raw inference without applying response validation.
	Complete(ctx context.Context, text string, maxNewTokens int) (Response, error)
	// Embed returns text features from Needle 3 without resetting the conversation.
	// Engines without embedding support return an error.
	Embed(ctx context.Context, text string) ([]float32, error)
	// Run uses ValidateResponse to reject an entire flagged turn before executing
	// any tool calls, without retrying. It returns the raw flagged response and
	// results from earlier completed rounds.
	Run(ctx context.Context, query string, maxSteps, maxNewTokens int) (Response, error)
	Reset(ctx context.Context) error
}

Agent defines the Needle conversation lifecycle.

func New 

func New(ctx context.Context, config Config) (Agent, error)

New initializes an Agent, downloading the selected engine and any required base weights when they are not cached. Generation 3 is the default.

type AudioOptions 

type AudioOptions struct {
	// Language is empty for detection or en, de, fr, es, it, nl or pl.
	Language       string
	Keywords       []string
	WordTimestamps bool
}

AudioOptions controls language detection, keyword biasing and word timestamps.

type Config 

type Config struct {
	Tools  []Tool
	System string
	// Stateless resets the conversation before each Complete or Run request.
	// Run preserves context between tool rounds. The default is stateful; use
	// stateful mode for manual tool loops that continue through Complete.
	Stateless bool
	// Generation selects 2 or 3. Zero defaults to 3; WeightsPath takes precedence.
	Generation int
	// WeightsPath selects custom .cact weights and determines the generation.
	// Custom Needle 3 weights retain confidence when their archive carries a
	// recognized confidence head; other custom weights report nil. The header
	// and model-kind metadata are checked before loading. Remaining model bytes
	// and engine revision compatibility are validated by the native loader.
	WeightsPath   string
	ToolIndexPath string
	BufferSize    int

	// LibraryPath selects a trusted shared library for the selected generation.
	// When empty, New checks NEEDLE2_LIB_PATH or NEEDLE3_LIB_PATH, then fetches
	// the engine. NEEDLE_LIB_PATH is a legacy override for generation 2 only.
	LibraryPath string
	// CacheDir overrides the engine and base-weight download directory.
	CacheDir string
}

Config configures an Agent.

type FetchOptions 

type FetchOptions struct {
	// Generation selects 2 or 3; zero defaults to 3.
	Generation int
	Platform   Platform
	CacheDir   string
	Client     *http.Client
}

FetchOptions configures an engine download. An empty Platform selects the current process platform. CacheDir is the exact destination directory.

type FunctionCall 

type FunctionCall struct {
	Name      string          `json:"name"`
	Arguments json.RawMessage `json:"arguments"`
}

FunctionCall is a tool invocation selected by the model.

type Platform 

type Platform string

Platform identifies a downloadable desktop engine build. 

const (
	PlatformDarwinAMD64    Platform = "darwin-amd64"
	PlatformDarwinARM64    Platform = "darwin-arm64"
	PlatformLinuxAMD64     Platform = "linux-amd64"
	PlatformLinuxARM64     Platform = "linux-arm64"
	PlatformLinuxAMD64Musl Platform = "linux-amd64-musl"
	PlatformLinuxARM64Musl Platform = "linux-arm64-musl"
	PlatformWindowsAMD64   Platform = "windows-amd64"
	PlatformWindowsARM64   Platform = "windows-arm64"
)

func CurrentPlatform 

func CurrentPlatform() (Platform, error)

CurrentPlatform returns the engine build matching the running process.

func SupportedPlatforms 

func SupportedPlatforms() []Platform

SupportedPlatforms returns the desktop engine builds known to this version.

type Response 

type Response struct {
	Type          ResponseType   `json:"type"`
	Success       bool           `json:"success"`
	Error         *string        `json:"error"`
	ErrorCode     *string        `json:"error_code"`
	FunctionCalls []FunctionCall `json:"function_calls"`
	// SuppressedCalls are withheld by the engine and must not be executed automatically.
	SuppressedCalls []FunctionCall  `json:"suppressed_calls,omitempty"`
	Reasoning       string          `json:"reasoning"`
	Confidence      *float64        `json:"confidence"`
	PrefillTPS      float64         `json:"prefill_tps"`
	DecodeTPS       float64         `json:"decode_tps"`
	PeakRAMMB       float64         `json:"peak_ram_mb"`
	Results         []any           `json:"results,omitempty"`
	Validation      *Validation     `json:"validation,omitempty"`
	AudioText       *string         `json:"audio_text,omitempty"`
	AudioLanguage   *string         `json:"audio_language,omitempty"`
	AudioWords      []WordTimestamp `json:"audio_words,omitempty"`
	AudioTTFTMS     *float64        `json:"audio_ttft_ms,omitempty"`
	AudioDecodeTPS  *float64        `json:"audio_decode_tps,omitempty"`
}

Response is the structured response envelope returned by Needle.

type ResponseType 

type ResponseType string

ResponseType identifies the action returned by Needle. 

const (
	ResponseCall    ResponseType = "call"
	ResponseRespond ResponseType = "respond"
	ResponseRefuse  ResponseType = "refuse"
	ResponseText    ResponseType = "text"
)

type Tool 

type Tool struct {
	Schema  ToolSchema
	Handler ToolHandler
}

Tool pairs a JSON schema with an optional Go implementation. A tool without a handler can be used with Complete, while Run reports it as unknown if the model selects it.

func NewTool 

func NewTool[Arguments, Result any](
	name string,
	description string,
	handler func(context.Context, Arguments) (Result, error),
) Tool

NewTool builds a Tool whose parameter schema is derived from Arguments. Model-produced JSON arguments are decoded before the typed handler is called.

A nil handler produces a schema-only tool suitable for use with Complete.

type ToolHandler 

type ToolHandler func(context.Context, json.RawMessage) (any, error)

ToolHandler executes a model-selected function call.

type ToolSchema 

type ToolSchema struct {
	Name        string         `json:"name"`
	Description string         `json:"description,omitempty"`
	Parameters  map[string]any `json:"parameters"`
	// Triggers opt into native regex routing and can bypass the confidence floor.
	Triggers []string `json:"triggers,omitempty"`
}

ToolSchema describes a function that the model may call.

func SchemaFor 

func SchemaFor[T any](name, description string) ToolSchema

SchemaFor builds a tool schema whose parameters are derived from T. JSON field names and optionality follow encoding/json tags; JSON Schema annotations can be supplied with jsonschema tags.

type Transcript 

type Transcript struct {
	Text      string          `json:"text"`
	Language  string          `json:"language"`
	Words     []WordTimestamp `json:"words,omitempty"`
	TTFTMS    float64         `json:"ttft_ms"`
	DecodeTPS float64         `json:"decode_tps"`
}

Transcript is a Whistle response. Silence has empty Text and Language.

type Validation 

type Validation struct {
	Ungrounded []string `json:"ungrounded"`
	Negation   bool     `json:"negation"`
}

Validation contains warnings produced by the Needle engine.

type Whistle 

type Whistle struct {
	// contains filtered or unexported fields
}

Whistle transcribes caller-provided 16 kHz mono float32 PCM in [-1, 1]. Speech and Needle 3 text models share a serialized process-global engine.

func NewWhistle 

func NewWhistle(ctx context.Context, config WhistleConfig) (*Whistle, error)

NewWhistle loads speech weights without downloading or initializing text weights.

func (*Whistle) Complete 

func (w *Whistle) Complete(ctx context.Context, textAgent Agent, pcm []float32, options AudioOptions, maxNewTokens int) (Response, error)

Complete transcribes PCM and completes a raw Needle 3 turn inside the engine. It returns tool calls without executing handlers or applying ValidateResponse. agent must be a Needle 3 Agent created by New and share this Whistle's engine.

func (*Whistle) Transcribe 

func (w *Whistle) Transcribe(ctx context.Context, pcm []float32, options AudioOptions) (Transcript, error)

Transcribe accepts up to 30 seconds of PCM. Empty audio and silence return an empty transcript. Calls are serialized with all text and speech native work.

type WhistleConfig 

type WhistleConfig struct {
	WeightsPath string
	LibraryPath string
	CacheDir    string
	BufferSize  int
}

WhistleConfig selects speech weights and a Needle 3.1-compatible engine. LibraryPath and CacheDir follow Config. Empty WeightsPath fetches Whistle.

type WordTimestamp 

type WordTimestamp struct {
	Word        string  `json:"word"`
	Start       float64 `json:"start"`
	End         float64 `json:"end"`
	Probability float64 `json:"probability"`
}

WordTimestamp describes a recognized word, with times in seconds.

Source Files

View all Source files

Directories

Path Synopsis
cmd
needlez command
examples
embeddings command
fetch-engine command
manual-loop command
structured-extraction command
tool-call command
transcribe command
Transcribe a file of little-endian float32 mono samples recorded at 16000 Hz.
 Transcribe a file of little-endian float32 mono samples recorded at 16000 Hz.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.