Documentation
¶
Overview ¶
Package needle provides Go access to the Needle tool-calling model.
Index ¶
- Constants
- Variables
- func CachedEngine(options FetchOptions) (string, error)
- func EngineVersionFor(generation int) (string, error)
- func Extract[T any](response Response) (T, error)
- func FetchEngine(ctx context.Context, options FetchOptions) (string, error)
- func FetchWhistle(ctx context.Context, options FetchOptions) (string, error)
- func ValidateResponse(response Response) error
- func WeightsGeneration(path string) (int, error)
- type Agent
- func (a *Agent) Close() error
- func (a *Agent) Complete(ctx context.Context, text string, maxNewTokens int) (Response, error)
- func (a *Agent) Embed(ctx context.Context, text string) ([]float32, error)
- func (a *Agent) Reset(ctx context.Context) error
- func (a *Agent) Run(ctx context.Context, query string, maxSteps, maxNewTokens int) (Response, error)
- type AudioOptions
- type Config
- type FetchOptions
- type FunctionCall
- type NativeError
- type Platform
- type Response
- type ResponseType
- type Stream
- type StreamTranscript
- type Tool
- type ToolHandler
- type ToolSchema
- type Transcript
- type Validation
- type Whistle
- func (w *Whistle) Close() error
- func (w *Whistle) Complete(ctx context.Context, textAgent *Agent, pcm []float32, options AudioOptions, ...) (Response, error)
- func (w *Whistle) Stream(options AudioOptions) (*Stream, error)
- func (w *Whistle) Transcribe(ctx context.Context, pcm []float32, options AudioOptions) (Transcript, error)
- type WhistleConfig
- type WordTimestamp
Constants ¶
const ( // EngineVersion is the default (Needle 3) native engine version. EngineVersion = "3.3.1" // EnvLibraryPath overrides Needle 2 engine discovery for legacy clients. EnvLibraryPath = "NEEDLE_LIB_PATH" )
const ( // DefaultGeneration is used when no generation or custom weights are selected. DefaultGeneration = 3 // DefaultMaxSteps is the default maximum number of tool-calling rounds. DefaultMaxSteps = 8 // DefaultMaxNewTokens is the default generation limit for each completion. DefaultMaxNewTokens = 512 // DefaultBufferSize is the default native response buffer size. DefaultBufferSize = 64 * 1024 )
const ( // SpeechSampleRate is the required rate for mono float32 PCM. SpeechSampleRate = 16000 // DefaultSpeechBufferSize accommodates transcripts with word timestamps for // clips up to about a minute; longer clips grow the buffer. DefaultSpeechBufferSize = 1 << 18 )
Variables ¶
var ( // ErrUnsupportedPlatform indicates that no shared-library backend or engine // build exists for the target. ErrUnsupportedPlatform = errors.New("needle: unsupported platform") // ErrEngineNotFound indicates that a cached engine library is unavailable. ErrEngineNotFound = errors.New("needle: engine library not found") // ErrUnsupportedGeneration indicates a generation other than 2 or 3, or a // feature the selected generation lacks. ErrUnsupportedGeneration = errors.New("needle: unsupported generation") // ErrUnsupportedABI indicates a native library with an incomplete // capability set. ErrUnsupportedABI = errors.New("needle: unsupported native ABI") // ErrEmbeddingUnsupported indicates that the model or engine has no // embeddings. ErrEmbeddingUnsupported = errors.New("needle: embeddings unavailable") // ErrSpeechUnsupported indicates that the engine has no speech ABI. ErrSpeechUnsupported = errors.New("needle: speech unavailable") // ErrValidation is wrapped by ValidateResponse, and by Run, when the engine // flags a response. ErrValidation = errors.New("needle: response validation") // ErrStreamActive indicates that a live transcription is open on the // engine, so other speech calls must wait for its Stop. ErrStreamActive = errors.New("needle: a live transcription is in progress") // ErrClosed is returned by an Agent or Whistle after Close. ErrClosed = errors.New("needle: closed") )
Sentinel errors for errors.Is. Returned errors wrap them with detail.
Functions ¶
func CachedEngine ¶
func CachedEngine(options FetchOptions) (string, error)
CachedEngine returns the expected cached library path without downloading anything. It accepts libraries installed by needle-go or another client.
func EngineVersionFor ¶
EngineVersionFor returns the pinned engine version for a generation (zero means 3).
func Extract ¶
Extract decodes a structured response containing exactly one function call into a value of type T. It does not validate engine warnings; callers that act on the value should call ValidateResponse first.
func FetchEngine ¶
func FetchEngine(ctx context.Context, options FetchOptions) (string, error)
FetchEngine downloads, verifies, and caches the shared library and required base weights for one desktop platform. It returns the library path.
func FetchWhistle ¶
func FetchWhistle(ctx context.Context, options FetchOptions) (string, error)
FetchWhistle caches the Needle 3 engine and Whistle speech weights for offline use. It returns the library path and does not download text weights. Generation must be zero or 3; Platform and CacheDir follow FetchEngine.
func ValidateResponse ¶
ValidateResponse checks only engine validation warnings, accepting absent metadata. It rejects negation or any ungrounded field path, including possible false positives; a matching number elsewhere in the prompt does not exempt a warning. Unflagged output still needs application validation.
func WeightsGeneration ¶
WeightsGeneration reads a .cact header and returns its model generation. It does not validate the rest of the archive.
Types ¶
type Agent ¶
type Agent struct {
// contains filtered or unexported fields
}
Agent is a Needle conversation bound to a process-global native runtime. Create one with New. Methods are safe for concurrent use; native calls for one generation are serialized.
func New ¶
New initializes an Agent, downloading the selected engine and any required base weights when they are not cached. Generation 3 is the default.
func (*Agent) Close ¶
Close releases the agent's native binding and response buffer. Later calls return ErrClosed. Close is idempotent and does not unload the engine or its weights, which remain available to other agents in the process.
func (*Agent) Embed ¶
Embed returns text features from Needle 3 without resetting the conversation. Engines without embedding support return an error.
func (*Agent) Run ¶
func (a *Agent) Run(ctx context.Context, query string, maxSteps, maxNewTokens int) (Response, error)
Run completes a query, rejects engine validation warnings, and executes accepted tool calls until the model returns a final response. It uses ValidateResponse to reject an entire flagged turn before executing any tool calls, without retrying, and returns the raw flagged response together with results from earlier completed rounds.
type AudioOptions ¶
type AudioOptions struct {
// Language is empty for detection or en, de, fr, es, it, nl or pl.
Language string
Keywords []string
WordTimestamps bool
}
AudioOptions controls language detection, keyword biasing and word timestamps.
type Config ¶
type Config struct {
Tools []Tool
System string
// Stateless resets the conversation before each Complete or Run request.
// Run preserves context between tool rounds. The default is stateful; use
// stateful mode for manual tool loops that continue through Complete.
Stateless bool
// Generation selects 2 or 3. Zero defaults to 3; WeightsPath takes precedence.
Generation int
// WeightsPath selects custom .cact weights and determines the generation.
// Custom Needle 3 weights retain confidence when their archive carries a
// recognized confidence head; other custom weights report nil. The header
// and model-kind metadata are checked before loading. Remaining model bytes
// and engine revision compatibility are validated by the native loader.
WeightsPath string
ToolIndexPath string
BufferSize int
// LibraryPath selects a trusted shared library for the selected generation.
// When empty, New checks NEEDLE2_LIB_PATH or NEEDLE3_LIB_PATH, then fetches
// the engine. NEEDLE_LIB_PATH is a legacy override for generation 2 only.
LibraryPath string
// CacheDir overrides the engine and base-weight download directory.
CacheDir string
}
Config configures an Agent.
type FetchOptions ¶
type FetchOptions struct {
// Generation selects 2 or 3; zero defaults to 3.
Generation int
Platform Platform
CacheDir string
Client *http.Client
}
FetchOptions configures an engine download. An empty Platform selects the current process platform. CacheDir is the exact destination directory.
type FunctionCall ¶
type FunctionCall struct {
Name string `json:"name"`
Arguments json.RawMessage `json:"arguments"`
}
FunctionCall is a tool invocation selected by the model.
type NativeError ¶
NativeError reports a failed engine call. Detail carries the engine's diagnostic when one is available.
func (*NativeError) Error ¶
func (e *NativeError) Error() string
type Platform ¶
type Platform string
Platform identifies a downloadable desktop engine build.
const ( PlatformDarwinAMD64 Platform = "darwin-amd64" PlatformDarwinARM64 Platform = "darwin-arm64" PlatformLinuxAMD64 Platform = "linux-amd64" PlatformLinuxARM64 Platform = "linux-arm64" PlatformLinuxAMD64Musl Platform = "linux-amd64-musl" PlatformLinuxARM64Musl Platform = "linux-arm64-musl" PlatformWindowsAMD64 Platform = "windows-amd64" PlatformWindowsARM64 Platform = "windows-arm64" )
func CurrentPlatform ¶
CurrentPlatform returns the engine build matching the running process.
func SupportedPlatforms ¶
func SupportedPlatforms() []Platform
SupportedPlatforms returns the desktop engine builds known to this version.
type Response ¶
type Response struct {
Type ResponseType `json:"type"`
Success bool `json:"success"`
Error *string `json:"error"`
ErrorCode *string `json:"error_code"`
FunctionCalls []FunctionCall `json:"function_calls"`
// SuppressedCalls are withheld by the engine and must not be executed automatically.
SuppressedCalls []FunctionCall `json:"suppressed_calls,omitempty"`
Reasoning string `json:"reasoning"`
Confidence *float64 `json:"confidence"`
PrefillTPS float64 `json:"prefill_tps"`
DecodeTPS float64 `json:"decode_tps"`
PeakRAMMB float64 `json:"peak_ram_mb"`
Results []any `json:"results,omitempty"`
Validation *Validation `json:"validation,omitempty"`
AudioText *string `json:"audio_text,omitempty"`
AudioLanguage *string `json:"audio_language,omitempty"`
AudioWords []WordTimestamp `json:"audio_words,omitempty"`
AudioTTFTMS *float64 `json:"audio_ttft_ms,omitempty"`
AudioDecodeTPS *float64 `json:"audio_decode_tps,omitempty"`
}
Response is the structured response envelope returned by Needle.
type ResponseType ¶
type ResponseType string
ResponseType identifies the action returned by Needle.
const ( ResponseCall ResponseType = "call" ResponseRespond ResponseType = "respond" ResponseRefuse ResponseType = "refuse" ResponseText ResponseType = "text" )
type Stream ¶
type Stream struct {
// contains filtered or unexported fields
}
Stream is a live transcription started by Whistle.Stream. Feed about one second of PCM per Process call and finish with Stop, which commits the tail. The engine holds one stream per process: while a stream is open, speech calls other than its own Process and Stop return ErrStreamActive. A failed Process ends the stream. Closing the Whistle also ends it.
type StreamTranscript ¶
type StreamTranscript struct {
Text string `json:"text"`
Words []WordTimestamp `json:"words"`
Pending string `json:"pending"`
Language string `json:"language"`
Received float64 `json:"received"`
PassMS float64 `json:"pass_ms"`
}
StreamTranscript is one step of live transcription. Text and Words are the words the step committed, with times in seconds from the start of the stream, and Pending is the unconfirmed tail. Received counts the seconds of audio so far and PassMS the time the step took. Join the texts of consecutive steps with a space.
type Tool ¶
type Tool struct {
Schema ToolSchema
Handler ToolHandler
}
Tool pairs a JSON schema with an optional Go implementation. A tool without a handler can be used with Complete, while Run reports it as unknown if the model selects it.
func NewTool ¶
func NewTool[Arguments, Result any]( name string, description string, handler func(context.Context, Arguments) (Result, error), ) Tool
NewTool builds a Tool whose parameter schema is derived from Arguments. Model-produced JSON arguments are decoded before the typed handler is called.
A nil handler produces a schema-only tool suitable for use with Complete.
type ToolHandler ¶
ToolHandler executes a model-selected function call.
type ToolSchema ¶
type ToolSchema struct {
Name string `json:"name"`
Description string `json:"description,omitempty"`
Parameters map[string]any `json:"parameters"`
// Triggers opt into native regex routing and can bypass the confidence floor.
Triggers []string `json:"triggers,omitempty"`
}
ToolSchema describes a function that the model may call.
func SchemaFor ¶
func SchemaFor[T any](name, description string) ToolSchema
SchemaFor builds a tool schema whose parameters are derived from T. JSON field names and optionality follow encoding/json tags; JSON Schema annotations can be supplied with jsonschema tags.
type Transcript ¶
type Transcript struct {
Text string `json:"text"`
Language string `json:"language"`
Words []WordTimestamp `json:"words,omitempty"`
TTFTMS float64 `json:"ttft_ms"`
DecodeTPS float64 `json:"decode_tps"`
}
Transcript is a Whistle response. Silence has empty Text and Language.
type Validation ¶
Validation contains warnings produced by the Needle engine.
type Whistle ¶
type Whistle struct {
// contains filtered or unexported fields
}
Whistle transcribes caller-provided 16 kHz mono float32 PCM in [-1, 1]. Speech and Needle 3 text models share a serialized process-global engine.
func NewWhistle ¶
func NewWhistle(ctx context.Context, config WhistleConfig) (*Whistle, error)
NewWhistle loads speech weights without downloading or initializing text weights.
func (*Whistle) Close ¶
Close ends an open stream of this Whistle and releases the response buffer. Later calls return ErrClosed. Loaded speech weights stay in the engine for other Whistle values.
func (*Whistle) Complete ¶
func (w *Whistle) Complete(ctx context.Context, textAgent *Agent, pcm []float32, options AudioOptions, maxNewTokens int) (Response, error)
Complete transcribes PCM and completes a raw Needle 3 turn inside the engine. It returns tool calls without executing handlers or applying ValidateResponse. textAgent must be a Needle 3 Agent created by New that shares this engine.
func (*Whistle) Stream ¶
func (w *Whistle) Stream(options AudioOptions) (*Stream, error)
Stream prepares a live transcription with the language and keywords of options; WordTimestamps is ignored because every step carries word times. Nothing reaches the engine until the first Process.
func (*Whistle) Transcribe ¶
func (w *Whistle) Transcribe(ctx context.Context, pcm []float32, options AudioOptions) (Transcript, error)
Transcribe accepts PCM of any length: up to 30 seconds runs as one pass, and longer audio runs as consecutive 30 second windows with word times counted from the start of the clip. Empty audio and silence return an empty transcript. Calls are serialized with all text and speech native work.
type WhistleConfig ¶
WhistleConfig selects speech weights and a Needle 3.1-compatible engine. LibraryPath and CacheDir follow Config. Empty WeightsPath fetches Whistle.
Source Files
¶
Directories
¶
| Path | Synopsis |
|---|---|
|
cmd
|
|
|
needlez
command
|
|
|
examples
|
|
|
embeddings
command
|
|
|
fetch-engine
command
|
|
|
manual-loop
command
|
|
|
structured-extraction
command
|
|
|
tool-call
command
|
|
|
transcribe
command
Transcribe a file of little-endian float32 mono samples recorded at 16000 Hz, in one pass or, with -stream, live in one-second chunks.
|
Transcribe a file of little-endian float32 mono samples recorded at 16000 Hz, in one pass or, with -stream, live in one-second chunks. |