Help us shape the future of Go! Take the 2026 Go Developer Survey

needle

package module
v0.0.0-...-6887232 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Oct 17, 2026 License: Apache-2.0 Imports: 10 Imported by: 0

README

go-needle3

Go library + example CLIs for Needle 3 (weights) — Cactus Compute's tiny on-device tool-calling model. The API mirrors the Python cactus-needle package in idiomatic Go, and the demo binaries ship the entire model inside the executable (go:embed).

Two interchangeable backends, both implementing the needle.Engine interface (Complete / Run / Reset / Embed):

engine/cgo engine/pure
What FFI bindings to the prebuilt libneedle.a shipped in the HF repo (same engine as the Python wheel) full runtime ported to Go: .cact CQ dequantizer, tokenizer, incremental attention, mHC lanes, engrams, Monarch-Hadamard MLP, constrained greedy decoding
cgo yes (static lib per platform) no — zero non-Go dependencies
Toolchain C compiler plus static libc++ archives (libc++.a, libc++abi.a); the libc++ is linked statically, so the binary only needs the base system libs plain Go; not even a C toolchain required
Demo runtime (this machine, 8 cores) 0.9 s full demo ~12 s full demo (~3 s per Complete)
Weights .cact read in place (~0.1 s to first token) dequantized to FP32 at New (~1.5 s, ~1.1 GB RSS)

Quick start

go run ./examples/needle-pure   # pure-Go engine, no cgo
go run ./examples/needle-cgo    # prebuilt engine (linux/windows, x86_64/arm64)

Both CLIs run the same scripted tool-call demos (weather + smart-home tools) and an embedding query. One binary, one embedded needle3.cact (33.7 MB).

Usage

import (
	needle "github.com/lixquid/go-needle3"
	"github.com/lixquid/go-needle3/engine/pure"
	nmodel "github.com/lixquid/go-needle3/model"
)

agent, err := pure.New(needle.Options{
	Weights: nmodel.Needle3, // embedded .cact bytes
	Tools: []needle.Tool{{
		Name:        "get_weather",
		Description: "Current weather in a city.",
		Parameters: map[string]any{
			"type": "object",
			"properties": map[string]any{
				"city": map[string]any{"type": "string"},
			},
			"required": []string{"city"},
		},
	}},
})
defer agent.Close()

resp, err := agent.Run("what's it like in Lagos right now?")
fmt.Println(resp.Type, resp.FunctionCalls, resp.Confidence)

Python ↔ Go cheat sheet (also in needle.go):

Python (cactus-needle) Go
needle.Needle(tools=[...]) pure.New(needle.Options{Tools: ...}) (or cgo.New)
agent.complete(text) agent.Complete(text)
agent.run(query) agent.Run(query) — agentic loop with Tool.Fn handlers
agent.reset() agent.Reset()
agent.embed(text) agent.Embed(text) → 3072-float unit vector
needle.extract(text, schema) pure.Extract(text, schema, opts) (same in cgo)
extract(text, Contact) → model pure.ExtractAs[Contact](text, opts) → *Contact
ExtractionValidationError *needle.ExtractionError
res["function_calls"] resp.FunctionCalls
res["confidence"] resp.Confidence

Extraction

Structured extraction is the model's other job: declare a shape, hand over messy text, get typed fields back. It is tool calling with exactly one record-shaped tool — the decode grammar guarantees the output parses, and classification falls out of an enum field. Both backends expose it the way the Python package does, as a function of text and schema:

type Contact struct {
	Name  string `json:"name"  description:"the person's full name"`
	Email string `json:"email" description:"their email address"`
}

opts := needle.Options{Weights: model.Needle3}

// typed: the schema comes from the struct, the answer decodes into it
c, err := pure.ExtractAs[Contact]("Please email Jane Smith at jane@smith.example", opts)

// generic: any JSON-schema map (or JSON text) works too
ex, err := pure.Extract(text, map[string]any{
	"name":       "Contact",
	"parameters": map[string]any{"type": "object", "properties": map[string]any{...}},
}, opts)
var c Contact
err = ex.Decode(&c) // ex also carries Reasoning, Confidence and Suppressed

schema accepts a struct/map type or value (needle.SchemaOf[T](), needle.SchemaFrom(v)), a needle.Schema, a needle.Tool, a JSON-schema map — with or without the {"name":…,"parameters":{…}} wrapper — or that map as JSON text.

Struct tags beyond json: description:"…" (what the model sees), format:"date", enum:"low,normal,high", default:"…", minimum/maximum, and required:"true"|"false" to override the default. A field is required unless it is a pointer, omitempty/omitzero, or carries a default — the same rule the Python package applies to fields with defaults. Go types map by kind (strings, ints, floats, bools, slices→array, structs→object with inlined unnamed embedded structs, recursion cut off at object); []byte and interfaces travel as free-text strings and time.Time becomes format:"date-time", all as in the reference. A type may override its own fragment by implementing JSONSchema() map[string]any, and a record type its description by implementing Description() string. needle.Date accepts the bare "2026-09-01" the model emits (time.Time does not).

Strict grounding. Strict (default on) rejects values that are not in the input, with *needle.ExtractionError listing the paths — and the values are still returned alongside the error, so a caller can decide:

  • a format:"date" value may only carry a year written in the text, plus the system date's year when the text reasons relatively (tomorrow, next week);
  • the C engine's own validation report (resp.Validation, folded in automatically) marks fabricated paths and negated requests; a flagged number is cleared when the same number is written in the input, which is what lets $1,200.00 satisfy total: 1200 without forgiving a fabricated 9999.

That is not hypothetical: on the real weights, "Invoice from Acme for $1,200, due on 5th September 2031" comes back as due_date: 2026-10-05 — the model anchored the day to today's month — and the strict check names due_date while passing vendor and total.

Worth knowing:

  • An engine binds one toolset per agent, so Extract builds a fresh agent per call and closes it. cgo therefore needs the process-global slot to be free; pure caches the dequantized model per weight set (ReleaseModelCache frees it). For a loop over many texts, bind the schema once with opts.ForExtraction(schema), then agent.ExtractTurn(text) and Reset.
  • Off-topic text is needle.ErrNoExtraction; a call the confidence gate withheld still yields its values, with Extraction.Suppressed set.
  • The pure engine's grammar constrains flat argument objects; array and nested values stay valid JSON but lose their constraints mid-value.
  • Only the C engine reports validation, so the numeric/negation half of the strict check has nothing to work on in the pure engine; the temporal half applies to both.
  • Run's strict refusal of ungrounded calls (Python's run(strict=…)) is not ported; call needle.ValidateGrounding on a turn's arguments yourself.

Platforms

The four platforms the upstream project ships prebuilt engines for — linux-x86_64, linux-arm64, windows-x86_64, windows-arm64 — are covered by engine/cgo (static libs vendored under engine/cgo/libs/<triple>/).

engine/pure compiles anywhere Go does: cross-compile checks pass for windows/amd64, linux/arm64, darwin/arm64 with CGO_ENABLED=0.

[!NOTE] "Only a C toolchain" holds for the cgo binding only in the sense that no C++ compiler is invoked — but linking libneedle.a needs libc++ (and on Linux a libstdc++-only link fails on std::__1 symbols). libc++/ libc++abi are linked statically (-l:libc++.a -l:libc++abi.a), so the final binary requires no shared C++ library — just the base system. This is why the pure-Go variant exists, and why the module ships both: the pure engine has no such requirement at all.

Equivalence evidence

Verified against the upstream JAX reference (needle.model) and the C engine on this machine:

  • CQ dequantizer: FP16 + 4-bit codebook/Walsh–Hadamard records reproduce the JAX tensors exactly (golden test).
  • Forward pass: all 21 probe cells at the final position match the JAX oracle within ~1e-5 relative; the top-5 first-token logits are identical, in order.
  • Engines agree end-to-end: on the shared demo scripts the pure port produces the same calls and — for the Lagos/off-topic cases — reasoning text identical to the C engine, word for word.
  • Embeddings: cosine between pure.Embed and cgo.Embed for the same text is 0.9999 (embedcmp_test.go).
  • Confidence: geometric mean of greedy token probabilities, combined with the calibrated probe-head sigmoid — tracks the engine (e.g. 0.528 vs 0.529 on the Lagos query).

Known deviations (pure engine)

  • Constrained decoding covers the JSON tool-call region (structure, schema keys, tool names, []/</tool_call>/EOS ordering); the reasoning phase is free, so its wording can drift where the engines' near-tie logits disagree (e.g. "brighten" → brightness:80 alone vs also on:true at conf 0.11).
  • Everything runs in FP32; the C engine quantizes KV/attention internally, so occasional token-level divergence is expected under identical prompts.
  • The engine's process-global single-model constraint is preserved: one live agent per kind per process (backends guard with a mutex; the pure engine is internally parallel over matvecs).

Fuzz differential results

fuzzdiff_test.go (build cgo && linux, ~100 s) runs a seeded 35-prompt corpus — 6 weather templates, 5 smart-home commands, 8 gibberish strings, 6 edge cases, 5 unicode/numeric, 5 long (~180-token) — through both engines in one process and compares type, parsed tool calls, and reasoning per prompt. The C ABI exposes no quantization knob, so the reference side runs the engine's native quantized mode; the full per-case report is checked in at docs/fuzzdiff-report.txt.

Metric Pre-gate Post-gate
Type agreement 34/35 34/35
Tool-call agreement 14/35 31/35
Fully exact (type + calls + reasoning) 9/35 19/35
Confidence delta mean 0.041 / max 0.52 unchanged

What the fuzzing found:

  1. The engine suppresses low-confidence calls — pure was missing this gate. Ported (SuppressedCalls, threshold 0.1; corpus: suppressed at conf ≤ 0.093, emitted at ≥ 0.116 in every case), fixing 17 of 21 call mismatches.
  2. In-distribution behavior agrees fully: on all 14 clear weather / light requests, same type and same refusal behavior, 13/14 identical calls (the one diff adds an optional on field at conf 0.116 — a true tie).
  3. Residual divergences are quantization-sensitive near-ties: 15 cases differ only in unconstrained reasoning wording; 3 flip the gate boundary where the two confidence estimators disagree (e.g. 0.158 vs 0.044); 1 flips between EOS and <tool_call>[] (respond vs suppressed call — behaviorally identical: the agent stops either way).
  4. No panics or one-sided errors on either side; the pure engine is byte-reproducible across repeat runs (guarded in the test).

Design choices

Two backends, one interface. needle.Engine + Options/Response live in the root package; backends are subpackages, so the import picks the policy. engine/cgo pins you to a platform triple + static libc++ archives; engine/pure builds under CGO_ENABLED=0 anywhere. Keeping both was also a development strategy: the cgo binding is ground truth, and the pure port doubles as a differential oracle against the JAX reference.

Weights as bytes, model as go:embed. Options.Weights []byte matches the engine ABI (needle_load(cact, n) reads memory in place) and lets model.Needle3 ship the whole 33.7 MB model inside the binary — no runtime file resolution, one self-contained demo CLI.

Python-mirrored surface. New/Complete/Run/Reset/Embed and the JSON Response keys (function_calls, suppressed_calls, …) mirror cactus-needle, so ported examples read 1:1; the Go deltas are idiomatic: Tool.Fn callbacks instead of dict dispatch, error returns instead of exception-ish dicts, ParseResponse for wire JSON.

Process-global semantics preserved. needle.h declares the engine one model per process, non-thread-safe. The cgo backend enforces that (mutex + one live agent per kind); the pure backend keeps the same contract for API symmetry even though it could host several agents (each would hold its own ~1.1 GB dequant copy — see improvements).

FP32 eager dequant in the pure engine. All CQ/FP16 records are decoded to FP32 at New, making the per-step matvecs flat, memory-bound, and trivially parallel (rows split across goroutines). Chose simplicity + fidelity over memory: the result is closer to the JAX truth than the C engine (which runs fp8-quantized KV/attention internally) at the cost of RSS. Everything runs greedy argmax; the C engine's sampler knobs are not documented, so none were invented.

Grammar by parse-and-validate. After <tool_call>, every candidate token is checked by cloning the mini-JSON state and feeding the token's text (~8192 clones × a few bytes ≈ 0.1 ms per step). Exactness over precomputed char-class masks: a token is legal iff the whole span parses legal under the schema (keys ∈ properties, names ∈ loaded tools, </tool_call> only after the array closes). The reasoning phase stays free.

Confidence definition. Engine reports ≈ min(geometric mean of greedy token probabilities, sigmoid(probe head)) — the geometric mean (not the raw product) is what keeps long turns out of the 1e-30 basement and matches the engine's band; see the 0.528 vs 0.529 check above.

Where to improve next

  • Prefill batching: the pure engine advances one token at a time even for prompts; a batched prefill (and per-layer pipeline parallelism) is the biggest latency lever (~3 s Complete today, most of it prompt).
  • Memory: engram tables dominate the 1.1 GB RSS; keeping them FP16 (or dequant-on-access) roughly halves it. A shared dequantized Model handle would also allow multiple concurrent agents without duplicating weights.
  • Grammar completeness: constrain the reasoning phase (the engine does), validate value types against the schema (temperature: float), and support multi-call arrays / streaming callbacks.
  • Numerics: an optional fp8-KV emulation path would let the pure engine reproduce the C engine token-for-token instead of ~word-for-word.
  • Coverage: the windows/arm64 targets are compile-checked only — CI should run the differential test there (the seeded corpus landed as fuzzdiff_test.go); darwin has no upstream static lib, so it is pure-engine-only forever unless Cactus ships one. Tokenizer round-trip fuzzing against the upstream RefTokenizer and expanding the corpus to random toolsets remain open.
  • Confidence estimator: the engine's internal estimator is a black box; our geometric-mean + probe min tracks it (mean delta 0.041) but flips the suppression gate at the boundary on ~2 prompts in 35. Matching it exactly would require probing the engine with engineered logit patterns.
  • API sugar: sampling options (temperature/top-p), populated PrefillTPS/DecodeTPS, and a Kind-based multi-model registry mirroring needle_models() when more checkpoints land.

Repository layout

needle.go            shared API: Options, Tool, Response, Engine, Run loop
extract.go           extraction: Schema, Extraction, ExtractWith plumbing
schema_reflect.go    Go struct type -> JSON schema
grounding.go         strict grounding: years, numbers, engine validation
model/               needle3.cact + go:embed
engine/pure/         pure-Go runtime (cact.go, tokenizer.go, model.go,
                     forward.go, grammar.go, pure.go) + Extract
engine/cgo/          libneedle.a bindings, vendored needle.h + libs/<triple>/
                     + Extract
examples/needle-pure      demo CLI, pure engine
examples/needle-cgo       demo CLI, prebuilt engine
cmd/needle-ask            one-shot tool matcher CLI (cgo backend):
                          toolset JSON + prompt -> matched tool name,
                          optional -facts / -confidence / -json / -format
                          flags

Upstream

Documentation

Overview

Package needle defines the shared API surface for Needle 3 inference in Go.

Needle 3 (Cactus Compute) is a tiny on-device model for tool calling: text goes in, a JSON tool call comes back. Two interchangeable backends implement the Engine interface defined here:

  • engine/cgo: bindings to the prebuilt C/C++ engine (libneedle.a)
  • engine/pure: a pure-Go port of the runtime (no cgo, no libc)

The mirror of the Python cactus-needle package: 

python                         go
needle.Needle(tools=[...])     cgo.New(needle.Options{Tools: ...})
agent.complete(text)           agent.Complete(text)
agent.run(query)               agent.Run(query)
agent.reset()                  agent.Reset()
agent.embed(text)              agent.Embed(text)
needle.extract(text, schema)   pure.Extract(text, schema, opts) (same in cgo)

The underlying engine holds one process-global, non-thread-safe model per kind; backends serialize calls with a mutex and expect a single live Agent per process.

Index

Constants

View Source 
const (
	DefaultBufferSize   = 65536
	DefaultMaxNewTokens = 512
	DefaultMaxSteps     = 8
)

Defaults shared by the backends.

View Source 
const DefaultSchemaName = "extract"

DefaultSchemaName is the tool name used for a schema that does not carry one. It shows up in the prompt and in the model's "name" field.

Variables

View Source 
var ErrNoExtraction = errors.New("needle: no extraction (the model returned no call)")

ErrNoExtraction is returned when the model produced no call for the text: it read the input as off-topic, or answered instead of extracting.

Functions

func Bool 

func Bool(v bool) *bool

Bool returns a pointer to v, for turning Extract's defaults off: opts.Strict = needle.Bool(false).

func ExtractAsWith 

func ExtractAsWith[T any](newEngine EngineFactory, text string, opts Options) (*T, error)

ExtractAsWith is the typed form of ExtractWith: the schema comes from T and the arguments are decoded into a new T. A strict-grounding rejection still hands back the decoded values along with the *ExtractionError, so callers can use them or ignore them; no call at all gives (nil, ErrNoExtraction).

func ValidateGrounding 

func ValidateGrounding(text string, schema any, arguments map[string]any, system string, engineReport *Validation) error

ValidateGrounding is the strict check on its own, for callers who drive the conversation through Engine.Complete or Run instead of Extract: it reports whether arguments are grounded in text, given the schema and the environment facts the model saw. It returns an *ExtractionError, or nil.

The engine's own report folds into the same check: pass resp.Validation (nil for the pure backend) to include its flags.

func WithDateFact 

func WithDateFact(system string, now time.Time) string

WithDateFact prefixes the local date fact unless the caller already supplied one, so a relative date has something to be relative to. A system prompt that already carries a "date:" key, or any ISO stamp, is left alone; JSON facts (an object) are passed through untouched.

Types

type Call 

type Call struct {
	Name      string         `json:"name"`
	Arguments map[string]any `json:"arguments"`
}

Call is a single function call chosen by the model.

type Date 

type Date time.Time

Date is a calendar date for extraction schemas: it accepts "2026-09-01" (the format the model emits for "format":"date") and RFC 3339 stamps, and renders as the plain date. encoding/json's time.Time rejects the bare date.

func (Date) JSONSchema 

func (Date) JSONSchema() map[string]any

JSONSchema marks the field as a date, which is also what switches on the temporal grounding check in a strict Extract.

func (Date) MarshalJSON 

func (d Date) MarshalJSON() ([]byte, error)

MarshalJSON writes the "2006-01-02" form.

func (Date) String 

func (d Date) String() string

String renders the date as "2006-01-02".

func (Date) Time 

func (d Date) Time() time.Time

Time returns the wrapped time value.

func (*Date) UnmarshalJSON 

func (d *Date) UnmarshalJSON(data []byte) error

UnmarshalJSON accepts "2006-01-02", an RFC 3339 stamp (the time part is dropped), or an empty string (the zero Date).

type Described 

type Described interface {
	Description() string
}

Described lets a record type supply the schema description, the Go stand-in for the docstring that pydantic extraction uses. It is called on a zero value, so it must not read the receiver's fields.

type Engine 

type Engine interface {
	// Complete answers one turn. Feed tool results back as a JSON string.
	Complete(text string) (*Response, error)
	// Run executes the full agentic loop: the model picks calls, matching
	// Tool.Fn handlers are executed, results are fed back, and the final
	// response carries the executed outputs in Response.Results.
	Run(query string) (*Response, error)
	// Reset rewinds the conversation, keeping the tools loaded.
	Reset() error
	// Embed returns the model's embedding for a text (3072 floats on the
	// shipped needle3.cact).
	Embed(text string) ([]float32, error)
	// Close releases the agent. A new Agent may be created afterwards.
	Close() error
}

Engine is implemented by both backends.

type EngineFactory 

type EngineFactory func(Options) (Engine, error)

EngineFactory constructs an agent for a one-shot extraction. The backends' New functions match this signature.

type Extraction 

type Extraction struct {
	// Schema is the shape that was extracted, defaults filled in.
	Schema Schema
	// Name is the record name the model used (normally Schema.Name).
	Name string
	// Arguments holds the extracted fields, keyed by schema property.
	Arguments map[string]any

	// Reasoning is the model's <think> text for the turn.
	Reasoning string
	// Confidence is a calibrated score in [0,1]; HasConfidence is false when
	// the loaded weights carry no calibration head.
	Confidence    float64
	HasConfidence bool
	// Suppressed is true when the engine withheld the call below its
	// confidence gate. Extraction still returns the withheld values; a low
	// Confidence is the signal to distrust them.
	Suppressed bool
	// Validation is the engine's own grounding report, when it reports one.
	// The cgo backend passes the engine's "validation" object through; the
	// pure backend has no engine-side flags and leaves it nil.
	Validation *Validation
	// Raw is the raw generated text for the turn, useful for debugging.
	Raw string

	// Response is the full turn the extraction came from.
	Response *Response
}

Extraction is the outcome of one extraction turn.

func ExtractWith 

func ExtractWith(newEngine EngineFactory, text string, schema any, opts Options) (*Extraction, error)

ExtractWith runs one-shot extraction on an engine built by newEngine and closed afterwards, mirroring the Python package's extract(). The engine is bound to a single tool derived from schema, so a fresh agent is created per call; reuse one agent and call ExtractionFromResponse for a hot loop.

func ExtractionFromResponse 

func ExtractionFromResponse(resp *Response, schema Schema, text, system string, strict bool) (*Extraction, error)

ExtractionFromResponse turns a completed turn into an Extraction: the first function call, or the first suppressed one when the confidence gate withheld it. With strict set, values that are not grounded in text raise an *ExtractionError; the Extraction is still returned alongside it so the caller can decide what to do with the values.

func (*Extraction) Decode 

func (e *Extraction) Decode(out any) error

Decode copies the extracted arguments into out, which must be a non-nil pointer, typically *T for the same type SchemaOf was given.

type ExtractionError 

type ExtractionError struct {
	Paths []string
}

ExtractionError reports extracted values that are not grounded in the input, the Go counterpart of Python's ExtractionValidationError. Paths are argument-relative ("total", "items[0].price"); "negated request" marks a turn the engine read as a negation of the requested record.

func (*ExtractionError) Error 

func (e *ExtractionError) Error() string

type Options 

type Options struct {
	// Tools is the toolset bound to this agent. One toolset per session:
	// changing tools requires a new Agent.
	Tools []Tool
	// System is an optional environment-facts string, e.g.
	// "date: 2026-07-21 Tue 14:30; locale: en-US; device: phone".
	System string
	// Weights holds the .cact model bytes; required. Use github.com/.../model
	// for the embedded needle3.cact.
	Weights []byte
	// BufferSize sizes the engine's JSON output buffer. Default 65536.
	BufferSize int
	// MaxNewTokens caps each generated turn. Default 512.
	MaxNewTokens int
	// MaxSteps caps the Run agentic loop. Default 8.
	MaxSteps int
	// Strict makes Extract reject values that are not grounded in the input
	// with an *ExtractionError. Nil means true, mirroring the Python default;
	// set it to *false to return the model's values as-is. It says nothing
	// about Complete and Run, which do not validate.
	Strict *bool
	// AutoDate adds the local date to Options.System when it carries none, so
	// a relative date ("due tomorrow") can be resolved and licensed. Nil
	// means true. Used by Extract only.
	AutoDate *bool
}

Options configures an Agent.

func (Options) AutoDateFacts 

func (o Options) AutoDateFacts() bool

AutoDateFacts reports whether Extract may add the local date fact.

func (Options) ForExtraction 

func (o Options) ForExtraction(s Schema) Options

ForExtraction prepares options for a one-shot extraction of schema: the schema becomes the only tool, the environment facts gain the local date when they do not carry one (relative dates such as "tomorrow" are licensed against it), and the defaults are filled in.

func (Options) StrictExtraction 

func (o Options) StrictExtraction() bool

StrictExtraction reports whether Extract should validate grounding; unset means strict, like the Python package.

func (Options) ToolsJSON 

func (o Options) ToolsJSON() ([]byte, error)

ToolsJSON renders the tool list into the minified JSON array the engine consumes: [{"name":...,"description":...,"parameters":{...}}].

func (Options) WithDefaults 

func (o Options) WithDefaults() Options

type Response 

type Response struct {
	// Type is "call" when the model wants tool calls (an empty FunctionCalls
	// list is the off-topic refusal), or "respond" when the loop is finished.
	Type          string `json:"type"`
	Success       bool   `json:"success"`
	Error         string `json:"error,omitempty"`
	ErrorCode     string `json:"error_code,omitempty"`
	FunctionCalls []Call `json:"function_calls"`
	// SuppressedCalls holds a call the engine withheld (low confidence or a
	// grounding gate); FunctionCalls is then empty.
	SuppressedCalls []Call `json:"suppressed_calls,omitempty"`
	Reasoning       string `json:"reasoning,omitempty"`
	// Confidence is a calibrated score in [0,1]. HasConfidence is false when
	// the loaded weights carry no calibration head.
	Confidence    float64 `json:"confidence"`
	HasConfidence bool    `json:"-"`
	PrefillTPS    float64 `json:"prefill_tps,omitempty"`
	DecodeTPS     float64 `json:"decode_tps,omitempty"`

	// Results holds the outputs of executed calls; populated by Run only.
	Results []Result `json:"-"`

	// Raw carries the raw generated text for this turn (pure backend) or the
	// engine's raw field, useful for debugging.
	Raw string `json:"raw,omitempty"`

	// Validation is the engine's own grounding report for the turn. The C
	// engine returns one ("ungrounded" argument paths plus a negation flag);
	// the pure backend has no engine-side flags, so it stays nil there.
	Validation *Validation `json:"validation,omitempty"`
}

Response is the outcome of one turn, mirroring the JSON object returned by the Python package's complete()/run().

func ParseResponse 

func ParseResponse(data string) (*Response, error)

ParseResponse decodes the engine's per-turn JSON object into a Response, detecting whether a calibrated confidence was present.

func RunLoopFunc 

func RunLoopFunc(complete func(string) (*Response, error), tools []Tool, query string, maxSteps int) (*Response, error)

RunLoopFunc is the shared implementation of Engine.Run, driving a backend's Complete method and executing matching Tool.Fn handlers.

type Result 

type Result struct {
	Call   Call
	Output any
	Err    error
}

Result records one executed call during Run.

type Schema 

type Schema struct {
	// Name is the record name offered to the model, e.g. "Invoice". Empty
	// becomes DefaultSchemaName.
	Name string
	// Description is an optional one-liner shown to the model; it is what
	// steers ambiguous fields.
	Description string
	// Parameters is the JSON-schema object for the record: typically
	// {"type":"object","properties":{...},"required":[...]}. A nil value
	// means an empty object.
	Parameters map[string]any
}

Schema is the shape to extract to: one record, expressed as the JSON-schema object the engine consumes for a single tool.

func SchemaFrom 

func SchemaFrom(v any) (Schema, error)

SchemaFrom accepts the schema forms Extract takes:

  • Schema (or *Schema) — used as given
  • Tool — name, description, parameters
  • map[string]any — a tool entry with "parameters", or a bare JSON-schema object
  • string / []byte — the same two forms as JSON text
  • reflect.Type, struct, *struct, map type — derived like SchemaOf

func SchemaOf 

func SchemaOf[T any]() Schema

SchemaOf derives a Schema from a Go type using encoding/json conventions. See schema_reflect.go for the mapping and the struct tags it honours.

func (Schema) Tool 

func (s Schema) Tool() Tool

Tool renders the schema as the single tool entry an Agent binds.

type SchemaProvider 

type SchemaProvider interface {
	JSONSchema() map[string]any
}

SchemaProvider lets a type decide what JSON schema is generated for it. A pointer receiver works too: the method is called on a zero value.

type Tool 

type Tool struct {
	Name        string
	Description string
	// Parameters is a JSON-schema object, e.g.
	// {"type":"object","properties":{"city":{"type":"string"}},"required":["city"]}.
	// A nil value renders as an empty parameter object.
	Parameters map[string]any
	// Fn executes this tool during Run. Its result is fed back to the model
	// as JSON.
	Fn func(args map[string]any) (any, error)
}

Tool describes one callable function exposed to the model, mirroring the raw JSON-schema form that the engine consumes. Fn is optional and only used by Run to execute calls automatically.

type Validation 

type Validation struct {
	// Ungrounded lists the argument paths the engine considers fabricated,
	// qualified by the tool name ("Invoice.due_date").
	Ungrounded []string `json:"ungrounded,omitempty"`
	// Negation is set when the engine read the turn as a negation of the
	// requested record ("no invoice needed").
	Negation bool `json:"negation,omitempty"`
}

Validation is the grounding report the engine attaches to a turn.

Source Files

View all Source files

Directories

Path Synopsis
cmd
needle-ask command
Command needle-ask is a one-shot Needle 3 tool matcher built on the cgo backend.
 Command needle-ask is a one-shot Needle 3 tool matcher built on the cgo backend.
engine
cgo
Package cgo is a Go binding to the prebuilt Needle 3 C/C++ engine (libneedle.a from https://huggingface.co/Cactus-Compute/needle3).
 Package cgo is a Go binding to the prebuilt Needle 3 C/C++ engine (libneedle.a from https://huggingface.co/Cactus-Compute/needle3).
pure
Package pure is a pure-Go port of the Needle 3 runtime: it parses the .cact container, dequantizes CQ weights, runs the Simple Attention Network forward pass, and decodes greedily.
 Package pure is a pure-Go port of the Needle 3 runtime: it parses the .cact container, dequantizes CQ weights, runs the Simple Attention Network forward pass, and decodes greedily.
examples
needle-cgo command
Command needle-cgo demonstrates the cgo-backed Needle 3 agent with the model embedded in the binary.
 Command needle-cgo demonstrates the cgo-backed Needle 3 agent with the model embedded in the binary.
needle-pure command
Command needle-pure demonstrates the cgo-backed Needle 3 agent with the model embedded in the binary.
 Command needle-pure demonstrates the cgo-backed Needle 3 agent with the model embedded in the binary.
Package model embeds the needle3.cact weights so a binary can carry the whole model in memory with no runtime file dependencies.
 Package model embeds the needle3.cact weights so a binary can carry the whole model in memory with no runtime file dependencies.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.