Help us shape the future of Go! Take the 2026 Go Developer Survey

indecis

package module
v0.1.0 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Sep 28, 2026 License: MIT Imports: 23 Imported by: 0

README

indecis

indecis builds small decision models in Go: a pretrained text encoder, fully fine-tuned on your examples, that answers typed questions with calibrated probabilities instead of generated text.

bin/indecis synth -templates examples/guide/templates -n 900 -out data.jsonl
bin/indecis train -backbone $BEKKO -schema examples/guide/schema.json -train data.jsonl -out model
echo "Mon colis n'est pas arrivé, je veux parler à quelqu'un." | bin/indecis predict -model model

{"answers": {"sujet": {"choice": "livraison", …}, "urgence": {"score": 2.0, …}, "humain": {"p": 1.0, …}}, …}

There are three question types: yes/no (noul), one option among several (choice), and a level on an ordered scale (score). One pass of the model answers all of them.

In numbers

Measured on a laptop (Core Ultra 7 265U) with the bekko-embedding-v1-a8m backbone (7.7M parameters, multilingual):

  • 1.5 ms to judge a 15-token sentence, 23 ms for 256 tokens, on a single core;
  • 18 to 30 MB of memory for a model ready to serve, and a 58 to 290 MB file;
  • half an hour to train on 20,000 examples;
  • no GPU, no cgo, no dependency outside the Go standard library.

Documentation

Status

Experimental: the API may still change. Requires Go 1.27. The experimental SIMD support (GOEXPERIMENT=simd) makes it fast; indecis also runs without it. License: MIT.

Documentation

Overview

Package indecis builds small decision models: a pretrained text encoder, fully fine-tuned in pure Go, that answers typed questions with calibrated probabilities instead of generated text.

A Schema lists the questions, of three types:

  • noul (NewNoul): a yes/no question, answered with P(yes);
  • choice (NewChoice): one option among several, answered with a distribution;
  • score (NewScore): a level on an ordered scale, answered with the expected level and its distribution.

One pass of the encoder answers every question of a schema: each question is one more head on the same representation of the text. 

m, _ := indecis.New(backboneDir, schema, 1)
_ = m.Fit(ctx, train, indecis.DefaultTrainOptions())
_, _ = m.Calibrate(ctx, calib)
_ = m.Save("model")

m, _ = indecis.Load("model", indecis.WithInt8())
d, _ := m.Decide(ctx, "My parcel has not arrived")

In open mode, the options are described in text at call time and compared with the text through embeddings (Model.ChooseNearest, Model.PrepareCandidates, Model.DecideOpen); Model.FitEmbeddings fine-tunes the encoder for it.

The guides in the docs directory of the repository cover the whole process, from the schema to the deployment.

Index

Constants

View Source 
const DefaultEmbedScale = 20

DefaultEmbedScale multiplies the cosines before the softmax (inverse of a temperature), the usual value in sentence-transformers.

View Source 
const NoulAnchor = "Something else"

NoulAnchor describes the "false" option of OpenNoul.

Variables

This section is empty.

Functions

func CandidateContext 

func CandidateContext(c Candidate) string

CandidateContext puts an option in the form the model reads as context, both in training and inference.

func CandidatePairs 

func CandidatePairs(question, text string, correct Candidate, others []Candidate) []dataset.Example

CandidatePairs turns a text whose correct option is correct into examples for the Noul question named question: one positive, and one negative per other option.

Types

type Answer 

type Answer struct {
	Question string `json:"question"`
	Kind     Kind   `json:"kind"`
	// P is the probability that the answer is "true" (Noul).
	P float64 `json:"p,omitempty"`
	// Choice is the most probable option (Choice), or the most probable
	// level (Score).
	Choice string `json:"choice,omitempty"`
	// Score is the expected level, between 0 and the number of levels
	// minus one.
	Score float64 `json:"score,omitempty"`
	// Probs is the distribution over the options or the levels.
	Probs map[string]float64 `json:"probs,omitempty"`
	// Confidence is the probability of the chosen answer.
	Confidence float64 `json:"confidence"`
	// Margin (Choice) is the gap between the chosen probability and the
	// average of the others, CLM's confidence: close to 0 when the
	// options are equally likely, even if there are many of them.
	Margin float64 `json:"margin,omitempty"`
}

Answer is the answer to a question.

type Candidate 

type Candidate struct {
	Name        string   `json:"name"`
	Description string   `json:"description,omitempty"`
	Examples    []string `json:"examples,omitempty"`
}

Candidate is an option described in natural language. Examples are texts that fall under the option: ChooseNearest uses them to locate the option (a few examples are often worth more than a long description); ChooseAmong ignores them.

type CandidateSet 

type CandidateSet struct {
	// contains filtered or unexported fields
}

CandidateSet is a list of options prepared for ChooseIn: the embedding of each option is computed once. It remains valid as long as the model is not retrained.

func (*CandidateSet) Candidates 

func (s *CandidateSet) Candidates() []Candidate

Candidates returns the options of the list.

type ChoiceBatch 

type ChoiceBatch struct {
	Texts      []string
	Candidates []Candidate
	Correct    [][]int
}

ChoiceBatch is a training batch: texts and a list of shared options. Correct[i] gives the right options for text i; the first is the target, the others are excluded from the comparison (a correct option is not an error).

func ChoiceBatches 

func ChoiceBatches(examples []dataset.Example, questions []OpenQuestion, size int, seed int64) []ChoiceBatch

ChoiceBatches forms the FitEmbeddings batches from labeled examples: for each question, the examples that carry its label are grouped into batches of size texts, with all the question's options as the batch's options. A label is an option name, a distribution (the most probable option is kept), a level index (Score), or a boolean (Noul: the first option for true, the second for false).

type Decision 

type Decision map[string]Answer

Decision groups a text's answers, by question name.

type Info 

type Info struct {
	Backbone string `json:"backbone,omitempty"`
	// TrainPrior is, for each Noul question, the proportion of "true"
	// answers in the training corpus. This is what is needed to correct
	// the probability when the proportion differs in production (see
	// calibrate.PriorShift).
	TrainPrior map[string]float64 `json:"train_prior,omitempty"`
	Steps      int                `json:"steps,omitempty"`
	// EmbedScale multiplies the cosines in ChooseNearest (0: the default
	// value).
	EmbedScale float64 `json:"embed_scale,omitempty"`
}

Info describes a model's origin.

type Input 

type Input struct {
	Context string
	Text    string
}

Input is a text to judge and its optional context.

type Kind 

type Kind string

Kind is the type of a question. 

const (
	Noul   Kind = "noul"
	Choice Kind = "choice"
	Score  Kind = "score"
)

type Metrics 

type Metrics struct {
	Question string `json:"question"`
	Kind     Kind   `json:"kind"`
	N        int    `json:"n"`
	// Accuracy: most probable answer = label. For Noul, threshold 0.5;
	// for a soft label, it is its rounding.
	Accuracy float64 `json:"accuracy"`
	// Noul: precision, recall and F1 of the "true" class, area under the
	// ROC curve.
	Precision float64 `json:"precision,omitempty"`
	Recall    float64 `json:"recall,omitempty"`
	F1        float64 `json:"f1,omitempty"`
	AUC       float64 `json:"auc,omitempty"`
	// Choice: average F1 over the options present.
	MacroF1 float64 `json:"macro_f1,omitempty"`
	// Score: mean absolute deviation between expected level and actual level.
	MAE float64 `json:"mae,omitempty"`
	// NLL is the mean negative log-likelihood, Brier the Brier score
	// (Noul), ECE the calibration error over 10 confidence bins.
	NLL   float64 `json:"nll"`
	Brier float64 `json:"brier,omitempty"`
	ECE   float64 `json:"ece"`
}

Metrics measures a question over a set of examples.

func (Metrics) String 

func (mt Metrics) String() string

String presents the metrics on one line.

type Model 

type Model struct {
	// contains filtered or unexported fields
}

Model is a decision model: an encoder and one head per question.

func Load 

func Load(dir string, opts ...Option) (*Model, error)

Load reads a model written by Save.

func New 

func New(backboneDir string, schema Schema, seed int64, opts ...Option) (*Model, error)

New creates an untrained model from a backbone in transformers format (config.json, model.safetensors, tokenizer.json). The heads are randomly initialized with seed.

func Open 

func Open(dir string, opts ...Option) (*Model, error)

Open loads a model written by Save, or, if dir does not contain one (no indecis.json), a backbone in transformers format: it then only answers open questions (ChooseNearest, DecideOpen), without training. Its learned question, "match", is only an untrained anchor point.

func (*Model) Calibrate 

func (m *Model) Calibrate(ctx context.Context, examples []dataset.Example) (map[string]float64, error)

Calibrate tunes each question's temperature on examples distinct from those used in training, by minimizing the negative log-likelihood. Returns the temperatures used.

A question whose calibration examples are all perfectly classified keeps its temperature: these examples say nothing about the model's error, only that they are too easy.

func (*Model) ChooseAmong 

func (m *Model) ChooseAmong(ctx context.Context, question string, candidates []Candidate, texts ...string) ([]Answer, error)

ChooseAmong picks, for each text, the option that suits it best among candidates, using the paired model's Noul question named question.

Answer.Probs is the distribution over the options (softmax of the logits, including the question's temperature) and Answer.Choice the most probable one. Answer.P is the probability that this best option really suits the text: a low value signals a text that no option describes.

func (*Model) ChooseIn 

func (m *Model) ChooseIn(ctx context.Context, set *CandidateSet, texts ...string) ([]Answer, error)

ChooseIn ranks each text among the options of a prepared list. Answer.Probs is the softmax of the scaled cosines (Info.EmbedScale, DefaultEmbedScale by default). Answer.Score is the cosine of the chosen option: a threshold on this cosine, tuned on examples, separates texts that no option describes.

func (*Model) ChooseNearest 

func (m *Model) ChooseNearest(ctx context.Context, candidates []Candidate, texts ...string) ([]Answer, error)

ChooseNearest picks, for each text, the option whose prototype is closest (see PrepareCandidates and ChooseIn).

func (*Model) Decide 

func (m *Model) Decide(ctx context.Context, texts ...string) ([]Decision, error)

Decide answers all the questions for each text, without context.

func (*Model) DecideInputs 

func (m *Model) DecideInputs(ctx context.Context, inputs ...Input) ([]Decision, error)

DecideInputs answers all the questions for each input.

func (*Model) DecideOpen 

func (m *Model) DecideOpen(ctx context.Context, questions []OpenQuestion, texts ...string) ([]Decision, error)

DecideOpen answers open questions for each text. The probabilities come from the softmax of the cosines (see ChooseIn): their calibration depends on the model, and a threshold is tuned on examples. WithEmbedCache avoids re-encoding the options from one call to the next.

func (*Model) Embed 

func (m *Model) Embed(ctx context.Context, texts ...string) ([][]float32, error)

Embed returns the embedding of each text: the mean of the encoder's states, normalized (norm 1). The dot product of two embeddings is their cosine similarity. Each text is encoded alone, even by a paired model: on the non-fine-tuned backbone, it is the original model's sentence embedding.

func (*Model) Evaluate 

func (m *Model) Evaluate(ctx context.Context, examples []dataset.Example) ([]Metrics, error)

Evaluate measures the model, including temperatures.

func (*Model) Fit 

func (m *Model) Fit(ctx context.Context, examples []dataset.Example, opts TrainOptions) error

Fit fine-tunes the whole model, encoder included, on the examples.

func (*Model) FitEmbeddings 

func (m *Model) FitEmbeddings(ctx context.Context, batches []ChoiceBatch, opts TrainOptions) error

FitEmbeddings fine-tunes the encoder for ChooseNearest. opts.HeadLR and opts.Dropout are ignored: no head is involved. opts.Symmetric adds the option -> texts direction to the loss.

func (*Model) Info 

func (m *Model) Info() Info

Info returns the model's origin.

func (*Model) Logits 

func (m *Model) Logits(ctx context.Context, texts ...string) ([]map[string][]float64, error)

Logits returns, for each text, the raw logits of each question, before temperature: a caller that applies its own calibration or its own prior correction starts from there (see Temperatures and Info).

func (*Model) LogitsInputs 

func (m *Model) LogitsInputs(ctx context.Context, inputs ...Input) ([]map[string][]float64, error)

LogitsInputs is Logits for inputs with context.

func (*Model) Paired 

func (m *Model) Paired() bool

Paired indicates whether the model reads pairs (context, text).

func (*Model) PrepareCandidates 

func (m *Model) PrepareCandidates(ctx context.Context, candidates []Candidate) (*CandidateSet, error)

PrepareCandidates computes the prototype of each option: the normalized average of the embedding of its name and its description and those of its examples.

func (*Model) PruneVocabulary 

func (m *Model) PruneVocabulary(corpus []Input, minCount int) (PruneStats, error)

PruneVocabulary reduces the vocabulary to the tokens that the corpus tokenization uses, including BPE intermediates, seen at least minCount times (1: all that appear). The embedding table, 93% of a small model's weights, shrinks accordingly: for use in a few languages, a multilingual vocabulary of 256,000 tokens reduces to a few tens of thousands.

A corpus text then tokenizes exactly as before; a text that would have needed a removed token tokenizes into smaller pieces, at worst into bytes. The corpus must therefore represent the texts the model will judge (languages, domains), and the effect is measured on texts outside the corpus (Evaluate before and after). The model is modified in place; Save writes the reduced tokenizer.

func (*Model) Save 

func (m *Model) Save(dir string) error

Save writes the model to dir. The directory stays readable by transformers: config.json and the encoder's weights keep their names.

func (*Model) Schema 

func (m *Model) Schema() Schema

Schema returns the model's schema.

func (*Model) Temperatures 

func (m *Model) Temperatures() map[string]float64

Temperatures returns the temperature of each question.

func (*Model) TokenIDs 

func (m *Model) TokenIDs(in Input) ([]int32, error)

TokenIDs returns the ids of the tokens the model reads for an input, truncation included.

func (*Model) Tokens 

func (m *Model) Tokens(in Input) (int, error)

Tokens returns the number of tokens an input occupies, truncation included: what the model actually reads.

type OpenQuestion 

type OpenQuestion struct {
	Name string
	Kind Kind
	// Instructions, if given, precede each option in what the model reads
	// ("Email urgency" before "high").
	Instructions string
	// Options: for Choice, the options; for Score, the levels from lowest
	// to highest; for Noul, exactly two criteria, the "true" one then the
	// "false" one.
	Options []Candidate
}

OpenQuestion is a question asked at inference time, without dedicated training: its possible answers are described in text and compared to the text being judged by embeddings (see ChooseIn). This is the form of the Jev and CLM questions.

func OpenNoul 

func OpenNoul(name, instructions string) OpenQuestion

OpenNoul builds an open yes/no question from its instructions alone: "true" is described by the instructions, "false" by a fixed anchor, "Something else". Opposing the instructions to their negation ("Yes: X" against "No: X") gives almost no signal, because the two embeddings are nearly identical. The anchor ranks texts much better, but its probabilities are not calibrated: tune a threshold on examples.

func (OpenQuestion) OptionIndex 

func (q OpenQuestion) OptionIndex(v any) int

OptionIndex returns the index of the option that a label designates (see ChoiceBatches), or -1.

type Option 

type Option func(*Model)

Construction options.

func WithBatching 

func WithBatching(workers, maxRows int) Option

WithBatching groups the computations of simultaneous requests: workers goroutines (0: one per core) each take the first pending request and all those already queued, up to maxRows positions (0: 512), and compute them in a single pass. Without load, a request leaves right away, without waiting; under load, the batches grow on their own and the larger matrix products are more efficient. The sequences of a batch are laid end to end without padding.

On CPU, the gain is small: each request already occupies a core, and requests of 100 to 256 tokens already make fairly large matrix products. Measured on the decision server (32 clients): a few % of throughput on single-sentence requests, nothing beyond that, for more memory.

func WithEmbedCache 

func WithEmbedCache(n int) Option

WithEmbedCache keeps in memory the embeddings of the n most recent distinct texts passed to Embed, and of the options prepared by PrepareCandidates (so ChooseNearest, DecideOpen): a reused option is not re-encoded. Texts judged by ChooseIn are not kept. The cache is cleared on every training run.

func WithInt8 

func WithInt8() Option

WithInt8 makes the encoder's layers compute in int8 (per-channel weights, per-token activations) when the processor has AVX-VNNI; without it, the option has no effect. Inference is about twice as fast and the layer weights four times smaller; the decisions of the injection detector (xolo-plugin-injection-guard) are unchanged on its reference set. The gap should still be checked for each model (Evaluate with and without the option).

func WithInt8Embeddings 

func WithInt8Embeddings() Option

WithInt8Embeddings makes Save write the embedding table in int8, one scale per row: half the size of bf16, on disk as well as in the pages read. Rows modified by training lose their exact value; the gap is measured with Evaluate before and after. A model saved this way is recognized on load and stays that way. Hugging Face tools do not read this format.

func WithMaxLen 

func WithMaxLen(n int) Option

WithMaxLen sets the maximum length in tokens; longer texts are truncated. 256 by default.

func WithPairs 

func WithPairs() Option

WithPairs makes the model read pairs (context, text): the system prompt and the message, for example. All inputs are then encoded as a pair, including an empty context, so that training and inference see the same shape. See tokenizer.EncodePair for truncation.

func WithThreads 

func WithThreads(n int) Option

WithThreads bounds the number of cores used by the computations (0: all). The setting applies to the whole process. At inference, a batch of fewer than 1024 positions is computed on a single core anyway, the fastest for one sentence; long texts and large batches spread out.

type Progress 

type Progress struct {
	Epoch, Step, Steps int
	Loss               float64 // mean loss of the step
	LR                 float64
	Tokens             int // tokens processed since the start
	Elapsed            time.Duration
}

Progress describes the training's progress.

type PruneStats 

type PruneStats struct {
	Before, After int // tokens before and after
}

PruneStats describes a vocabulary pruning.

type Question 

type Question struct {
	Name string `json:"name"`
	Kind Kind   `json:"kind"`
	// Instructions documents the question. The model does not read it: its
	// heads are learned, the question is in the data.
	Instructions string `json:"instructions,omitempty"`
	// Options lists the options of a Choice, or the levels of a Score in
	// increasing order.
	Options []string `json:"options,omitempty"`
}

Question describes a decision that the model learns to make.

func NewChoice 

func NewChoice(name, instructions string, options ...string) Question

func NewNoul 

func NewNoul(name, instructions string) Question

NewNoul, NewChoice and NewScore build the questions.

func NewScore 

func NewScore(name, instructions string, levels ...string) Question

type Schema 

type Schema []Question

Schema is the list of a model's questions.

func (Schema) Index 

func (s Schema) Index(name string) int

Index returns the position of a question, -1 if absent.

func (Schema) Validate 

func (s Schema) Validate() error

Validate checks the schema.

type TrainOptions 

type TrainOptions struct {
	Epochs    int
	BatchSize int
	// LR is the encoder's learning rate, HeadLR that of the heads, higher:
	// they start from scratch.
	LR, HeadLR  float64
	WeightDecay float64
	// Warmup is the fraction of steps spent raising the rate linearly,
	// which then decreases linearly to zero.
	Warmup float64
	// ClipNorm bounds the global gradient norm (0: no clipping).
	ClipNorm float64
	// Dropout applies to the pooled vector, before the heads.
	Dropout float64
	Seed    int64
	// Symmetric (FitEmbeddings) also optimizes the option -> texts
	// direction: each option in a batch must prefer its texts over the
	// others, like CLM's bidirectional InfoNCE loss.
	Symmetric bool
	// Progress, if provided, is called after each step.
	Progress func(Progress)
}

TrainOptions configures the fine-tuning.

func DefaultTrainOptions 

func DefaultTrainOptions() TrainOptions

DefaultTrainOptions are starting settings for a small encoder.

Source Files

View all Source files

Directories

Path Synopsis
Package calibrate combines probabilities in log-odds.
 Package calibrate combines probabilities in log-odds.
cmd
indecis command
Command indecis creates, evaluates and uses small decision models.
 Command indecis creates, evaluates and uses small decision models.
Package dataset defines the format of training examples and the splits that measure generalization.
 Package dataset defines the format of training examples and the splits that measure generalization.
synth
Gazetteers: weighted lists of values, taken from go-anon.
 Gazetteers: weighted lists of values, taken from go-anon.
internal
linalg
Package linalg provides the numerical kernels for training and inference: matrix product, vector operations.
 Package linalg provides the numerical kernels for training and inference: matrix product, vector operations.
modernbert
Package modernbert implements the ModernBERT encoder in pure Go, for both inference and training.
 Package modernbert implements the ModernBERT encoder in pure Go, for both inference and training.
optim
Package optim implements AdamW, identical to torch.optim.AdamW, and its sparse variant for embedding tables.
 Package optim implements AdamW, identical to torch.optim.AdamW, and its sparse variant for embedding tables.
safetensors
Package safetensors reads and writes the safetensors format: a JSON header describing each tensor, followed by raw little-endian data.
 Package safetensors reads and writes the safetensors format: a JSON header describing each tensor, followed by raw little-endian data.
Package tokenizer reproduces in pure Go the BPE tokenizer of the Gemma model family, as a Hugging Face tokenizer.json file describes it.
 Package tokenizer reproduces in pure Go the BPE tokenizer of the Gemma model family, as a Hugging Face tokenizer.json file describes it.
tools
infbench command
Command infbench measures the memory footprint and inference latency of an indecis model.
 Command infbench measures the memory footprint and inference latency of an indecis model.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.