confidence

Documentation

Overview

Package confidence derives a decision-confidence signal from a grammar- constrained model's per-token logprobs. With a grammar active, llama.cpp reports the RAW (pre-mask) distribution, so at the decision-value position the top alternatives reveal the model's genuine preference among the legal options. We aggregate that raw mass by legal class (folding spelling variants like "Yes"/"yes", dropping grammar-illegal tokens) and return the normalized margin between the top two classes — a low margin means the model was genuinely torn (e.g. an "eager-YES" borderline call) and the pipeline should escalate to a larger tier rather than accept it.

Index

• func Margin(toks []llamaclient.TokenLogprob, jsonKey string, classes []string) (float64, bool)

Functions

func Margin

func Margin(toks []llamaclient.TokenLogprob, jsonKey string, classes []string) (float64, bool)

Margin returns a 0..1 separation between the model's top-1 and top-2 legal decision classes at the value position of jsonKey (e.g. "decision" / "label"), computed on the raw logprob distribution. ok=false when the position or a usable class distribution can't be resolved — the caller then falls back to other signals (self-reported confidence) rather than escalating blindly.