semindex

Documentation

Overview

Package semindex is Aura's reusable embedding-index core: one shared lock-free pure-math layer (cosine/centroid/margin/bank ops over [][]float64) plus two typed wrappers — Classifier (Centroid mode: argmax over per-group means → best label + top-2 Margin) and Ranker (PerItem mode: top-K cosine over individual docs → []Scored). Both wrappers hold the Embedder seam and one sync.RWMutex; the math core stays pure (no ctx, no I/O, no lock). Brute-force cosine only — no ANN/HNSW (Req-8): the immutable, small banks make a naive O(N) scan sub-millisecond.

Index

• func Normalize(v []float64) []float64
• type Classifier
  • func NewClassifier(embed Embedder) *Classifier
  • func (c *Classifier) Add(ctx context.Context, label string, texts ...string) error
  • func (c *Classifier) AddVecs(label string, vecs ...[]float64)
  • func (c *Classifier) GroupCosine(label string, vec []float64) (float64, bool)
  • func (c *Classifier) Rank(ctx context.Context, text string) (Verdict, error)
  • func (c *Classifier) RankVecs(vec []float64) Verdict
• type Embedder
• type Item
• type RankResult
• type Ranker
  • func NewRanker(embed Embedder) *Ranker
  • func (r *Ranker) Add(ctx context.Context, texts ...string) error
  • func (r *Ranker) AddVecs(items ...Item)
  • func (r *Ranker) Len() int
  • func (r *Ranker) Rank(ctx context.Context, query string, k int) RankResult
  • func (r *Ranker) RankVecs(vec []float64, k int) []Scored
• type Scored
• type Verdict

Functions

func Normalize

func Normalize(v []float64) []float64

Normalize is the exported L2-normalization a consumer needs when it must hold the normalized query vector itself (e.g. the reasoning classifier feeds it to its self-improvement learner) without duplicating the core math. RankVecs/ RankVecs already normalize internally, so Normalize is idempotent there.

Types

type Classifier

type Classifier struct {
	// contains filtered or unexported fields
}

Classifier is the Centroid-mode wrapper (D-01): it folds each group's exemplars into a per-group mean (centroid) and answers a query with the argmax label plus the top-2 Margin. It holds the Embedder seam and one sync.RWMutex; the math stays in the lock-free core. Mirrors the reasoning classifier's anchor bank (reasoning_classifier.go:95-209) — same Centroid mode, made label-generic. Margin lives on this result ONLY, never on Ranker.

func NewClassifier

func NewClassifier(embed Embedder) *Classifier

NewClassifier returns a Classifier over the given embedder, or nil if embed is nil (callers treat nil as "no classifier wired").

func (*Classifier) Add

func (c *Classifier) Add(ctx context.Context, label string, texts ...string) error

Add embeds texts and folds them into the group (D-01 text-in door). A build failure is NOT persisted: nothing is added on error, so a retry self-heals (mirror of ensureAnchors). Returns the embedder error verbatim.

func (*Classifier) AddVecs

func (c *Classifier) AddVecs(label string, vecs ...[]float64)

AddVecs folds caller-provided vectors into the group's exemplar bank under the write lock. The group centroid is invalidated so the next Rank recomputes it.

func (*Classifier) GroupCosine

func (c *Classifier) GroupCosine(label string, vec []float64) (float64, bool)

GroupCosine returns the cosine of a query vector to ONE named group's centroid (lazily memoizing that centroid), plus whether the group exists. Unlike RankVecs (which returns the argmax across all groups), this scores a single caller-chosen label — the lever a stage-2 per-tool boost needs: "how close is this query to THIS tool's learned centroid?". The query is L2-normalized internally (parity with RankVecs). A missing group or an empty bank returns (0, false).

func (*Classifier) Rank

func (c *Classifier) Rank(ctx context.Context, text string) (Verdict, error)

Rank embeds a single query text and ranks it (D-01 text-in door). The embedder error is surfaced verbatim (Req-6 error surface).

func (*Classifier) RankVecs

func (c *Classifier) RankVecs(vec []float64) Verdict

RankVecs scores a query vector against every group centroid and returns the argmax Verdict (label + score + top-2 Margin). An empty bank yields Ok=false.

type Embedder

type Embedder interface {
	Embed(ctx context.Context, texts []string) ([][]float64, error)
}

Embedder is the narrow embedding seam the index needs. It matches documents.EmbeddingClient.Embed exactly so the production granite sidecar client satisfies it without an adapter. Owned here (the shared core) so both the tool ranker and the reasoning classifier depend on one interface; prompt.Embedder becomes a type alias of this (D-01, no-adapter preserved).

type Item

type Item struct {
	Label string
	Vec   []float64
}

Item is one PerItem bank entry: a label paired with its (caller-provided or embedded) vector. The Ranker appends these incrementally (D-03).

type RankResult

type RankResult struct {
	Results []Scored
	Err     error
}

RankResult is the text-in Rank result: the top-K Results plus any embedder Err (Req-6 error surface — never a silent empty ranking on infra failure).

type Ranker

type Ranker struct {
	// contains filtered or unexported fields
}

Ranker is the PerItem-mode wrapper (D-01): top-K cosine over individual document vectors. It holds the Embedder seam and one sync.RWMutex; Add/AddVecs take the write lock and APPEND to the bank (the D-03 incremental door), while Rank/RankVecs take the read lock. The top-K selection mirrors bm25 rank (bm25.go:130-145): stable sort desc with a deterministic insertion-index tie-break — load-bearing for top-K stability and the byte-stable manifest. No Margin here — that lives on the Classifier only.

func NewRanker

func NewRanker(embed Embedder) *Ranker

NewRanker returns a Ranker over the given embedder, or nil if embed is nil.

func (*Ranker) Add

func (r *Ranker) Add(ctx context.Context, texts ...string) error

Add embeds texts and appends them as items labeled by their own text (D-01 text-in door). On an embedder error nothing is appended and the error is returned verbatim (Req-6). The label is the source text so the caller can map a result back to its query/tool string.

func (*Ranker) AddVecs

func (r *Ranker) AddVecs(items ...Item)

AddVecs appends caller-provided items to the bank under the write lock, L2-normalizing each vector (the D-03 incremental-append door).

func (*Ranker) Len

func (r *Ranker) Len() int

Len reports the current bank size (the incremental-append assertion lever).

func (*Ranker) Rank

func (r *Ranker) Rank(ctx context.Context, query string, k int) RankResult

Rank embeds a single query text and returns its top-K (D-01 text-in door). The embedder error is carried on RankResult.Err (Req-6), never swallowed.

func (*Ranker) RankVecs

func (r *Ranker) RankVecs(vec []float64, k int) []Scored

RankVecs returns the top-K items by cosine to the query vector, descending, ties broken by ascending insertion index (deterministic). k<=0 or k>bank returns all matching items. An empty bank returns an empty slice.

type Scored

type Scored struct {
	Label string
	Score float64
}

Scored pairs a label with its cosine score against a query. It is the Ranker's PerItem result shape (mirrors the bm25 scoredDoc analog, exported with a label). Margin lives on the Classifier result only — it does not pollute this.

type Verdict

type Verdict struct {
	Label  string
	Score  float64
	Margin float64
	Ok     bool
}

Verdict is the Classifier's Centroid result: the argmax label, its score, and the top-2 Margin (best - second). Ok is false when the bank is empty or the query vector cannot be scored, so callers can fall back.