reranker

package
v0.1.0 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Aug 19, 2026 License: MIT Imports: 12 Imported by: 0

README

reranker

Fine-ranking (reranking) of coarse retrieval results. All rerankers implement the Reranker interface and plug into pipeline.RAGPipeline.WithReranker or index.SearchOptions.

Rerankers

Reranker Constructor Mechanism
SparseReranker NewSparseReranker() BM25-based rescoring (fast, no model)
CrossEncoderReranker NewCrossEncoderReranker(cfg) / NewCrossEncoderFile(path, cfg) ONNX cross-encoder (local, zero CGO); pluggable tokenizer
LLMReranker NewLLMReranker(cfg) LLM-judge scoring via an llm.Backend
LTRanker NewLTRanker(cfg) pointwise learning-to-rank over DefaultFeatures
AdaptiveLTRanker NewAdaptiveLTRanker(cfg) LTR with adaptive feature weights
EnsembleReranker NewEnsembleReranker(rerankers...) score-fuses several rerankers

Training data & experimentation

  • RelevanceSample / MarkRelevantIDs / MarkTopRelevant — capture labeled examples for LTR training.
  • NewABTest(cfg) / Experiment — run A/B comparisons between reranker variants (VariantMetrics, NDCG at DefaultNDCGCutoff).

Documentation

Overview

Package reranker implements fine-ranking (reranking) of coarse retrieval results to improve top-k precision. All rerankers are pure Go with no CGO or external model downloads: the cross-encoder runs on the bundled ONNX runtime, the sparse reranker reuses the BM25 package, the LLM reranker consumes an injected llm.Backend, and the ensemble fuses several rerankers via the fuse package.

Index

Constants

View Source 
const DefaultLLMRerankSystemPrompt = `` /* 303-byte string literal not displayed */

DefaultLLMRerankSystemPrompt is the system prompt used by LLMReranker when none is supplied.

View Source 
const DefaultLLMRerankUserPrompt = "Query:\n%s\n\nPassage:\n%s\n\nRelevance score (0-10):"

DefaultLLMRerankUserPrompt is the user prompt template. Placeholders are substituted with the query and passage.

View Source 
const DefaultNDCGCutoff = 3

DefaultNDCGCutoff is the cutoff K used for NDCG when the experiment config does not specify one.

Variables

This section is empty.

Functions

func DefaultFeatures 

func DefaultFeatures(query string, res index.SearchResult) []float64

DefaultFeatures returns the built-in feature vector: coarse retrieval score, query-term overlap fraction, and an inverse-length term.

Types

type ABConfig 

type ABConfig struct {
	// NDCGCutoff is the K used for the NDCG@K metric. Defaults to
	// DefaultNDCGCutoff.
	NDCGCutoff int
}

ABConfig configures an ABTest.

type AdaptiveConfig 

type AdaptiveConfig struct {
	// LTR configures the underlying LTRanker (features, learning rate,
	// epochs, L2). Zero values use the LTRanker defaults.
	LTR LTRConfig

	// RefitThreshold is the number of accumulated feedback examples that
	// triggers an automatic refit. Must be >= 1. Defaults to 100.
	RefitThreshold int
}

AdaptiveConfig configures an AdaptiveLTRanker.

type AdaptiveLTRanker 

type AdaptiveLTRanker struct {
	// contains filtered or unexported fields
}

AdaptiveLTRanker is a feedback-driven reranker: it wraps an LTRanker and adapts its weights from live relevance feedback (e.g. clicks, explicit thumbs up/down, or human judgments) instead of a static training set.

Call Fit once with an initial labeled corpus, then feed fresh judgments through RecordFeedback. Each call appends the examples to an internal buffer; once the buffer reaches RefitThreshold the model is retrained on the full accumulated set. The reranker always returns a valid ranking, so it is safe to keep wired into a pipeline while it learns.

All methods are safe for concurrent use.

func NewAdaptiveLTRanker 

func NewAdaptiveLTRanker(cfg AdaptiveConfig) *AdaptiveLTRanker

NewAdaptiveLTRanker creates a feedback-driven LTRanker.

func (*AdaptiveLTRanker) ExamplesFitted 

func (a *AdaptiveLTRanker) ExamplesFitted() int

ExamplesFitted returns the number of examples the current model was trained on.

func (*AdaptiveLTRanker) FeedbackRecorded 

func (a *AdaptiveLTRanker) FeedbackRecorded() int

FeedbackRecorded returns the total number of feedback examples accepted through RecordFeedback since construction.

func (*AdaptiveLTRanker) Fit 

func (a *AdaptiveLTRanker) Fit(ctx context.Context, examples []LTRExample) error

Fit trains the initial model on the given labeled examples. It can also be called again later to reset the accumulated corpus.

func (*AdaptiveLTRanker) Fitted 

func (a *AdaptiveLTRanker) Fitted() bool

Fitted reports whether the underlying LTRanker has been trained.

func (*AdaptiveLTRanker) Name 

func (a *AdaptiveLTRanker) Name() string

Name implements Reranker.

func (*AdaptiveLTRanker) RecordFeedback 

func (a *AdaptiveLTRanker) RecordFeedback(ctx context.Context, examples []LTRExample) (int, error)

RecordFeedback accepts new labeled examples observed since the last call. The examples are appended to the internal buffer and, once the buffer reaches the configured threshold, the model is retrained on the full accumulated set (initial Fit examples plus all recorded feedback). It returns the total number of examples the current model was trained on.

func (*AdaptiveLTRanker) RefitNow 

func (a *AdaptiveLTRanker) RefitNow(ctx context.Context) (int, error)

RefitNow retrains the model immediately on the full accumulated example set (initial Fit examples plus all recorded feedback) and returns the total number of examples used.

func (*AdaptiveLTRanker) Rerank 

func (a *AdaptiveLTRanker) Rerank(ctx context.Context, query string, results []index.SearchResult) ([]index.SearchResult, error)

Rerank implements Reranker, delegating to the current model.

type CrossEncoderConfig 

type CrossEncoderConfig struct {
	// Model is a loaded ONNX cross-encoder (required).
	Model *onnx.Model

	// Tokenize converts a (query, passage) pair into the model's inputs
	// (required).
	Tokenize CrossEncoderTokenizerFunc

	// Output names the model output to read as the relevance score. When
	// empty, the model's last declared output is used.
	Output string

	// Sigmoid applies the logistic function to the raw model output. Cross
	// -encoders trained for classification typically emit logits that are
	// mapped through sigmoid to a [0,1] relevance probability.
	Sigmoid bool
}

CrossEncoderConfig configures a CrossEncoderReranker.

type CrossEncoderReranker 

type CrossEncoderReranker struct {
	// contains filtered or unexported fields
}

CrossEncoderReranker scores each (query, passage) pair with a lightweight cross-encoder running on the bundled pure-Go ONNX runtime. Unlike bi-encoder similarity, a cross-encoder attends jointly over the query and passage, which usually yields a sharper relevance signal for fine ranking.

func NewCrossEncoderFile 

func NewCrossEncoderFile(path string, cfg CrossEncoderConfig) (*CrossEncoderReranker, error)

NewCrossEncoderFile loads a cross-encoder ONNX model from disk.

func NewCrossEncoderReranker 

func NewCrossEncoderReranker(cfg CrossEncoderConfig) (*CrossEncoderReranker, error)

NewCrossEncoderReranker creates a cross-encoder reranker from a loaded model.

func (*CrossEncoderReranker) Name 

func (r *CrossEncoderReranker) Name() string

Name implements Reranker.

func (*CrossEncoderReranker) Rerank 

func (r *CrossEncoderReranker) Rerank(ctx context.Context, query string, results []index.SearchResult) ([]index.SearchResult, error)

Rerank scores every result's chunk against the query with the cross-encoder and returns them ordered by the resulting relevance score.

type CrossEncoderTokenizerFunc 

type CrossEncoderTokenizerFunc func(query, passage string) (map[string]*onnx.Tensor, error)

CrossEncoderTokenizerFunc converts a (query, passage) pair into the named input tensors a cross-encoder ONNX model expects (e.g. "input_ids", "attention_mask", "token_type_ids"). Like the embedder's tokenizer it is dependency-injected so the reranker stays model-agnostic.

type EnsembleReranker 

type EnsembleReranker struct {
	// Rerankers is the set of sub-rerankers to run (at least one required).
	Rerankers []Reranker

	// Fusion, when set, combines the per-reranker score maps via the given
	// fusion strategy. When nil, Weights (if set) drive a weighted mean,
	// otherwise Reciprocal Rank Fusion is used as a robust default.
	Fusion fuse.Fusion

	// Weights optionally supplies one weight per sub-reranker for a weighted
	// mean of min-max-normalized scores. It is only used when Fusion is nil.
	Weights []float64

	// RRFK is the Reciprocal Rank Fusion constant used when no Fusion or
	// Weights is supplied. Defaults to 60.
	RRFK int
}

EnsembleReranker combines several rerankers into one by fusing their individual scores. This is useful when different rerankers are strong in different regimes (e.g. a cross-encoder for semantic fit plus a sparse reranker for exact-term matches).

func NewEnsembleReranker 

func NewEnsembleReranker(rerankers ...Reranker) (*EnsembleReranker, error)

NewEnsembleReranker creates an ensemble from the given rerankers.

func (*EnsembleReranker) Name 

func (r *EnsembleReranker) Name() string

Name implements Reranker.

func (*EnsembleReranker) Rerank 

func (r *EnsembleReranker) Rerank(ctx context.Context, query string, results []index.SearchResult) ([]index.SearchResult, error)

Rerank runs every sub-reranker over the same candidate set, fuses their per-chunk scores, and returns the results ordered by the fused score.

type Experiment 

type Experiment struct {
	// contains filtered or unexported fields
}

Experiment is an A/B test over rerankers: each arm is a named variant whose reranking quality is measured against labeled RelevanceSamples. Collect samples per arm (typically by running each variant's Rerank and labeling the results), then call Complete for a full comparison.

func NewABTest 

func NewABTest(cfg ABConfig) *Experiment

NewABTest creates an empty experiment.

func (*Experiment) AddVariant 

func (e *Experiment) AddVariant(name string, rr Reranker) error

AddVariant registers a named reranker variant. Names must be unique.

func (*Experiment) Complete 

func (e *Experiment) Complete(a, b string) (*ExperimentResult, error)

Complete finalizes the experiment by comparing the two named arms. Each arm must have at least one sample; arm order is irrelevant.

func (*Experiment) RecordSample 

func (e *Experiment) RecordSample(variant string, s RelevanceSample) error

RecordSample appends one labeled sample to the named variant.

func (*Experiment) Rerank 

func (e *Experiment) Rerank(ctx context.Context, variant, query string, results []index.SearchResult) ([]index.SearchResult, error)

Rerank runs the named variant's reranker over the given candidates, returning the ranking so it can be labeled and fed back via RecordSample.

func (*Experiment) SampleCount 

func (e *Experiment) SampleCount(variant string) (int, error)

SampleCount returns the number of labeled samples recorded for the named variant.

type ExperimentResult 

type ExperimentResult struct {
	// A and B are the metrics for each arm.
	A, B VariantMetrics
	// WinRateA is the fraction of pairwise sample comparisons where A's
	// NDCG strictly exceeds B's.
	WinRateA float64
	// TStat and PValue come from a Welch t-test on the per-sample NDCG
	// values (A vs B). PValue is approximated with the normal CDF.
	TStat  float64
	PValue float64
	// Significant reports whether PValue < 0.05.
	Significant bool
}

ExperimentResult is the completed comparison of two arms.

type FeatureFunc 

type FeatureFunc func(query string, res index.SearchResult) []float64

FeatureFunc computes the feature vector for a (query, result) pair. The vector length must be constant across all calls for a given reranker.

type LLMReranker 

type LLMReranker struct {
	// contains filtered or unexported fields
}

LLMReranker uses an LLM as a judge to score each candidate passage's relevance to the query. It is the highest-quality (and highest-cost) reranker in the package; the LLM backend is dependency-injected so no specific provider is required.

func NewLLMReranker 

func NewLLMReranker(cfg LLMRerankerConfig) (*LLMReranker, error)

NewLLMReranker creates an LLMReranker. It returns an error if no backend is provided.

func (*LLMReranker) Name 

func (r *LLMReranker) Name() string

Name implements Reranker.

func (*LLMReranker) Rerank 

func (r *LLMReranker) Rerank(ctx context.Context, query string, results []index.SearchResult) ([]index.SearchResult, error)

Rerank asks the LLM to score each candidate passage and returns the results ordered by the judge's score (0-1 normalized to 0-1). Candidates beyond MaxCandidates (when set) are appended at the tail in coarse order with the coarse score used as their RerankScore.

type LLMRerankerConfig 

type LLMRerankerConfig struct {
	// Backend is the LLM used to judge relevance (required).
	Backend llm.Backend

	// SystemPrompt overrides the judge's system message. Defaults to
	// DefaultLLMRerankSystemPrompt when empty.
	SystemPrompt string

	// UserPrompt overrides the per-passage prompt template. It must contain
	// two verbs for the query and passage placeholders, in that order.
	// Defaults to DefaultLLMRerankUserPrompt when empty.
	UserPrompt string

	// MaxCandidates limits how many top coarse results are judged. The rest
	// keep their coarse ordering at the tail. Zero means judge all.
	MaxCandidates int

	// Temperature controls sampling (0 recommended for deterministic judging).
	Temperature float64
}

LLMRerankerConfig configures an LLMReranker.

type LTRConfig 

type LTRConfig struct {
	// Features overrides the default feature extraction. When nil, the
	// built-in features (coarse score, keyword overlap, length) are used.
	Features FeatureFunc

	// LearningRate is the gradient-descent step size. Defaults to 0.5.
	LearningRate float64

	// Epochs is the number of full passes over the training set. Defaults to 200.
	Epochs int

	// L2 is the L2 weight-decay coefficient. Defaults to 0.001.
	L2 float64
}

LTRConfig configures an LTRanker's training.

type LTRExample 

type LTRExample struct {
	// Query is the retrieval query.
	Query string
	// Result is the candidate (its coarse Score and Chunk are used by the
	// default feature function).
	Result index.SearchResult
	// Label is the ground-truth relevance in [0,1].
	Label float64
}

LTRExample is a single labeled training example: a (query, result) pair and a relevance label in [0,1] (1 = relevant, 0 = not).

type LTRanker 

type LTRanker struct {
	// contains filtered or unexported fields
}

LTRanker is a pointwise learning-to-rank model: a logistic-regression classifier over per-candidate features. After Fit, it scores candidates by the learned relevance probability. Before any training it acts as an identity reranker (coarse score preserved), so it is always safe to wire in.

func NewLTRanker 

func NewLTRanker(cfg LTRConfig) *LTRanker

NewLTRanker creates an LTRanker from a config.

func (*LTRanker) Fit 

func (r *LTRanker) Fit(_ context.Context, examples []LTRExample) error

Fit trains the logistic model on the given labeled examples using full-batch gradient descent with L2 decay.

func (*LTRanker) Fitted 

func (r *LTRanker) Fitted() bool

Fitted reports whether the model has been trained.

func (*LTRanker) Name 

func (r *LTRanker) Name() string

Name implements Reranker.

func (*LTRanker) Rerank 

func (r *LTRanker) Rerank(_ context.Context, query string, results []index.SearchResult) ([]index.SearchResult, error)

Rerank scores each candidate with the learned model (or the coarse score when unfitted) and returns them ordered by the relevance score.

type RankedRelevance 

type RankedRelevance struct {
	// ChunkID identifies the chunk.
	ChunkID string
	// Relevant is 1 for relevant, 0 for not.
	Relevant float64
}

RankedRelevance pairs a chunk with its ground-truth relevance.

type RelevanceSample 

type RelevanceSample struct {
	// Query is the query this sample belongs to.
	Query string
	// Ranked is the candidate set in the order the variant ranked them,
	// each with its binary relevance.
	Ranked []RankedRelevance
}

RelevanceSample is one query's labeled evaluation: the candidate set and the ground-truth relevance of each candidate in the order the variant ranked them. Relevance uses the same binary convention as LTRExample (1 = relevant, 0 = not).

func MarkRelevantIDs 

func MarkRelevantIDs(query string, chunkIDs, relevantIDs []string) RelevanceSample

MarkRelevantIDs returns a sample for the given ordered ranking in which exactly the listed chunk IDs are marked relevant.

func MarkTopRelevant 

func MarkTopRelevant(query string, chunkIDs []string) RelevanceSample

MarkTopRelevant returns a sample that marks the first chunk of the ranking as relevant and every other chunk as not.

type Reranker 

type Reranker interface {
	// Rerank reorders and scores the given results for the query.
	Rerank(ctx context.Context, query string, results []index.SearchResult) ([]index.SearchResult, error)

	// Name returns a stable identifier for this reranker, recorded in
	// SearchResult.Reranker for score attribution.
	Name() string
}

Reranker is the interface satisfied by all fine-ranking strategies.

Rerank takes a query and an already-retrieved (coarse) list of results and returns a new, reordered list. Implementations MUST:

  • preserve the same set of chunks (no additions, no drops),
  • set each result's RerankScore, RerankRank, and Reranker fields,
  • return results sorted by RerankScore descending (RerankRank 1 = best).

The original Score (coarse retrieval score) is left untouched so callers can compare coarse vs fine ranking.

type SparseReranker 

type SparseReranker struct {
	// Config are the BM25 saturation / length parameters.
	Config bm25.Config

	// KeepUnscored leaves chunks that scored zero (no keyword overlap) at
	// the end of the list, ranked by their coarse score. When false, only
	// chunks with a positive BM25 score are kept.
	KeepUnscored bool
}

SparseReranker is a BM25-based reranker that re-scores coarse results by keyword relevance. It builds a BM25 index over just the candidate chunks and uses BM25 scores as the fine-rank score, which tends to promote exact-term matches that dense vectors can under-rank.

func NewSparseReranker 

func NewSparseReranker() *SparseReranker

NewSparseReranker creates a SparseReranker with default BM25 parameters.

func (*SparseReranker) Name 

func (r *SparseReranker) Name() string

Name implements Reranker.

func (*SparseReranker) Rerank 

func (r *SparseReranker) Rerank(_ context.Context, query string, results []index.SearchResult) ([]index.SearchResult, error)

Rerank re-scores the given results with BM25 and returns them ordered by the resulting score. Results are never dropped when KeepUnscored is true; otherwise chunks with zero keyword overlap are filtered out.

type VariantMetrics 

type VariantMetrics struct {
	// Name is the variant name.
	Name string
	// Samples is the number of labeled samples this arm was scored on.
	Samples int
	// NDCGAtK is the mean NDCG@K across samples.
	NDCGAtK float64
	// MRRAtK is the mean reciprocal rank of the first relevant result.
	MRRAtK float64
	// PrecisionAtK is the fraction of relevant results in the top K.
	PrecisionAtK float64
}

VariantMetrics holds the retrieval-quality metrics for one arm.

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.