index

Documentation

Index

• Variables
• func InValues(v any) []any
• func IsSemantic(idx Index) bool
• func ValidateKey(key string) error
• type Condition
  • func Eq(key string, value any) Condition
  • func Exists(key string) Condition
  • func Gt(key string, value any) Condition
  • func Gte(key string, value any) Condition
  • func In(key string, values ...any) Condition
  • func Lt(key string, value any) Condition
  • func Lte(key string, value any) Condition
  • func Ne(key string, value any) Condition
  • func NotExists(key string) Condition
• type ConditionallyFilterable
• type Filter
  • func (f Filter) CanonicalBytes() []byte
  • func (f Filter) Matches(meta map[string]any) bool
  • func (f Filter) Validate() error
• type FilterOp
• type FilterableIndex
  • func AsFilterable(idx Index) (FilterableIndex, bool)
• type Index
• type OptionFunc
  • func WithOnProgress(onProgress func(progress float32)) OptionFunc
• type Options
  • func NewOptions(funcs ...OptionFunc) *Options
• type SearchOptions
• type SearchResult
• type Semantic
• type Unwrapper

Variables

var ErrInvalidFilterKey = errors.New("invalid filter key")

ErrInvalidFilterKey is returned by Filter.Validate for a key outside KeyPattern. Wrapped errors match it with errors.Is.

var KeyPattern = regexp.MustCompile(`^[A-Za-z0-9_-]{1,128}$`)

KeyPattern is the set of accepted metadata keys in a filter condition.

Functions

func InValues

func InValues(v any) []any

InValues normalizes the operand of an OpIn condition into the candidate values it stands for. Backends translating a filter need it to expand the membership test the same way the Go evaluator does.

func IsSemantic

func IsSemantic(idx Index) bool

IsSemantic reports whether idx declares itself as a semantic (vector) index via the Semantic capability, unwrapping decorators along the way (see Unwrapper).

func ValidateKey

func ValidateKey(key string) error

ValidateKey reports whether key is usable as a metadata filter key.

Types

type Condition

type Condition struct {
	Key   string
	Op    FilterOp
	Value any
}

Condition is a single predicate over one metadata key.

func Eq

func Eq(key string, value any) Condition

Eq builds an equality condition (key == value).

func Exists

func Exists(key string) Condition

Exists builds a presence condition: the document carries the key, whatever its value.

func Gt

func Gt(key string, value any) Condition

Gt builds a strictly-greater-than condition (key > value).

func Gte

func Gte(key string, value any) Condition

Gte builds a greater-than-or-equal condition (key >= value).

func In

func In(key string, values ...any) Condition

In builds a membership condition (key ∈ values). Without values it matches nothing.

func Lt

func Lt(key string, value any) Condition

Lt builds a strictly-less-than condition (key < value).

func Lte

func Lte(key string, value any) Condition

Lte builds a less-than-or-equal condition (key <= value).

func Ne

func Ne(key string, value any) Condition

Ne builds an inequality condition (key present and key != value). It does not match documents lacking the key; combine with Or-less semantics in mind, or use NotExists for absence.

func NotExists

func NotExists(key string) Condition

NotExists builds an absence condition: the document does not carry the key. It is the way to select documents missing a metadata, since Ne requires presence.

type ConditionallyFilterable

type ConditionallyFilterable interface {
	// Filterable reports whether SearchFiltered can currently be relied upon.
	Filterable() bool
}

ConditionallyFilterable is implemented by indexes whose ability to push the filter down depends on their configuration — typically a composite index, which can only honour the FilterableIndex contract when every one of its legs can.

Declaring SearchFiltered is a static, all-or-nothing claim; this reports the runtime answer. It mirrors Semantic, which the pipeline consults the same way.

type Filter

type Filter []Condition

Filter is a conjunction (implicit AND) of conditions evaluated against a document's metadata. The zero value (nil / empty) matches everything.

Semantics

These rules are normative: they are the contract every implementation must honour, whether the filter is evaluated in Go over metadata loaded from the store or pushed down into a backend's search query. The shared conformance suite in index/filtertest encodes them case by case.

Evaluation is total: a condition never errors at match time, it matches or it does not.

• Key presence. A key is present when it appears in the metadata map, whatever its value — including a JSON null. Every operator except OpExists requires presence, OpNe included: Ne("author", "x") does not match a document without an "author" key. This is the SQL NULL-like reading, chosen because it is the one a SQL backend can express faithfully; use NotExists to select documents lacking a key.

• Numbers. All Go numeric types are unified into float64, on both sides and for every operator, so Eq("count", 3) matches the float64(3) that JSON decoding produces.

• Dates. time.Time values and RFC 3339 strings are canonicalized to UTC with fixed nanosecond precision (filternorm.CanonicalTimeLayout), on the write path and on the filter operand alike. Equality is then an exact string comparison and ordering a lexicographic one, which for canonical dates is chronological order.

• Strings. Compared exactly: case-sensitive, accent-sensitive. The unaccenting applied to full-text search does not extend to metadata.

• Mixed types. Comparing values of different kinds — Gt("count", "abc") against a numeric metadata, a bool against a string — never matches.

• Containers. A metadata value that is a slice or a map is present but never equal nor ordered, so Eq("tags", "go") does not match tags: ["go", "db"]. Membership in an array is deliberately left out of this version; it would need an explicit Contains operator.

• Empty In. In(key) without any value matches nothing, like SQL IN ().

Keys

Filter keys are restricted to KeyPattern. Metadata keys usually come from a caller-facing surface (CLI flags, the MCP search tool), and the restriction keeps them safely expressible as a JSON path in every backend. Validate reports offending keys as ErrInvalidFilterKey.

func (Filter) CanonicalBytes

func (f Filter) CanonicalBytes() []byte

CanonicalBytes returns a stable encoding of the filter, independent of the order in which the conditions were declared and of the Go types used for the operands.

Two filters that select the same documents encode identically: conditions are sorted, numeric operands collapse to float64 and dates to their canonical layout — the same normalization the evaluator applies — and the operand of OpIn is treated as the set it is. Callers use it to fingerprint a filter, for instance to detect that a paginated search was resumed with a different one.

An empty filter encodes to nil, not to an empty non-nil slice, so that "no filter" is unambiguous.

func (Filter) Matches

func (f Filter) Matches(meta map[string]any) bool

Matches reports whether meta satisfies every condition. An empty filter always matches; a nil meta has no key present, so it only satisfies NotExists conditions.

func (Filter) Validate

func (f Filter) Validate() error

Validate checks every condition's key against KeyPattern. It is called by the search pipeline before a filter is evaluated or translated to SQL; callers building filters from user input can call it earlier to report the problem at its source.

type FilterOp

type FilterOp string

FilterOp enumerates the comparison operators supported by a metadata Filter.

const (
	// OpEq matches when the metadata value equals the condition value.
	OpEq FilterOp = "eq"
	// OpNe matches when the key is present and its value differs from the
	// condition value. An absent key does NOT match (see Filter).
	OpNe FilterOp = "ne"
	// OpGt matches when the metadata value is strictly greater than the value.
	OpGt FilterOp = "gt"
	// OpGte matches when the metadata value is greater than or equal to the value.
	OpGte FilterOp = "gte"
	// OpLt matches when the metadata value is strictly less than the value.
	OpLt FilterOp = "lt"
	// OpLte matches when the metadata value is less than or equal to the value.
	OpLte FilterOp = "lte"
	// OpIn matches when the metadata value equals any element of the value slice.
	OpIn FilterOp = "in"
	// OpExists matches on key presence only, whatever the value. The condition
	// value is a bool: true requires the key, false requires its absence.
	OpExists FilterOp = "exists"
)

type FilterableIndex

type FilterableIndex interface {
	Index

	// SearchFiltered searches like Search, keeping only the sections whose
	// document metadata satisfies filter.
	SearchFiltered(ctx context.Context, query string, filter Filter, opts SearchOptions) ([]*SearchResult, error)
}

FilterableIndex is an optional capability: backends implementing it apply the metadata filter inside the search query instead of leaving it to a post-hoc pass in Go.

The point is top-k semantics. Filtering after the fact means asking a backend for k results and possibly keeping none of them, so the caller has to over-fetch and guess how much; a backend that filters inside its query returns k results that already satisfy the filter.

Contract

SearchFiltered behaves exactly like Search, restricted to the sections whose document metadata satisfies the filter. An empty (or nil) filter is equivalent to Search. SearchOptions.MaxResults bounds the results *after* filtering — that is the whole point of the capability.

The filter semantics an implementation must reproduce are specified on Filter, and encoded case by case in the shared conformance suite index/filtertest. A backend MUST pass that suite before advertising this capability: an implementation that disagrees with the Go evaluator on absent keys, on int-versus-float or on a date offset would silently return different documents than another backend for the same query, and nothing would fail loudly.

Implementations may assume the filter keys have been validated (see Filter.Validate), which the search pipeline does before calling. They must nonetheless treat keys and values as data — bind parameters, never string interpolation — since a validated key is not the same thing as a trusted one.

Note the capability is deliberately a dedicated method rather than a Filter field on SearchOptions: a backend that ignored such a field would return unfiltered results that the caller would believe filtered, which is precisely the class of silent corruption the type system should prevent.

func AsFilterable

func AsFilterable(idx Index) (FilterableIndex, bool)

AsFilterable reports whether idx natively supports filter push-down, unwrapping decorators that implement Unwrapper. It is the single detection point for the capability: callers must not type-assert FilterableIndex directly, or they will miss a decorated backend — or a composite one that declares the method but cannot currently honour it.

type Index

type Index interface {
	Index(ctx context.Context, document model.Document, funcs ...OptionFunc) error
	DeleteBySource(ctx context.Context, source *url.URL) error
	DeleteByID(ctx context.Context, ids ...model.SectionID) error
	All(ctx context.Context, yield func(model.SectionID) bool) error
	Search(ctx context.Context, query string, opts SearchOptions) ([]*SearchResult, error)
}

type OptionFunc

type OptionFunc func(opts *Options)

func WithOnProgress

func WithOnProgress(onProgress func(progress float32)) OptionFunc

type Options

type Options struct {
	OnProgress func(progress float32)
}

func NewOptions

func NewOptions(funcs ...OptionFunc) *Options

type SearchOptions

type SearchOptions struct {
	MaxResults  int
	Collections []model.CollectionID
}

type SearchResult

type SearchResult struct {
	Source   *url.URL
	Sections []model.SectionID
	// Score is the relevance score of the result (higher is better). Its scale
	// is backend-specific and only meaningful for ranking results within the
	// same Search response — not as an absolute confidence across queries or
	// backends.
	Score float64
	// SectionScores holds the per-section relevance scores when the backend
	// exposes them, keyed by section ID. It may be nil.
	SectionScores map[model.SectionID]float64
}

type Semantic

type Semantic interface {
	// Semantic reports whether the index performs vector similarity search.
	Semantic() bool
}

Semantic is an optional capability implemented by indexes performing embedding/vector similarity search, which therefore benefit from query expansion such as HyDE. Full-text (lexical) indexes must not implement it — or must return false — so the search pipeline sends them the lexical variant of the query instead (see pipeline.LexicalQueryTransformer, which carries transformations such as translation that only a lexical index needs). Hybrid indexes that manage their own lexical/vector fusion internally should also report false, to avoid polluting their lexical leg with an expanded query.

type Unwrapper

type Unwrapper interface {
	// Unwrap returns the decorated index.
	Unwrap() Index
}

Unwrapper is the convention a decorating Index must follow to stay transparent to capability detection.

Capabilities such as FilterableIndex or Semantic are discovered by type assertion, which fails against a decorator (logging, metrics, retry) that does not itself declare the method. A decorator must therefore either re-declare the capability methods it wants to expose, or implement Unwrap so the helpers below can look through it — the same pattern as errors.Is/As, familiar to Go developers.