Documentation
¶
Overview ¶
Package search implements V0 retrieval over the SQLite FTS5 index. It exposes a Retriever seam (bm25 now, hybrid later) and an FTS5-backed engine that MATCHes a sanitized query, ranks with bm25() column weights, applies small deterministic Go-side boosts, and enforces exact document filters.
Index ¶
Constants ¶
This section is empty.
Variables ¶
var ErrEmptyQuery = errors.New("search: empty query after sanitization")
ErrEmptyQuery is returned when a query reduces to no searchable terms after sanitization (e.g. it was empty or pure punctuation).
Functions ¶
func Reindex ¶
Reindex (re)computes and stores a vector for every chunk in db using emb. It embeds in batches and commits every reindexCommitEvery chunks so the single SQLite writer is not held for the whole run; the ON DELETE CASCADE on embeddings means re-ingesting a document already drops its stale vectors, so Reindex only needs to (re)write current chunks. It returns the number of vectors written.
Types ¶
type Engine ¶
type Engine struct {
// contains filtered or unexported fields
}
Engine is the FTS5-backed Retriever. It MATCHes a sanitized query against chunks_fts, ranks with bm25() column weights, then applies a small deterministic heading boost and re-sorts so higher score = better.
func NewEngine ¶
NewEngine builds an FTS5 Engine over db. A nil logger is replaced with a discard logger so the engine never panics on a missing dependency.
func (*Engine) Search ¶
Search implements Retriever. It sanitizes q.Text into a safe FTS5 MATCH expression, runs the ranked join with q's exact filters applied, converts the negative bm25 rank into a positive score, layers the heading boost, and returns results sorted best-first. An empty query (after sanitization) is a friendly ErrEmptyQuery rather than an FTS5 syntax error.
type HybridRetriever ¶
type HybridRetriever struct {
// contains filtered or unexported fields
}
HybridRetriever fuses a lexical (bm25) Retriever and a vector Retriever with Reciprocal Rank Fusion. It implements Retriever, so MCP/CLI callers are unchanged. When the vector side yields nothing (no embeddings, or a no-op embedder), fusion of a single non-empty list reproduces the lexical ranking — so hybrid degrades gracefully to bm25-only.
func NewHybridRetriever ¶
func NewHybridRetriever(lexical, vector Retriever, logger *slog.Logger) *HybridRetriever
NewHybridRetriever builds a HybridRetriever from a lexical and a vector Retriever. A nil logger is replaced with a discard logger.
func (*HybridRetriever) Search ¶
Search implements Retriever. It runs both retrievers at a widened candidate depth, fuses them with RRF, and truncates to q.Limit. A lexical query that sanitizes to empty is tolerated when the vector side still returns hits; ErrEmptyQuery surfaces only when neither side can produce results.
type Query ¶
type Query struct {
// Text is the raw user query string. It may contain arbitrary punctuation;
// the engine sanitizes it before handing it to FTS5 MATCH.
Text string
// Collection, when set, restricts results to documents.collection = it.
Collection string
// PathPrefix, when set, restricts results to documents whose uri starts
// with it (LIKE prefix match).
PathPrefix string
// FileType, when set, restricts results to documents whose uri ends with a
// matching extension (e.g. "md" or ".md").
FileType string
// ModifiedSince, when set, restricts results to documents.modified_at >= it.
// It is compared lexically, so an RFC3339 timestamp sorts correctly.
ModifiedSince string
// Limit caps the result count. The caller supplies the configured default
// when it is not overridden on the command line.
Limit int
}
Query is a single retrieval request. Text is the raw user query (sanitized into FTS5 barewords by the engine). The remaining fields are exact document filters compiled into the SQL WHERE clause; the zero value of each means "no constraint". Limit caps the number of returned results.
type Retriever ¶
type Retriever interface {
// Search runs q against the index and returns ranked results.
Search(ctx context.Context, q Query) ([]model.Result, error)
}
Retriever is the retrieval seam: a query in, ranked results out. The FTS5 engine implements it now; a hybrid (vector + bm25) retriever can implement the same interface later without touching callers.
type VectorRetriever ¶
type VectorRetriever struct {
// contains filtered or unexported fields
}
VectorRetriever implements Retriever via a linear cosine scan over the stored embeddings. It embeds the query with an Embedder, then walks every vector for the embedder's model (cosine = dot, because vectors are L2-normalized) and returns the top-K. When the embedder is the no-op (default build) it degrades to returning no results so a HybridRetriever falls back to lexical-only.
func NewVectorRetriever ¶
NewVectorRetriever builds a VectorRetriever over db using emb. A nil logger is replaced with a discard logger.
func (*VectorRetriever) Search ¶
Search implements Retriever. It returns the top-K chunks by cosine similarity to the embedded query, applying the same exact document filters as the lexical engine. A query the embedder cannot handle (no-op build) yields no results and no error, so semantic search degrades gracefully to lexical-only.
Source Files