search

Documentation

Overview

Package search implements full-text search (BM25 via Bleve) and hybrid search (RRF merging) for the Cartograph knowledge graph.

Package search — vector.go provides brute-force vector similarity search and hybrid BM25+vector search via RRF fusion.

Index

• Constants
• Variables
• func CleanQuery(raw string) string
• func CosineSimilarity(a, b []float32) float64
• func DeleteIndex(path string) error
• type Index
  • func NewIndex(path string) (*Index, error)
  • func NewMemoryIndex() (*Index, error)
  • func NewReadOnlyIndex(path string) (*Index, error)
  • func (ix *Index) Close() error
  • func (ix *Index) DocCount() (uint64, error)
  • func (ix *Index) IndexGraph(g *lpg.Graph) (int, error)
  • func (ix *Index) Search(query string, limit int) ([]SearchResult, error)
  • func (ix *Index) SearchMulti(rawQuery string, limit int) ([]SearchResult, error)
• type IndexDoc
• type RRFResult
  • func HybridSearch(bm25Results []SearchResult, vectorResults []VectorResult, limit int, ...) []RRFResult
  • func RRFMerge(resultSets ...[]SearchResult) []RRFResult
  • func WeightedRRFMerge(lists ...RankedList) []RRFResult
• type RankedList
• type RegexBuildFile
• type RegexBuildStats
  • func BuildRegexIndex(dir string, files []RegexBuildFile) (RegexBuildStats, error)
  • func BuildRegexIndexFromOpener(dir string, paths []string, openFile func(string) (io.ReadCloser, error)) (RegexBuildStats, error)
• type RegexIndex
  • func OpenRegexIndex(dir string) (*RegexIndex, error)
  • func (ix *RegexIndex) Search(opts RegexSearchOptions) (RegexSearchResult, error)
• type RegexMatch
• type RegexSearchOptions
• type RegexSearchResult
• type SearchResult
• type VectorEntry
• type VectorResult
  • func VectorSearch(query []float32, entries []VectorEntry, topK int, minScore float64) []VectorResult

Constants

const (
	RegexIndexVersion   = "codesearch-v1"
	RegexIndexFile      = "index"
	RegexStatusIndexed  = "indexed"
	RegexStatusDegraded = "degraded"

	DefaultRegexSearchLimit   = 20
	MaxRegexSearchLimit       = 200
	DefaultRegexContextLines  = 1
	MaxRegexContextLines      = 5
	MaxRegexFallbackFiles     = 5000
	MaxRegexFallbackBytes     = int64(256 * 1024 * 1024)
	MaxRegexReturnedLineBytes = 2000
)

Variables

var ErrRegexIndexMissing = errors.New("regex index missing")

Functions

func CleanQuery

func CleanQuery(raw string) string

CleanQuery normalises a raw query string for code search: lowercases, expands camelCase, splits on punctuation, strips stop words.

func CosineSimilarity

func CosineSimilarity(a, b []float32) float64

CosineSimilarity computes the cosine similarity between two vectors. Returns 0 if either vector is zero-length or they have different dimensions.

func DeleteIndex

func DeleteIndex(path string) error

DeleteIndex removes the index directory from disk.

Types

type Index

type Index struct {
	// contains filtered or unexported fields
}

Index wraps a Bleve index for searching graph nodes.

func NewIndex

func NewIndex(path string) (*Index, error)

NewIndex creates or opens a Bleve index at the given path.

func NewMemoryIndex

func NewMemoryIndex() (*Index, error)

NewMemoryIndex creates an in-memory Bleve index (useful for testing).

func NewReadOnlyIndex

func NewReadOnlyIndex(path string) (*Index, error)

NewReadOnlyIndex opens a persisted Bleve index in read-only mode. Uses a shared file lock so multiple processes can read concurrently.

func (*Index) Close

func (ix *Index) Close() error

Close closes the Bleve index.

func (*Index) DocCount

func (ix *Index) DocCount() (uint64, error)

DocCount returns the number of documents in the index.

func (*Index) IndexGraph

func (ix *Index) IndexGraph(g *lpg.Graph) (int, error)

IndexGraph indexes all searchable nodes from the graph. Returns the number of documents indexed.

func (*Index) Search

func (ix *Index) Search(query string, limit int) ([]SearchResult, error)

Search performs a BM25 search over the index. Returns results sorted by descending score, limited to the given count.

func (*Index) SearchMulti

func (ix *Index) SearchMulti(rawQuery string, limit int) ([]SearchResult, error)

SearchMulti performs multi-field, phrase-aware BM25 search with stop word removal and weighted RRF fusion across name and content fields. Name field gets 2× RRF weight. Returns results sorted by descending fused score.

type IndexDoc

type IndexDoc struct {
	ID          string `json:"id"`
	Name        string `json:"name"`
	Label       string `json:"label"`
	FilePath    string `json:"filePath"`
	Content     string `json:"content,omitempty"`
	Signature   string `json:"signature,omitempty"`
	Description string `json:"description,omitempty"`
	Annotations string `json:"annotations,omitempty"`
}

IndexDoc is the document structure stored in the Bleve index.

type RRFResult

type RRFResult struct {
	ID       string
	RRFScore float64
}

RRFResult represents a fused search result with combined score.

func HybridSearch

func HybridSearch(bm25Results []SearchResult, vectorResults []VectorResult, limit int, vectorWeight float64) []RRFResult

HybridSearch merges BM25 and vector results using Reciprocal Rank Fusion. vectorWeight controls vector influence relative to BM25 (default 1.0).

func RRFMerge

func RRFMerge(resultSets ...[]SearchResult) []RRFResult

RRFMerge combines multiple ranked result lists using Reciprocal Rank Fusion (constant k=60) with equal weights. This is a convenience wrapper around WeightedRRFMerge.

func WeightedRRFMerge

func WeightedRRFMerge(lists ...RankedList) []RRFResult

WeightedRRFMerge combines multiple weighted ranked result lists using Reciprocal Rank Fusion (k=60). Each list's rank contributions are multiplied by its weight.

type RankedList

type RankedList struct {
	Results []SearchResult
	Weight  float64
}

RankedList is a ranked result set with an associated weight for WeightedRRFMerge. A weight of 2.0 means each rank contribution from this list is doubled.

type RegexBuildFile

type RegexBuildFile struct {
	Path string
	Data []byte
}

type RegexBuildStats

type RegexBuildStats struct {
	Files int
	Bytes int64
}

func BuildRegexIndex

func BuildRegexIndex(dir string, files []RegexBuildFile) (RegexBuildStats, error)

BuildRegexIndex builds the regex search index from in-memory file contents and atomically publishes it under dir.

func BuildRegexIndexFromOpener

func BuildRegexIndexFromOpener(dir string, paths []string, openFile func(string) (io.ReadCloser, error)) (RegexBuildStats, error)

BuildRegexIndexFromOpener builds the regex search index by reading each path through openFile and atomically publishes it under dir. Each reader is closed before the next path is opened.

type RegexIndex

type RegexIndex struct {
	// contains filtered or unexported fields
}

func OpenRegexIndex

func OpenRegexIndex(dir string) (*RegexIndex, error)

func (*RegexIndex) Search

func (ix *RegexIndex) Search(opts RegexSearchOptions) (RegexSearchResult, error)

type RegexMatch

type RegexMatch struct {
	FilePath string
	Line     int
	Column   int
	LineText string
	Before   []string
	After    []string
}

type RegexSearchOptions

type RegexSearchOptions struct {
	Pattern      string
	FixedStrings bool
	IgnoreCase   bool
	Limit        int
	ContextLines int
	FilesGlob    string
	ExcludeTests bool
	AllFiles     []string
	ReadFile     func(string) ([]byte, error)
	IsTestFile   func(string) bool
}

type RegexSearchResult

type RegexSearchResult struct {
	Status    string
	Duration  time.Duration
	Matches   []RegexMatch
	FileCount int
	Truncated bool
	Degraded  bool
}

type SearchResult

type SearchResult struct {
	ID    string
	Score float64
}

SearchResult is a single result from a BM25 search.

type VectorEntry

type VectorEntry struct {
	ID     string
	Vector []float32
}

VectorEntry is a node ID + vector pair used as input to VectorSearch.

type VectorResult

type VectorResult struct {
	ID    string
	Score float64 // cosine similarity ∈ [-1, 1]
}

VectorResult is a single result from a vector similarity search.

func VectorSearch

func VectorSearch(query []float32, entries []VectorEntry, topK int, minScore float64) []VectorResult

VectorSearch performs brute-force cosine similarity search over a slice of entries. Returns the top-k results sorted by descending similarity, filtered to only include results above minScore.