vecstore

package
v0.0.0-...-a3bda8b Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Apr 6, 2026 License: MIT Imports: 8 Imported by: 0

Documentation

Index

Constants

View Source 
const (
	DefaultM              = 16  // max edges per node per layer
	DefaultEfConstruction = 200 // beam width during construction
	DefaultEfSearch       = 50  // beam width during search (tunable at query time)
)

HNSW parameters.

View Source 
const (
	// DefaultHNSWThreshold is the vector count at which we switch from brute-force to HNSW.
	DefaultHNSWThreshold = 10_000
)

Variables

This section is empty.

Functions

func CosineDistance 

func CosineDistance(a, b []float32) float32

CosineDistance returns 1 - CosineSimilarity, in [0, 2]. Suitable as a distance metric where 0 = identical.

func CosineSimilarity 

func CosineSimilarity(a, b []float32) float32

CosineSimilarity returns the cosine similarity between two vectors. Returns a value in [-1, 1] where 1 means identical direction. Both vectors must have the same length; panics otherwise.

func DotProduct 

func DotProduct(a, b []float32) float32

DotProduct returns the dot product of two vectors.

func L2Distance 

func L2Distance(a, b []float32) float32

L2Distance returns the Euclidean distance between two vectors.

func L2DistanceSquared 

func L2DistanceSquared(a, b []float32) float32

L2DistanceSquared returns the squared Euclidean distance (avoids sqrt). Useful for comparisons where only relative ordering matters.

func MarshalHNSW 

func MarshalHNSW(h *HNSW) []byte

MarshalHNSW serializes an HNSW index to bytes.

func Normalize 

func Normalize(v []float32) []float32

Normalize returns a unit-length copy of v. Returns a zero vector if v has zero magnitude.

func QuantizedCosineDistance 

func QuantizedCosineDistance(qv QuantizedVector, query []float32, queryNorm float32) float32

QuantizedCosineDistance computes approximate cosine distance using a quantized stored vector and a float32 query. queryNorm should be precomputed for batch efficiency. Uses the precomputed Norm field instead of recomputing per call.

func QuantizedDotProduct 

func QuantizedDotProduct(qv QuantizedVector, b []float32) float32

QuantizedDotProduct computes an approximate dot product between a quantized stored vector and a float32 query vector without full dequantization. 

dot(dequant(q), b) = scale * sum(q_i * b_i) + offset * sum(b_i)

func VectorNorm 

func VectorNorm(v []float32) float32

VectorNorm returns the L2 norm of a vector.

Types

type BruteForce 

type BruteForce struct {
	// contains filtered or unexported fields
}

BruteForce is a linear scan index. Efficient for small collections (<10K vectors). Supports optional int8 quantization for faster distance computation.

func NewBruteForce 

func NewBruteForce(dim int, quantize bool) *BruteForce

NewBruteForce creates a brute-force index for the given dimension. If quantize is true, vectors are also stored in int8 form for faster search.

func (*BruteForce) Add 

func (bf *BruteForce) Add(id string, vec []float32)

func (*BruteForce) Dim 

func (bf *BruteForce) Dim() int

func (*BruteForce) ForEach 

func (bf *BruteForce) ForEach(fn func(id string, vec []float32))

ForEach calls fn for each stored vector. Holds read lock for the duration.

func (*BruteForce) Get 

func (bf *BruteForce) Get(id string) []float32

Get returns the raw float32 vector for the given ID, or nil if not found.

func (*BruteForce) Has 

func (bf *BruteForce) Has(id string) bool

func (*BruteForce) Len 

func (bf *BruteForce) Len() int

func (*BruteForce) Remove 

func (bf *BruteForce) Remove(id string)

func (*BruteForce) Search 

func (bf *BruteForce) Search(query []float32, k int) []SearchHit

type Engine 

type Engine struct {
	// contains filtered or unexported fields
}

Engine manages per-KB vector indexes with adaptive strategy switching.

func NewEngine 

func NewEngine(cfg EngineConfig) *Engine

NewEngine creates a vector search engine.

func (*Engine) Add 

func (e *Engine) Add(kbID, id string, vec []float32) error

Add inserts or updates a vector in the KB's index. Returns an error on dimension mismatch instead of panicking.

func (*Engine) DropIndex 

func (e *Engine) DropIndex(kbID string)

DropIndex removes and frees a KB's index.

func (*Engine) EnsureIndex 

func (e *Engine) EnsureIndex(kbID string, dim int)

EnsureIndex creates a per-KB index if it doesn't exist. Thread-safe.

func (*Engine) HasIndex 

func (e *Engine) HasIndex(kbID string) bool

HasIndex returns true if a KB's index is loaded.

func (*Engine) Len 

func (e *Engine) Len(kbID string) int

Len returns the number of vectors in a KB's index.

func (*Engine) LoadIndex 

func (e *Engine) LoadIndex(kbID string, dim int, vectors map[string][]float32)

LoadIndex populates a KB's index from pre-existing vectors. Typically called at startup to hydrate the index from DB.

func (*Engine) Remove 

func (e *Engine) Remove(kbID, id string) error

Remove deletes a vector from the KB's index.

func (*Engine) Search 

func (e *Engine) Search(kbID string, query []float32, k int) ([]SearchHit, error)

Search finds the k nearest neighbors in the given KB. Returns an error on dimension mismatch instead of panicking.

type EngineConfig 

type EngineConfig struct {
	HNSWThreshold  int
	M              int
	EfConstruction int
	EfSearch       int
	Quantize       bool
}

EngineConfig configures the vector search engine.

type HNSW 

type HNSW struct {
	// contains filtered or unexported fields
}

HNSW implements the Hierarchical Navigable Small World graph. Reference: Malkov & Yashunin, 2016/2018.

func NewHNSW 

func NewHNSW(dim, m, efConstruction int) *HNSW

NewHNSW creates an HNSW index with the given parameters.

func UnmarshalHNSW 

func UnmarshalHNSW(data []byte) (*HNSW, error)

UnmarshalHNSW deserializes an HNSW index from bytes.

func (*HNSW) Add 

func (h *HNSW) Add(id string, vec []float32)

Add inserts a vector into the index. Thread-safe.

If a node with the same ID already exists, only the stored vector is updated. Graph edges are NOT re-linked. For correct results after a vector change, callers should Remove then Add to rebuild edges.

func (*HNSW) Dim 

func (h *HNSW) Dim() int

func (*HNSW) Get 

func (h *HNSW) Get(id string) []float32

func (*HNSW) Has 

func (h *HNSW) Has(id string) bool

func (*HNSW) Len 

func (h *HNSW) Len() int

func (*HNSW) Remove 

func (h *HNSW) Remove(id string)

Remove deletes a node from the index. Repairs neighbor connections.

func (*HNSW) Search 

func (h *HNSW) Search(query []float32, k int, efSearch int) []SearchHit

Search finds the k nearest neighbors to query. efSearch controls the beam width (higher = more accurate but slower). Pass 0 for the default.

type Index 

type Index interface {
	Add(id string, vec []float32)
	Remove(id string)
	Search(query []float32, k int) []SearchHit
	Len() int
	Dim() int
	Has(id string) bool
	Get(id string) []float32
}

Index is the common interface for vector search indexes.

type QuantizedVector 

type QuantizedVector struct {
	Data   []int8
	Scale  float32
	Offset float32
	Norm   float32 // precomputed L2 norm of dequantized vector
}

QuantizedVector holds an int8-quantized vector with scale/offset for reconstruction. Original value ≈ (int8_value * scale) + offset

func Quantize 

func Quantize(v []float32) QuantizedVector

Quantize converts a float32 vector to int8 using min-max scalar quantization. Maps [min, max] to [-127, 127]. Achieves ~4x memory reduction with <1% accuracy loss on normalized embedding vectors.

func (QuantizedVector) Dequantize 

func (qv QuantizedVector) Dequantize() []float32

Dequantize reconstructs a float32 vector from quantized form.

type SearchHit 

type SearchHit struct {
	ID       string
	Distance float32 // lower is more similar (cosine distance)
}

SearchHit is a single result from a vector search.

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.