vec

package
v0.7.1 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Jun 15, 2026 License: Apache-2.0 Imports: 15 Imported by: 0

Documentation

Overview

Package vec adds first-class vector-search support to gosqlite.org by bundling the sqlite-vec extension and a small Go API layered on top of it.

Activating the extension

Blank-importing this package auto-registers sqlite-vec on every database connection opened thereafter: 

import (
    "database/sql"
    _ "gosqlite.org"
    _ "gosqlite.org/vec"
)

db, _ := sql.Open("sqlite3", ":memory:")
// Now CREATE VIRTUAL TABLE ... USING vec0(...) works.

Raw SQL usage (option a) 

CREATE VIRTUAL TABLE docs USING vec0(embedding float[8]);
INSERT INTO docs(rowid, embedding) VALUES (1, '[0.1, 0.2, ...]');
SELECT rowid, distance
FROM   docs
WHERE  embedding MATCH '[0.5, 0.5, ...]'
ORDER BY distance LIMIT 5;

To bind embeddings as parameters instead of hard-coding them, use Encode (or the methods on Encoding) so you get the right placeholder fragment + value pair for the configured wire format: 

ph, val := vec.Encode(query, vec.Binary)
rows, _ := db.QueryContext(ctx,
    "SELECT rowid, distance FROM docs WHERE embedding MATCH "+ph+
        " ORDER BY distance LIMIT 5", val)

Typed Go API (option b)

See Create, Open, and the Table type for an iter.Seq2-based KNN cursor and typed Insert/BatchInsert/Delete helpers that handle the wire encoding for you. The Encoding option selects both the column storage type and the bind form: JSON or Binary (float[N]), Int8 (int8[N], 4× smaller — quantized via vec_quantize_int8 assuming unit [-1, 1] range), or Bit (bit[N], 32× smaller, ranked by the Hamming metric Create forces). The typed API is built strictly on top of the raw SQL layer above — anything you can do in SQL you can do in Go.

Metadata, partition, and auxiliary columns

Declare non-embedding columns via Options.Columns to filter, partition, and carry payloads on a vec0 table. A Column is Metadata (indexed, filterable), Partition (a partition-key shard, also filterable), or Auxiliary (an unindexed `+col` payload returned but not filterable). Set per-row values via [Row.Values] on Table.InsertRow / Table.BatchInsert; filter on metadata / partition columns in KNN with WithFilter; read metadata / auxiliary columns back with WithSelect + Table.KNNSQL. Options.ChunkSize sets vec0's chunk_size=.

Explicit primary keys (UUID / slug)

Table addresses rows by the implicit int64 rowid. When your keys are strings (or you want an explicit key column), use CreateKeyed / OpenKeyed to get a KeyedTable[K] (K = int64 | string): Insert / KNN take and return K-typed keys. WithKeyColumn names the key column (default "id"). This is the form the gorm sidecar needs for models with non-int64 primary keys.

Filtered KNN

WithFilter AND's a custom WHERE conjunct onto the MATCH clause. Same trust contract as [gorm.DB.Where] — the fragment is interpolated as-is; pass literals through args. Use it for per-tenant gating, rowid sub-selection, or any other column-level filter on the vec0 table.

Custom projection / JOIN

When you need to project additional columns or JOIN a companion table (the common "sidecar to canonical row table" pattern), use Table.KNNSQL with WithSelect, WithJoin, and WithOrderBy to build the query, then execute it through `db.QueryContext` (or `gorm.DB.Raw(sql, args...).Scan(&out)`): 

sql, args, _ := tbl.KNNSQL(query, 10,
    vec.WithSelect("items.id, items.title"),
    vec.WithJoin("JOIN items ON items.id = items_vec.rowid"),
    vec.WithFilter("items.tenant = ?", "acme"),
)
rows, _ := db.QueryContext(ctx, sql, args...)

Table.KNN / Table.KNNSlice reject WithSelect / WithJoin because they change the row shape the typed scanner expects.

Note: when a JOIN is in play, KNNSQL emits `k = N` as a literal alongside the MATCH conjunct rather than relying on `LIMIT N`. sqlite-vec's planner doesn't extract LIMIT through a join boundary, but the `k = N` predicate is a vec0-recognized vtab hint that survives. Callers writing raw SQL by hand against vec0 should follow the same pattern — parameterizing `k` with a `?` placeholder will silently return an unbounded scan.

Idempotent migrations

Create errors with ErrAlreadyExists (wrapped) if the table already exists. Pass WithIfNotExists to make the call a no-op on subsequent runs — useful for migrate-on-startup. The existing table's schema is NOT validated; use Open for strict matching.

Platform coverage

The underlying transpiled vec extension is built per-target by modernc.org/sqlite/vec. Some GOOS/GOARCH combinations may not be supported upstream; see that package's build tags for the authoritative list.

Practical consequence for downstream consumers: if you compile on a target the upstream `modernc.org/sqlite/vec` does not cover, `go build ./...` against your module will fail at this sub-package while the rest of gosqlite.org still compiles. The remaining sub-packages (root driver, fts, gorm, vfs, fts/gorm) work on every supported target. Build with `go build ./... 2>/dev/null || go build ./` if you want to skip vec/ on niche arches, or list the packages you actually consume explicitly.

Observability

Insert / BatchInsert / Update / Delete / KNN can be wrapped with slog logging or a metrics recorder by composing the optional decorators in observability.go: Wrap, WithLogger, WithRecorder. Parallel to the matching surface in gosqlite.org/fts.

See also

  • gosqlite.org/vec/gorm — tag-driven sqlite-vec sidecars wired into gorm models. Define an Embedding field on a gorm model and the plugin owns CRUD sync, soft-delete filtering, and DropTable cascade.
  • examples/features/search/vec-search — runnable demo of the raw vec.Table API.
  • examples/features/gorm/vec-tagged — the same data flow expressed via the gorm bridge.
  • dev/coverage/vec.md — every documented sqlite-vec feature with its current status (typed / raw / inherited).
  • gosqlite.org/vfs/crypto — pure-Go encryption at rest. Composes with vec0 — encrypted vector databases work end-to-end through the same wrapping VFS. Same Recorder-shaped observability surface, opted in via Options.Recorder.

Index

Constants

This section is empty.

Variables

View Source 
var ErrAlreadyExists = errors.New("vec: virtual table already exists")

ErrAlreadyExists wraps the error returned by Create when the named virtual table already exists and WithIfNotExists was not passed. Match via errors.Is to branch between create-or-open without duplicating the existence check.

Note that ErrAlreadyExists does NOT signal a schema mismatch — if the existing table was created with a different dim, metric, or encoding, you'll still get this error. Use Open to verify the schema you expect.

Functions

func CosineDistance 

func CosineDistance(ctx context.Context, db *sql.DB, a, b []float32) (float64, error)

CosineDistance returns the cosine distance via vec_distance_cosine.

func DotDistance 

func DotDistance(ctx context.Context, db *sql.DB, a, b []float32) (float64, error)

DotDistance returns sqlite-vec's negative-dot-product distance. sqlite-vec exposes this as the L1 distance fn but calls the metric "dot" in the vec0 options; we expose the SQL fn here under DotDistance for symmetry with the Metric constants.

func Encode 

func Encode(embedding []float32, enc Encoding) (placeholder string, value any)

Encode is the package-function form of Encoding.Placeholder + Encoding.Encode. Use it from raw-SQL escape hatches when you're composing a query by hand and want the typed encoding pipeline without going through Table.KNN / Table.KNNSlice.

Returns the SQL placeholder fragment (`?` for JSON, `vec_f32(?)` for Binary) and the bind value that pairs with it (a string for JSON, a []byte for Binary). The placeholder is meant for string interpolation into your SQL; the value is passed as a parameter to `db.Query` / `db.Exec`.

Example: 

ph, val := vec.Encode(embedding, vec.Binary)
rows, err := db.QueryContext(ctx,
    "SELECT rowid, distance FROM items_vec WHERE embedding MATCH "+ph+
        " AND rowid IN (?, ?, ?) ORDER BY distance LIMIT 10",
    val, id1, id2, id3)

For the typed path (no manual SQL), use Table.KNN / Table.KNNSlice instead.

func L2Distance 

func L2Distance(ctx context.Context, db *sql.DB, a, b []float32) (float64, error)

L2Distance returns the L2 (Euclidean) distance between two vectors using sqlite-vec's vec_distance_l2 SQL function. The vectors must be the same length; we round-trip through SQL so callers get exactly the same value the vec0 virtual table would compute internally — useful for spot-checking fixtures or scoring known pairs without inserting them into a table.

The supplied db can be any *sql.DB that has loaded the sqlite-vec extension (any *sql.DB obtained from this package's parent driver after `import _ "gosqlite.org/vec"` qualifies).

func QuoteIdent 

func QuoteIdent(name string) string

QuoteIdent returns name in backticks, escaping any embedded backticks. Used for SQL identifier interpolation outside the vec0 constructor — SQLite treats table/column names as identifiers, not bind parameters. Thin re-export of internal/sqlid.QuoteIdentBacktick so the internals share one implementation.

Note: vec0's CREATE VIRTUAL TABLE column-argument parser does NOT accept quoted identifiers — only bare names — so identifiers fed into that constructor must pass ValidIdent and be interpolated raw.

func ValidIdent 

func ValidIdent(s string) bool

ValidIdent reports whether name is a safe SQL identifier — the conservative ASCII subset: leading letter or underscore, then letters, digits, or underscores. Used to guard against SQL injection at the API boundary when callers pass arbitrary strings as table or column names. Thin re-export of internal/sqlid.ValidIdent.

Types

type Column 

type Column struct {
	// Name is the column identifier (must satisfy ValidIdent).
	Name string
	// Type is the SQL storage type: "text", "integer", "float", or "blob".
	Type string
	// Kind selects metadata (filterable), partition-key, or auxiliary (payload).
	Kind ColumnKind
}

Column declares a non-embedding column on a vec0 table — sqlite-vec's mechanism for filtered, partitioned, and payload-carrying vector search.

type ColumnKind 

type ColumnKind int

ColumnKind classifies a non-embedding vec0 column. 

const (
	// Metadata is an indexed, filterable column: a KNN query may constrain it
	// with a WHERE predicate (via [WithFilter]) evaluated alongside MATCH.
	Metadata ColumnKind = iota
	// Partition declares a partition-key column: sqlite-vec shards the index by
	// its value so a query filtered to one partition skips the others. Also
	// filterable like Metadata.
	Partition
	// Auxiliary is an unindexed stored payload (vec0's `+col`): returned with a
	// row but not usable in a KNN WHERE predicate. Cheaper than Metadata when
	// you only need to read the value back, not filter on it.
	Auxiliary
)

type CreateOption 

type CreateOption func(*createConfig)

CreateOption configures a single Create call. Compose via the variadic Create(ctx, db, name, dim, opts, createOpts...) tail.

func WithIfNotExists 

func WithIfNotExists() CreateOption

WithIfNotExists makes Create idempotent: if the table already exists, Create returns a Table handle for it instead of erroring with ErrAlreadyExists. The existing table's schema is NOT validated against the dim / metric / encoding you pass — if those differ from what the table was created with, Insert / KNN may fail at runtime. Use Open instead when you want strict schema-match semantics on an existing table.

Typical use is migrate-on-startup where you want the create to be a no-op on subsequent runs: 

tbl, err := vec.Create(ctx, db, "docs", 384, vec.Options{Metric: vec.Cosine},
    vec.WithIfNotExists())

func WithKeyColumn 

func WithKeyColumn(name string) CreateOption

WithKeyColumn overrides the primary-key column name of a KeyedTable (default "id"). Ignored by the rowid-based Create.

type Encoding 

type Encoding int

Encoding chooses how this package serializes []float32 vectors when sending them to SQLite. Both encodings are accepted by sqlite-vec; binary is more compact and avoids the JSON parse on every insert, while JSON is human- readable and the form used in sqlite-vec's documentation examples. 

const (
	// JSON encodes vectors as the text `[v0, v1, ...]` per sqlite-vec's
	// canonical example syntax. Default for backwards compatibility. Column
	// type float[N].
	JSON Encoding = iota
	// Binary encodes vectors as a packed little-endian float32 BLOB and lets
	// sqlite-vec parse it via the vec_f32(?) constructor. Recommended for
	// performance when bulk-inserting. Column type float[N].
	Binary
	// Int8 stores vectors as a quantized int8[N] column — 4x smaller than
	// float32. The caller still works in []float32; sqlite-vec quantizes at
	// insert/query time via vec_quantize_int8(?, 'unit'), which assumes each
	// component is in [-1, 1] (L2-normalize first, or scale into range).
	Int8
	// Bit stores vectors as a packed bit[N] column — 32x smaller than float32,
	// one bit per dimension (sign). The caller works in []float32; sqlite-vec
	// quantizes via vec_quantize_binary(?). Bit columns rank by [Hamming]
	// distance only; Create forces it.
	Bit
)

func ParseEncoding 

func ParseEncoding(s string) (Encoding, error)

ParseEncoding maps a case-insensitive keyword to an Encoding. Recognised values: "json" (default), "binary", "int8", "bit".

func (Encoding) Encode 

func (e Encoding) Encode(v []float32) any

Encode serializes a []float32 into a value suitable for sql.DB.Exec bindings. Binary yields a packed float32 []byte; every other encoding yields the JSON-array string. For Int8 / Bit the float vector is sent as JSON text and sqlite-vec quantizes it at the SQL layer (see Placeholder), so the Go side stays float32 throughout.

func (Encoding) Placeholder 

func (e Encoding) Placeholder() string

Placeholder returns the SQL fragment that binds a vector argument on the right-hand side of MATCH (and in INSERT / UPDATE):

  • JSON: "?" — the text JSON array is parsed directly.
  • Binary: "vec_f32(?)" — the packed float32 BLOB is reconstructed.
  • Int8: "vec_quantize_int8(?, 'unit')" — the JSON float vector is quantized to int8 assuming unit ([-1, 1]) range.
  • Bit: "vec_quantize_binary(?)" — the JSON float vector is quantized to a 1-bit-per-dimension sign vector.

type Key 

type Key interface {
	~int64 | ~string
}

Key is the primary-key type a KeyedTable supports: a 64-bit integer or a string (e.g. a UUID or slug). Named types with those underlying kinds are accepted too.

type KeyedNeighbor 

type KeyedNeighbor[K Key] struct {
	Key      K
	Distance float64
}

KeyedNeighbor is one KNN result: a key and its distance to the query.

type KeyedRow 

type KeyedRow[K Key] struct {
	Key       K
	Embedding []float32
}

KeyedRow is a single (key, embedding) pair for KeyedTable.BatchInsert.

type KeyedTable 

type KeyedTable[K Key] struct {
	// contains filtered or unexported fields
}

KeyedTable is a sqlite-vec table whose rows are addressed by an explicit primary-key column of type K, instead of the implicit int64 rowid that Table uses. Use it when your keys are strings (UUIDs, slugs) — the form the gorm sidecar needs for models with non-int64 primary keys.

It mirrors Table's lifecycle and KNN surface (and the same [QueryOption]s) but its Insert / KNN take and return K-typed keys. Construct it with CreateKeyed or OpenKeyed. Safe for concurrent use as long as the *sql.DB is. Metadata / partition / auxiliary columns are available on the rowid Table; KeyedTable carries only the key and embedding.

func CreateKeyed 

func CreateKeyed[K Key](ctx context.Context, db *sql.DB, name string, dim int, opts Options, createOpts ...CreateOption) (*KeyedTable[K], error)

CreateKeyed runs CREATE VIRTUAL TABLE name USING vec0(id <type> primary key, embedding <enc>[dim] distance=<metric>) and returns a handle. The key column is named "id"; override it with WithKeyColumn. By default the call wraps ErrAlreadyExists if name already exists; pass WithIfNotExists.

func OpenKeyed 

func OpenKeyed[K Key](db *sql.DB, name string, dim int, opts Options, createOpts ...CreateOption) (*KeyedTable[K], error)

OpenKeyed returns a handle to a keyed vec0 table that already exists. It does not validate the schema; the dim / encoding / metric / key column must match what the table was created with (override the key column via WithKeyColumn).

func (*KeyedTable[K]) BatchInsert 

func (t *KeyedTable[K]) BatchInsert(ctx context.Context, items []KeyedRow[K]) error

BatchInsert inserts every item in a single transaction.

func (*KeyedTable[K]) Close 

func (t *KeyedTable[K]) Close() error

Close is a no-op kept for API symmetry with Table.Close.

func (*KeyedTable[K]) Delete 

func (t *KeyedTable[K]) Delete(ctx context.Context, key K) error

Delete removes the row with the given key. A missing key is a no-op.

func (*KeyedTable[K]) Dim 

func (t *KeyedTable[K]) Dim() int

func (*KeyedTable[K]) Drop 

func (t *KeyedTable[K]) Drop(ctx context.Context) error

Drop removes the vtab and its shadow storage.

func (*KeyedTable[K]) Encoding 

func (t *KeyedTable[K]) Encoding() Encoding

func (*KeyedTable[K]) Insert 

func (t *KeyedTable[K]) Insert(ctx context.Context, key K, embedding []float32) error

Insert adds a single row keyed by key. A duplicate key is an error (vec0's INSERT does not honor OR REPLACE) — use Update to overwrite.

func (*KeyedTable[K]) KNN 

func (t *KeyedTable[K]) KNN(ctx context.Context, query []float32, k int, opts ...QueryOption) iter.Seq2[KeyedNeighbor[K], error]

KNN issues a k-nearest-neighbour query, yielding results in ascending-distance order. Accepts the same [QueryOption]s as Table.KNN (WithFilter, …); WithSelect / WithJoin require KeyedTable.KNNSQL.

func (*KeyedTable[K]) KNNSQL 

func (t *KeyedTable[K]) KNNSQL(query []float32, k int, opts ...QueryOption) (string, []any, error)

KNNSQL returns the SQL + bind args KNN would execute, for the WithSelect / WithJoin escape hatch (run it through db.QueryContext and scan your own shape).

func (*KeyedTable[K]) KNNSlice 

func (t *KeyedTable[K]) KNNSlice(ctx context.Context, query []float32, k int, opts ...QueryOption) ([]KeyedNeighbor[K], error)

KNNSlice collects the first k matches into a slice.

func (*KeyedTable[K]) KeyColumn 

func (t *KeyedTable[K]) KeyColumn() string

func (*KeyedTable[K]) Metric 

func (t *KeyedTable[K]) Metric() Metric

func (*KeyedTable[K]) Name 

func (t *KeyedTable[K]) Name() string

Name / Dim / Metric / Encoding / KeyColumn report the table's configuration.

func (*KeyedTable[K]) Update 

func (t *KeyedTable[K]) Update(ctx context.Context, key K, embedding []float32) error

Update replaces the embedding for an existing key. A missing key is a no-op.

type Metric 

type Metric int

Metric identifies the distance function sqlite-vec uses when comparing vectors. The three sqlite-vec-supported metrics are L1, L2, and Cosine; see https://alexgarcia.xyz/sqlite-vec/api-reference.html for the authoritative list. 

const (
	// L2 is the default; squared Euclidean distance. Matches sqlite-vec's
	// vec0() default ranking on float[N] columns.
	L2 Metric = iota
	// Cosine selects cosine distance. Range [0, 2]; smaller is more similar.
	Cosine
	// Dot maps to sqlite-vec's L1 (Manhattan / taxicab) distance — the name
	// is kept for plan/historical compatibility but the metric is L1.
	// Smaller is more similar. Callers wanting a true inner-product score
	// can compute it client-side from the raw vectors.
	Dot
	// Hamming is the bit-vector distance: the number of differing bits. It is
	// the only metric sqlite-vec supports for bit[N] columns (the [Bit]
	// encoding), where it is the implicit default.
	Hamming
)

func ParseMetric 

func ParseMetric(s string) (Metric, error)

ParseMetric maps a case-insensitive keyword to a Metric. Recognised values: "l2" (default), "cosine", "dot", "l1", "hamming". Useful when reading metric names from tags, configs, or user input.

func (Metric) Keyword 

func (m Metric) Keyword() string

Keyword returns the sqlite-vec vec0() constructor keyword for this metric: "l2", "cosine", "l1" (for the Dot alias), or "hamming". Use this when building CREATE VIRTUAL TABLE statements outside vec.Create.

func (Metric) String 

func (m Metric) String() string

String renders a metric for logging and inspection. The strings are approximate human labels, not the keywords sqlite-vec accepts in the vec0 constructor — see Keyword for that.

type Neighbor 

type Neighbor struct {
	Rowid    int64
	Distance float64
}

Neighbor is one row returned by KNN: the matched rowid and the distance scored by the table's Metric.

type Observable 

type Observable struct {
	// contains filtered or unexported fields
}

Observable wraps a Table with optional logging and metrics hooks. It exposes the same surface so callers can drop it in without changing call sites: Insert / BatchInsert / Delete / KNN / KNNSlice all delegate to the underlying Table after firing the configured hooks.

Wrap without Options returns a pass-through wrapper — useful for tests that toggle observability via a build-time flag.

func Wrap 

func Wrap(tbl *Table, opts ...Option) *Observable

Wrap returns an Observable view of tbl that fires the supplied hooks on every operation. Passing no Option makes Wrap a no-op pass-through.

func (*Observable) BatchInsert 

func (o *Observable) BatchInsert(ctx context.Context, items []Row) error

BatchInsert records the operation and delegates.

func (*Observable) Delete 

func (o *Observable) Delete(ctx context.Context, rowid int64) error

Delete records the operation and delegates.

func (*Observable) Dim 

func (o *Observable) Dim() int

Dim returns the underlying table's vector dimension.

func (*Observable) Inner 

func (o *Observable) Inner() *Table

Inner returns the underlying Table. Useful for accessing methods not surfaced through Observable (e.g. Drop, accessors) without losing the wrapped instance.

func (*Observable) Insert 

func (o *Observable) Insert(ctx context.Context, rowid int64, embedding []float32) error

Insert records the operation and delegates.

func (*Observable) KNN 

func (o *Observable) KNN(ctx context.Context, query []float32, k int, opts ...QueryOption) iter.Seq2[Neighbor, error]

KNN wraps the underlying iter.Seq2 so the recorder/logger fire exactly once per KNN invocation, after the iterator is fully drained or cleaned up by an early break. Forwards QueryOptions to the underlying Table.KNN.

func (*Observable) KNNSlice 

func (o *Observable) KNNSlice(ctx context.Context, query []float32, k int, opts ...QueryOption) ([]Neighbor, error)

KNNSlice mirrors Table.KNNSlice with observability. Forwards QueryOptions.

The output slice is pre-sized using the same `min(max(k,0), 1024)` clamp as Table.KNNSlice so the decorator doesn't silently re-pay the slice-growth overhead the bare Table path avoids.

func (*Observable) Name 

func (o *Observable) Name() string

Name returns the underlying table name.

type Option 

type Option func(*obsConfig)

Option configures Wrap.

func WithLogger 

func WithLogger(l *slog.Logger) Option

WithLogger attaches an slog.Logger that receives Info-level events for every successful operation and Error-level events for failures. The logger is invoked synchronously.

func WithRecorder 

func WithRecorder(r Recorder) Option

WithRecorder attaches a Recorder for metrics/tracing.

type Options 

type Options struct {
	// Metric selects the distance function used during MATCH queries.
	// Defaults to L2 when zero.
	Metric Metric
	// Encoding selects how Insert/BatchInsert/KNN serialize []float32
	// vectors. Defaults to JSON when zero.
	Encoding Encoding
	// Columns declares non-embedding metadata / partition-key / auxiliary
	// columns. Their values are carried per row by [Row.Values]; metadata and
	// partition columns are filterable in KNN via [WithFilter]; auxiliary and
	// metadata columns are retrievable via [WithSelect] + [Table.KNNSQL].
	Columns []Column
	// ChunkSize sets vec0's chunk_size= (vectors per index chunk). Zero leaves
	// the sqlite-vec default. Must be a multiple of 8 when set.
	ChunkSize int
}

Options configures Create.

type QueryOption 

type QueryOption func(*queryConfig)

QueryOption configures a single KNN call. Compose with KNN, KNNSlice, and KNNSQL.

func WithFilter 

func WithFilter(sql string, args ...any) QueryOption

WithFilter appends a custom WHERE conjunct to the KNN query. The SQL fragment is AND'd with the MATCH clause; bind parameters in args bind in declaration order. This is the supported way to do filtered KNN with sqlite-vec — restricting to "this user's documents", "items priced under $50", etc.

Trust model

The fragment is **caller-trusted raw SQL**, interpolated into the query as-is. Values passed via args... are bound as parameters and are safe; the fragment text is not validated and not escaped. Same trust contract as gorm.Where(fragment, args...). Callers MUST:

  • validate any identifier they interpolate into the fragment via ValidIdent before passing the fragment in,
  • route every literal through args... rather than building it into the fragment string,
  • never pass user-controlled SQL through here.

The fragment must reference columns the vec0 table actually has — rowid is always available; user-declared metadata columns appear under their bare names.

Example: 

tbl.KNN(ctx, query, 5, vec.WithFilter("rowid > ?", lastSeenID))

The args slice is variadic; pass values inline.

func WithJoin 

func WithJoin(joinClause string) QueryOption

WithJoin inserts a JOIN clause after "FROM <table>". The fragment must include the JOIN keyword and the ON predicate. Example: 

vec.WithJoin("JOIN items ON items.id = items_vec.rowid")

Trust model

joinClause is **caller-trusted raw SQL** — interpolated as-is. Same rules as WithFilter / WithSelect / WithOrderBy.

IMPORTANT: WithJoin (like WithSelect) is only honored by Table.KNNSQL; passing it to Table.KNN or Table.KNNSlice returns an error.

func WithOrderBy 

func WithOrderBy(expr string) QueryOption

WithOrderBy replaces the default "ORDER BY distance" with the given expression. Use it when you want to order results by something other than vec0 distance — e.g. a sidecar table's created_at after JOINing.

Trust model

expr is **caller-trusted raw SQL** — interpolated as-is. Validate identifiers via ValidIdent before interpolating. Same contract as WithFilter / WithSelect / WithJoin.

IMPORTANT: WithOrderBy is honored by all three of Table.KNN, Table.KNNSlice, and Table.KNNSQL — it does not change the row shape, only the order.

func WithSelect 

func WithSelect(extraCols string) QueryOption

WithSelect appends extra projected columns to the SELECT list of a KNN query. The default projection is "rowid, distance"; passing "items.title, items.tenant" yields "SELECT rowid, distance, items.title, items.tenant ...". Pair with WithJoin to source the extra columns from another table.

Trust model

extraCols is **caller-trusted raw SQL** — interpolated as-is. Validate identifiers via ValidIdent before interpolating; never pass user-controlled SQL through here. Same contract as WithFilter / WithJoin / WithOrderBy.

IMPORTANT: WithSelect changes the row shape returned by the query, which means Table.KNN and Table.KNNSlice cannot scan the result — they expect exactly (rowid int64, distance float64). Use Table.KNNSQL together with database/sql.DB.QueryContext or gorm's `db.Raw(sql, args...).Scan(&out)` to scan the projected shape. Calling KNN / KNNSlice with WithSelect set returns an error.

type Recorder 

type Recorder interface {
	OnInsert(ctx context.Context, table string, n int, dim int, d time.Duration, err error)
	OnDelete(ctx context.Context, table string, n int, d time.Duration, err error)
	OnKNN(ctx context.Context, table string, dim, k int, d time.Duration, err error)
}

Recorder is the metrics/tracing hook surface for vec.Observable. Each method is called after the corresponding Table operation has returned; the err argument is the operation's result. Implementations should be cheap — heavy work inside a Recorder method will block the calling goroutine.

The shape mirrors fts.Recorder but with vec-native fields (vector dimension on inserts, k on KNN queries) so the per-package detail is preserved without a generic super-interface.

type Row 

type Row struct {
	Rowid     int64
	Embedding []float32
	Values    map[string]any
}

Row is a single (rowid, embedding) pair consumed by BatchInsert and InsertRow. Values supplies the declared metadata / partition / auxiliary column values keyed by column name; a column absent from the map binds NULL.

Note: sqlite-vec requires a value for metadata and partition-key columns — a NULL (i.e. an omitted entry) is rejected at insert. Only auxiliary columns may be omitted. The simple Table.Insert therefore works only on tables with no extra columns (or auxiliary-only); use InsertRow / BatchInsert with Values for tables that declare metadata or partition columns.

type Table 

type Table struct {
	// contains filtered or unexported fields
}

Table is a typed handle to a sqlite-vec virtual table backed by an embedding column of fixed dimension. Construct one with Create (which runs the CREATE VIRTUAL TABLE statement) or Open (which assumes the table already exists).

Table is safe for concurrent use as long as the underlying *sql.DB is — it holds no per-conn state.

func Create 

func Create(ctx context.Context, db *sql.DB, name string, dim int, opts Options, createOpts ...CreateOption) (*Table, error)

Create runs `CREATE VIRTUAL TABLE name USING vec0(embedding float[dim])` with the supplied options and returns a Table handle. By default the call errors with ErrAlreadyExists (wrapped) if name is already a virtual table; pass WithIfNotExists to make the call idempotent.

dim is required and must be positive. opts may be the zero value, in which case Metric defaults to L2 and Encoding defaults to JSON.

func Open 

func Open(db *sql.DB, name string, dim int, opts Options) (*Table, error)

Open returns a Table handle for a vec0 virtual table that already exists in db. It does not validate the table's schema — the dim/encoding/metric arguments must match what was declared when the table was created. Use Create instead when you want the package to issue CREATE for you.

func (*Table) BatchInsert 

func (t *Table) BatchInsert(ctx context.Context, items []Row) error

BatchInsert inserts every item in a single transaction. Each rowid must be unique within the table; conflicts surface as errors (sqlite-vec's vec0 INSERT does not honor OR REPLACE).

func (*Table) Close 

func (t *Table) Close() error

Close is a no-op kept for API symmetry with [fts.Index.Close] and io.Closer-shaped consumer code. Table holds no per-instance resource (the *sql.DB is caller-owned), so `defer tbl.Close()` is safe to add without any teardown cost. Always returns nil.

func (*Table) Delete 

func (t *Table) Delete(ctx context.Context, rowid int64) error

Delete removes the row with the given rowid. Returns nil if the row didn't exist.

func (*Table) Dim 

func (t *Table) Dim() int

Dim returns the vector dimension this table was created with.

func (*Table) Drop 

func (t *Table) Drop(ctx context.Context) error

Drop removes the underlying virtual table. After calling Drop, the Table handle is no longer usable.

func (*Table) Encoding 

func (t *Table) Encoding() Encoding

Encoding returns the wire encoding used for vector values.

func (*Table) Insert 

func (t *Table) Insert(ctx context.Context, rowid int64, embedding []float32) error

Insert adds a single row keyed by rowid. Returns an error if the rowid already exists — sqlite-vec's vec0 INSERT does not honor SQLite's OR REPLACE conflict resolution, so we report the conflict honestly. Use Update to overwrite an existing row.

The embedding length must match the table's dim.

func (*Table) InsertRow 

func (t *Table) InsertRow(ctx context.Context, row Row) error

InsertRow inserts a single row including its declared column values (Row.Values). Use it instead of Insert when the table has metadata / partition / auxiliary columns. Like Insert, a duplicate rowid is an error (vec0 INSERT ignores OR REPLACE).

func (*Table) KNN 

func (t *Table) KNN(ctx context.Context, query []float32, k int, opts ...QueryOption) iter.Seq2[Neighbor, error]

KNN issues an approximate k-nearest-neighbour query for the given vector and returns an iter.Seq2 cursor over the results in ascending-distance order. Yielding stops at k rows or on error.

Optional QueryOptions filter the result set. WithFilter appends a custom WHERE conjunct (e.g. "rowid IN (1, 2, 3)" to restrict to known IDs); see WithFilter for details.

The yielded error is always nil except for the final iteration after a row-scan failure, where it carries the scan error and the Neighbor is the zero value. Iterating with `for m, err := range tbl.KNN(...)` follows the idiomatic Go-1.23 range-over-func convention.

func (*Table) KNNSQL 

func (t *Table) KNNSQL(query []float32, k int, opts ...QueryOption) (string, []any, error)

KNNSQL returns the SQL statement and bound arguments that KNN would execute, without actually running it. Pair with database/sql.DB.QueryContext or gorm's `db.Raw(sql, args...).Scan(&out)` when you want to extend the projection (via WithSelect) or join companion data (via WithJoin) and scan rows into a custom struct.

The returned SQL is parameterized: the query embedding is bound as the first argument (after any [WithRanking]-style positional args upstream of MATCH — there are none today, but the contract is "args in the order they appear in the SQL"). The vector encoding matches the Table's configured Encoding.

Example with WithJoin + WithSelect: 

sql, args, err := tbl.KNNSQL(query, 10,
    vec.WithSelect("items.id, items.title"),
    vec.WithJoin("JOIN items ON items.id = items_vec.rowid"),
    vec.WithFilter("items.tenant = ?", "acme"),
)
if err != nil { return err }
rows, _ := db.QueryContext(ctx, sql, args...)

k=0 returns an empty SQL string and no args; callers should short-circuit. Negative k is treated the same.

func (*Table) KNNSlice 

func (t *Table) KNNSlice(ctx context.Context, query []float32, k int, opts ...QueryOption) ([]Neighbor, error)

KNNSlice is a convenience wrapper that collects the first k matches into a slice. Use it when you don't need streaming behavior. Accepts the same QueryOptions as KNN; WithSelect / WithJoin are not honored (use Table.KNNSQL instead).

func (*Table) Metric 

func (t *Table) Metric() Metric

Metric returns the distance metric this table uses.

func (*Table) Name 

func (t *Table) Name() string

Name returns the table name as known to SQLite.

func (*Table) Update 

func (t *Table) Update(ctx context.Context, rowid int64, embedding []float32) error

Update replaces the embedding for an existing rowid via a real UPDATE statement. If the rowid doesn't exist, Update is a no-op and returns nil (matching SQL's UPDATE-without-matching-row semantics).

Use Insert to add a new rowid. Callers wanting upsert behavior should dispatch on a prior existence check or fall back from Update to Insert based on rows-affected.

Source Files

View all Source files

Directories

Path Synopsis
Package vecgorm provides a tag-driven bridge between gorm models and sqlite-vec virtual-table sidecars.
 Package vecgorm provides a tag-driven bridge between gorm models and sqlite-vec virtual-table sidecars.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.