catalog

package
v0.0.0-...-effd846 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Jun 29, 2026 License: Apache-2.0 Imports: 8 Imported by: 0

Documentation

Overview

Package catalog implements the vec data model and schema authority (spec 02): the value type system, point identity, distance-metric binding, collection schemas, the three system collections, schema definition and evolution rules, constraints, and the write-path validation every insert and upsert must pass.

The catalog sits above the storage engine ([04], the storage package) and below the query layer ([10]-[13]). It owns the logical schema (column names, types, dimensions, metrics, nullability, defaults) that the spec calls the authoritative source of truth (spec 02 §2.3), and it lowers each schema into a storage.CollectionDef so the engine can lay out segments (spec 02 §11). The model Value here is the application-facing value (spec 02 §16.1); the engine's storage.Value is the physical cell, and Value.lower bridges the two.

Index

Constants

This section is empty.

Variables

View Source 
var (
	// ErrDuplicateKey is a point id that already exists, live or deleted
	// (spec 02 §13.2, DUPLICATE_KEY). The deleted-id record also triggers it.
	ErrDuplicateKey = errors.New("vec: duplicate point id")
	// ErrDimMismatch is a vector whose length differs from the column dimension
	// (spec 02 §13.3, DIM_MISMATCH).
	ErrDimMismatch = errors.New("vec: vector dimension does not match column")
	// ErrNaNInVector is a vector containing a NaN element (spec 02 §13.4).
	ErrNaNInVector = errors.New("vec: vector contains NaN")
	// ErrInfInVector is a vector containing a +Inf or -Inf element (spec 02 §13.5).
	ErrInfInVector = errors.New("vec: vector contains Inf")
	// ErrNullViolation is a NULL for a NOT NULL column (spec 02 §13.6).
	ErrNullViolation = errors.New("vec: null value in NOT NULL column")
	// ErrTypeMismatch is a value whose kind is not convertible to the column type
	// (spec 02 §13.7, schema-fixed mode only).
	ErrTypeMismatch = errors.New("vec: value type does not match column")
	// ErrValueOutOfRange is an int8 or binary element outside its representable
	// range (spec 02 §4.5).
	ErrValueOutOfRange = errors.New("vec: vector element out of range")
	// ErrCheckViolation is a row failing a declared CHECK constraint (spec 02 §13.8).
	ErrCheckViolation = errors.New("vec: check constraint violated")
	// ErrUniqueViolation is a value conflicting with a UNIQUE constraint (spec 02 §13.9).
	ErrUniqueViolation = errors.New("vec: unique constraint violated")
	// ErrMetricUnsupported is an opclass incompatible with the column element type
	// (spec 02 §13.10, METRIC_UNSUPPORTED).
	ErrMetricUnsupported = errors.New("vec: metric not supported for element type")
	// ErrValueTooLarge is a TEXT, BLOB, or JSON value over the size limit (spec 02 §13.11).
	ErrValueTooLarge = errors.New("vec: metadata value too large")
	// ErrCollectionNotFound is a reference to an unknown collection (spec 02 §13.16).
	ErrCollectionNotFound = errors.New("vec: collection not found")
	// ErrCollectionExists is a CREATE TABLE for a name already in the catalog
	// without IF NOT EXISTS (spec 02 §2.2).
	ErrCollectionExists = errors.New("vec: collection already exists")
	// ErrIDTypeMismatch is a point id whose form does not match the collection's
	// declared id kind (spec 02 §13.17, ID_TYPE_MISMATCH).
	ErrIDTypeMismatch = errors.New("vec: point id type does not match collection")
	// ErrReservedName is a user collection named with the reserved vec_ prefix
	// (spec 02 §2.2).
	ErrReservedName = errors.New("vec: collection name uses the reserved vec_ prefix")
	// ErrInvalidSchema is a schema that violates a structural rule of spec 02 §9.1
	// (no vector column, duplicate column names, bad dimension, no primary key).
	ErrInvalidSchema = errors.New("vec: invalid schema")
	// ErrSequenceOverflow is the auto-increment counter passing 2^64-1 (spec 02 §3.5).
	ErrSequenceOverflow = errors.New("vec: auto-increment sequence overflow")
	// ErrIDRequired is a missing point id on a collection without AUTOINCREMENT
	// (spec 02 §16.4).
	ErrIDRequired = errors.New("vec: point id required")
)

Data-model error set (spec 02 §13). These are the named conditions the write path and the catalog raise; callers match with errors.Is. Each maps to the error name the library API ([14] §9) and the VectorSQL response surface use.

View Source 
var Null = Value{/* contains filtered or unexported fields */}

Null is the absent value (spec 02 §7.11).

Functions

func MetricSupported 

func MetricSupported(m Metric, e ElementType) bool

MetricSupported reports whether a metric is valid for an element type (spec 02 §8.2 normative table, §13.10). Binary vectors take only Hamming and Jaccard; float and int8 vectors take L2, cosine, and inner product.

Types

type Catalog 

type Catalog struct {
	// contains filtered or unexported fields
}

Catalog is the registry of collections in one database (spec 02 §2.6). It owns the schema authority and assigns the 32-bit-range collection ids the engine keys on (spec 02 §2.6). It is safe for concurrent use.

func New 

func New() *Catalog

New creates an empty catalog (spec 02 §2.5: the three system collections are virtual and computed on demand, not stored as rows).

func (*Catalog) CreateCollection 

func (cat *Catalog) CreateCollection(s *Schema, ifNotExists bool) (*Collection, bool, error)

CreateCollection registers a new collection from a schema (spec 02 §9.1). The schema is normalized and validated in place. ifNotExists makes a name clash a no-op returning the existing collection (spec 02 §2.2). It returns the live collection handle and whether a new collection was created.

func (*Catalog) DropCollection 

func (cat *Catalog) DropCollection(name string, ifExists bool) (*Collection, error)

DropCollection removes a collection from the catalog (spec 02 §13.15). The deleted-id records and all schema state are discarded; ifExists makes an absent collection a no-op (spec 02 §13.16). It returns the dropped collection so the db layer can drop the engine collection and free pages.

func (*Catalog) Get 

func (cat *Catalog) Get(name string) (*Collection, error)

Get returns the live collection by name (spec 02 §13.16).

func (*Catalog) List 

func (cat *Catalog) List() []string

List returns every collection name sorted (deterministic catalog order).

func (*Catalog) SetClock 

func (cat *Catalog) SetClock(now func() time.Time)

SetClock overrides the creation-time source, for tests that need a fixed clock.

func (*Catalog) VecCollections 

func (cat *Catalog) VecCollections() []SystemRow

VecCollections returns the rows of the vec_collections system collection (spec 02 §2.5): one row per user collection with its mode, live point count, and creation time.

func (*Catalog) VecColumns 

func (cat *Catalog) VecColumns() []SystemRow

VecColumns returns the rows of the vec_columns system collection (spec 02 §2.5): one row per column of every collection, in (collection, ordinal) order.

func (*Catalog) VecIndexes 

func (cat *Catalog) VecIndexes() []SystemRow

VecIndexes returns the rows of the vec_indexes system collection (spec 02 §2.5). Index registration is owned by the db/index layer ([14], [07], [08]); the catalog holds none in this slice, so the result is empty and grows when the db layer records index creation against the catalog.

type Collection 

type Collection struct {
	Schema    *Schema
	ID        uint64
	CreatedAt time.Time
	// contains filtered or unexported fields
}

Collection is a live, registered collection: its schema, the identity state (sequence and deleted-id set), and the storage addressing the engine needs (collection id and the metadata column-id map). It is the handle the db layer ([14]) drives for reads and writes.

func (*Collection) ColID 

func (c *Collection) ColID(name string) (storage.ColID, bool)

ColID returns the engine column id for a metadata column name, or false if the name is not a metadata column of this collection.

func (*Collection) LowerMeta 

func (c *Collection) LowerMeta(supplied map[string]Value, now time.Time) (storage.MetaRow, error)

LowerMeta resolves and validates the supplied metadata into the engine row, keyed by engine ColID (spec 02 §9.6, §10.5). now is the transaction start time for DEFAULT NOW(). It returns the storage MetaRow ready for an insert.

func (*Collection) NextID 

func (c *Collection) NextID() (PointID, error)

NextID returns the next auto-assigned point id (spec 02 §3.5), or ErrIDRequired if the collection does not auto-assign.

func (*Collection) NoteDeleted 

func (c *Collection) NoteDeleted(pid PointID)

NoteDeleted records a deleted point id so it is never reused (spec 02 §3.6).

func (*Collection) PrepareID 

func (c *Collection) PrepareID(supplied *PointID) (PointID, storage.PointID, error)

PrepareID validates or assigns the point id for a write (spec 02 §3.5, §3.6, §13.2, §13.17). A nil id auto-assigns; a supplied id is form-checked, rejected if it was deleted (non-reuse), and advances the sequence. It returns the resolved id and its engine-space fold.

func (*Collection) StorageDef 

func (c *Collection) StorageDef() storage.CollectionDef

StorageDef returns the engine collection definition lowered from the schema (spec 02 §11.2). The db layer passes it to storage.Engine.CreateCollection.

func (*Collection) ValidateVector 

func (c *Collection) ValidateVector(column string, vec []float32) error

ValidateVector checks the vector for the named vector column (spec 02 §4.5).

type ColumnDef 

type ColumnDef struct {
	Name    string
	Kind    ColumnKind
	Ordinal int // position in the schema, 0-based (spec 02 §2.3)

	// Vector-column fields (Kind == ColumnVector):
	Dim       uint32      // element count, 1..65535 (spec 02 §4.2)
	ElemType  ElementType // fp32/fp16/int8/binary
	VecMetric Metric      // bound metric, defaulted per element type if unset
	Normalize bool        // WITH NORMALIZATION ON (spec 02 §4.7)
	Int8Scale float32     // symmetric dequantization scale for int8 (spec 02 §4.3)

	// Metadata-column fields (Kind == ColumnMetadata):
	DataType   Kind   // the scalar/composite kind of the column
	ArrayElem  Kind   // element kind for KindArray columns, KindNull otherwise
	Nullable   bool   // whether the column accepts NULL (spec 02 §9.7)
	Default    *Value // default value expression, nil if none (spec 02 §9.6)
	DefaultNow bool   // DEFAULT NOW() for a timestamp column (spec 02 §9.6)
}

ColumnDef describes one column of a collection schema (spec 02 §16.3). The vector fields are set for ColumnVector, the metadata fields for ColumnMetadata.

func (*ColumnDef) IsVector 

func (c *ColumnDef) IsVector() bool

IsVector reports whether the column is a vector column.

type ColumnKind 

type ColumnKind uint8

ColumnKind distinguishes vector columns from metadata columns (spec 02 §16.3). 

const (
	ColumnVector   ColumnKind = 1 // a fixed-length vector column (spec 02 §4)
	ColumnMetadata ColumnKind = 2 // a typed scalar/composite column (spec 02 §7)
)

type ElementType 

type ElementType uint8

ElementType is the stored element representation of a vector column (spec 02 §4.3). 

const (
	ElemFP32   ElementType = 1 // 4 bytes/elem, default (spec 02 §4.3)
	ElemFP16   ElementType = 2 // 2 bytes/elem
	ElemInt8   ElementType = 3 // 1 byte/elem, scalar-quantized, dequantized on read
	ElemBinary ElementType = 4 // 1 bit/elem packed, Hamming/Jaccard only
)

func (ElementType) String 

func (e ElementType) String() string

String renders an ElementType as its SQL keyword (spec 02 §4.3).

type IDKind 

type IDKind uint8

IDKind is the point id form a collection uses for all its points (spec 02 §3.2). A collection uses exactly one form, fixed at creation. 

const (
	IDBigInt IDKind = 1 // uint64, the compact form; the only form auto-assignment uses
	IDText   IDKind = 2 // UTF-8 string up to 255 bytes
	IDBlob   IDKind = 3 // raw bytes up to 255 bytes
)

func (IDKind) String 

func (k IDKind) String() string

String renders an IDKind for the catalog.

type Kind 

type Kind uint8

Kind identifies the concrete type inside a Value (spec 02 §16.1). It is the metadata value type system of spec 02 §7: the scalar, temporal, text, binary, JSON, and array types a metadata column may hold. 

const (
	KindNull      Kind = 0  // the absent value (spec 02 §7.11)
	KindBigInt    Kind = 1  // int64 (spec 02 §7.3)
	KindInt       Kind = 2  // int32, stored in the int64 slot (spec 02 §7.3)
	KindDouble    Kind = 3  // float64 (spec 02 §7.4)
	KindReal      Kind = 4  // float32, stored in the float64 slot (spec 02 §7.4)
	KindBool      Kind = 5  // boolean (spec 02 §7.5)
	KindText      Kind = 6  // UTF-8 string (spec 02 §7.6)
	KindTimestamp Kind = 7  // microseconds since the Unix epoch, UTC (spec 02 §7.7)
	KindBlob      Kind = 8  // raw bytes (spec 02 §7.8)
	KindJSON      Kind = 9  // JSON value stored as UTF-8 (spec 02 §7.9)
	KindArray     Kind = 10 // homogeneous array of a scalar kind (spec 02 §7.10)
)

func (Kind) String 

func (k Kind) String() string

String renders a Kind as its canonical SQL type name (spec 02 §7.2).

type Metric 

type Metric uint8

Metric is the distance metric bound to a vector column (spec 02 §8). The metric is a property of the data, fixed once any index is built (spec 02 §8.1, §8.6). 

const (
	MetricCosine       Metric = 1 // <=> vector_cosine_ops (spec 02 §8.2)
	MetricL2           Metric = 2 // <-> vector_l2_ops
	MetricInnerProduct Metric = 3 // <#> vector_ip_ops
	MetricHamming      Metric = 4 // <~> vector_hamming_ops, binary only
	MetricJaccard      Metric = 5 // <%> vector_jaccard_ops, binary only
	MetricDotSparse    Metric = 6 // <#> sparsevec_ip_ops, sparse only
)

func DefaultMetric 

func DefaultMetric(e ElementType) Metric

DefaultMetric returns the metric applied to a vector column of the given element type when no METRIC clause is given (spec 02 §8.7).

func (Metric) Opclass 

func (m Metric) Opclass() string

Opclass renders the index operator class name bound to this metric (spec 02 §8.2, §15 glossary): the pgvector-compatible opclass identifier.

func (Metric) String 

func (m Metric) String() string

String renders a Metric for diagnostics and the catalog (spec 02 §8.2).

type PointID 

type PointID struct {
	Kind  IDKind
	U     uint64
	Bytes []byte // string ids are stored as their UTF-8 bytes
}

PointID is the application-facing identity of a point in one of its three forms (spec 02 §3.2): a uint64, a string, or a byte slice. Exactly one form is valid per collection, matching the schema's IDKind.

func BigIntID 

func BigIntID(n uint64) PointID

BigIntID builds a uint64 point id (spec 02 §3.2).

func BlobID 

func BlobID(b []byte) PointID

BlobID builds a bytes point id (spec 02 §3.2).

func TextID 

func TextID(s string) PointID

TextID builds a string point id (spec 02 §3.2).

func (PointID) String 

func (p PointID) String() string

String renders a point id for diagnostics and tie-break ordering.

type Schema 

type Schema struct {
	Name          string
	IDName        string // the primary-key column name (spec 02 §2.3, default "id")
	IDKind        IDKind
	AutoIncrement bool // id is auto-assigned from a sequence (spec 02 §3.5)
	Mode          SchemaMode
	Columns       []ColumnDef
}

Schema describes a collection's schema (spec 02 §16.3). It is the authoritative source of truth for column names, types, dimensions, metrics, nullability, and defaults (spec 02 §2.3).

func (*Schema) Column 

func (s *Schema) Column(name string) *ColumnDef

Column returns the column with the given name, or nil if absent.

func (*Schema) MetadataColumns 

func (s *Schema) MetadataColumns() []ColumnDef

MetadataColumns returns the metadata columns in schema order (spec 02 §7).

func (*Schema) VectorColumns 

func (s *Schema) VectorColumns() []ColumnDef

VectorColumns returns the vector columns in schema order (spec 02 §4.8).

type SchemaMode 

type SchemaMode uint8

SchemaMode is the schema enforcement mode (spec 02 §2.4). 

const (
	SchemaFixed    SchemaMode = 1 // declared types enforced on every write (default)
	SchemaOptional SchemaMode = 2 // metadata types inferred, may be heterogeneous
)

func (SchemaMode) String 

func (m SchemaMode) String() string

String renders a SchemaMode for the catalog (vec_collections.schema_mode).

type SystemRow 

type SystemRow map[string]string

SystemRow is one row of a system collection, a column-name to string map. Every cell renders as text, matching the read-only virtual-table surface of spec 02 §2.5 where every system column is TEXT/INT.

type Value 

type Value struct {
	// contains filtered or unexported fields
}

Value is a discriminated union for a metadata column value (spec 02 §16.1). Scalar kinds store in the i or f slot and never allocate; Text, Blob, JSON, and Array carry a payload. The zero Value is NULL.

func Array 

func Array(elem Kind, items []Value) Value

Array builds a homogeneous array value over elem-kinded scalars (spec 02 §7.10).

func BigInt 

func BigInt(n int64) Value

BigInt builds a 64-bit signed integer value (spec 02 §7.3).

func Blob 

func Blob(p []byte) Value

Blob builds a raw byte value (spec 02 §7.8).

func Bool 

func Bool(t bool) Value

Bool builds a boolean value (spec 02 §7.5).

func Double 

func Double(f float64) Value

Double builds a 64-bit float value (spec 02 §7.4).

func Int 

func Int(n int32) Value

Int builds a 32-bit signed integer value (spec 02 §7.3).

func JSON 

func JSON(s string) Value

JSON builds a JSON value stored as UTF-8 (spec 02 §7.9). The caller is responsible for supplying valid JSON; the binder validates it (spec 02 §7.9).

func Real 

func Real(f float32) Value

Real builds a 32-bit float value, widened to float64 for storage (spec 02 §7.4).

func Text 

func Text(s string) Value

Text builds a UTF-8 string value (spec 02 §7.6).

func Timestamp 

func Timestamp(t time.Time) Value

Timestamp builds a timestamp value stored as UTC epoch microseconds (spec 02 §7.7).

func (Value) Array 

func (v Value) Array() (Kind, []Value)

Array returns the element kind and the array elements (KindArray).

func (Value) BigInt 

func (v Value) BigInt() int64

BigInt returns the int64 payload (KindBigInt or KindInt).

func (Value) Blob 

func (v Value) Blob() []byte

Blob returns the byte payload (KindBlob).

func (Value) Bool 

func (v Value) Bool() bool

Bool returns the boolean payload (KindBool).

func (Value) Clone 

func (v Value) Clone() Value

Clone returns a deep copy so a stored value never aliases the caller's buffer (spec 02 §13.12 in-place overwrite path).

func (Value) Double 

func (v Value) Double() float64

Double returns the float64 payload (KindDouble or KindReal).

func (Value) Equal 

func (v Value) Equal(other Value) (result bool, known bool)

Equal evaluates v = other under SQL three-valued logic (spec 02 §7.11, §7.13). It returns (result, known): known is false when either side is NULL or when a float NaN is involved, in which case the comparison is UNKNOWN. Numeric kinds compare across the integer/float boundary by value; JSON is never equal via = at the column level (spec 02 §7.13).

func (Value) Int 

func (v Value) Int() int32

Int returns the int32 payload (KindInt).

func (Value) IsNull 

func (v Value) IsNull() bool

IsNull reports whether the value is NULL (spec 02 §7.11).

func (Value) Kind 

func (v Value) Kind() Kind

Kind returns the value's kind.

func (Value) Less 

func (v Value) Less(other Value) (result bool, known bool)

Less evaluates v < other under three-valued logic (spec 02 §7.13). It returns (result, known); known is false when either side is NULL or a NaN is involved.

func (Value) Order 

func (v Value) Order(other Value) int

Order returns a total-order comparison of two non-null values for ORDER BY (spec 02 §7.13): -1, 0, or 1. NaN sorts greatest among floats; NULL handling (NULLS LAST/FIRST) is the caller's, since it is a clause-level choice.

func (Value) Real 

func (v Value) Real() float32

Real returns the float32 payload (KindReal).

func (Value) Text 

func (v Value) Text() string

Text returns the string payload (KindText or KindJSON).

func (Value) Time 

func (v Value) Time() time.Time

Time returns the timestamp as a UTC time.Time (KindTimestamp).

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.