grub

package module
v0.1.17 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Mar 27, 2026 License: MIT Imports: 18 Imported by: 0

README

grub

CI codecov Go Report Card CodeQL Go Reference License Go Version Release

Provider-agnostic storage for Go.

Type-safe CRUD across key-value stores, blob storage, SQL databases, and vector similarity search with a unified interface.

One Interface, Any Backend

// Same code, different providers
sessions := grub.NewStore[Session](redis.New(client))
sessions := grub.NewStore[Session](badger.New(db))
sessions := grub.NewStore[Session](bolt.New(db, "sessions"))

// Type-safe operations
session, _ := sessions.Get(ctx, "session:abc")
sessions.Set(ctx, "session:xyz", &Session{UserID: "123"}, time.Hour)

Swap Redis for BadgerDB. Move from S3 to Azure. Switch databases from SQLite to PostgreSQL. Your business logic stays the same.

// Key-value, blob, SQL, or vector — same patterns
store := grub.NewStore[Config](provider)           // key-value
bucket := grub.NewBucket[Document](provider)       // blob storage
db := grub.NewDatabase[User](conn, "users", renderer)            // SQL
index := grub.NewIndex[Embedding](provider)        // vector search

Four storage modes, consistent API, semantic errors across all providers.

Install

go get github.com/zoobz-io/grub

Requires Go 1.24+.

Quick Start

package main

import (
    "context"
    "fmt"
    "time"

    "github.com/zoobz-io/grub"
    "github.com/zoobz-io/grub/redis"
    goredis "github.com/redis/go-redis/v9"
)

type Session struct {
    UserID    string `json:"user_id"`
    Token     string `json:"token"`
    ExpiresAt int64  `json:"expires_at"`
}

func main() {
    ctx := context.Background()

    // Connect to Redis
    client := goredis.NewClient(&goredis.Options{Addr: "localhost:6379"})
    defer client.Close()

    // Create type-safe store
    sessions := grub.NewStore[Session](redis.New(client))

    // Store with TTL
    session := &Session{
        UserID:    "user:123",
        Token:     "abc123",
        ExpiresAt: time.Now().Add(24 * time.Hour).Unix(),
    }
    _ = sessions.Set(ctx, "session:abc123", session, 24*time.Hour)

    // Retrieve
    s, _ := sessions.Get(ctx, "session:abc123")
    fmt.Println(s.UserID) // user:123
}

Capabilities

Feature Description Docs
Key-Value Store Sessions, cache, config with optional TTL Providers
Blob Storage Files and documents with metadata Lifecycle
SQL Database Structured records with query capabilities Concepts
Vector Search Similarity search with metadata filtering Providers
Atomic Views Field-level access for encryption pipelines Architecture
Semantic Errors ErrNotFound, ErrDuplicate across all providers API Reference
Custom Codecs JSON default, Gob available, or bring your own Concepts

Why grub?

  • Type-safe — Generics eliminate runtime type assertions
  • Swap backends — Change providers without touching business logic
  • Consistent errors — Same error types whether you're using Redis or S3
  • Atomic views — Field-level access for framework internals (encryption, pipelines)
  • Isolated dependencies — Each provider is a separate module; only pull what you use

Storage Without Coupling

Grub enables a pattern: define storage once, swap implementations freely.

Your domain code works with typed stores. Infrastructure decisions — Redis vs embedded, S3 vs local filesystem, PostgreSQL vs SQLite, Pinecone vs Qdrant — become configuration, not architecture.

// Domain code doesn't know or care about the backend
type SessionStore struct {
    store *grub.Store[Session]
}

func (s *SessionStore) Save(ctx context.Context, session *Session) error {
    return s.store.Set(ctx, "session:"+session.Token, session, 24*time.Hour)
}

// Production: Redis
store := grub.NewStore[Session](redis.New(redisClient))

// Development: embedded BadgerDB
store := grub.NewStore[Session](badger.New(localDB))

// Testing: in-memory
store := grub.NewStore[Session](badger.New(memDB))

One interface. Any backend. Zero vendor lock-in.

Documentation

  • Overview — Design philosophy and architecture
Learn
Guides
  • Providers — Setup and configuration for all backends
  • Lifecycle — CRUD operations and batch processing
  • Pagination — Listing and iterating large datasets
  • Testing — Mocks, embedded DBs, testcontainers
  • Best Practices — Key design, error handling, performance
Cookbook
  • Caching — Cache-aside, read-through, TTL strategies
  • Migrations — Switching providers without downtime
  • Multi-Tenant — Tenant isolation patterns
Reference
  • API — Complete API documentation
  • Providers — Provider-specific behaviors and limitations

Contributing

See CONTRIBUTING.md for guidelines.

License

MIT License — see LICENSE for details.

Documentation

Overview

Package grub provides a provider-agnostic storage interface. Atoms serve as the type-agnostic API boundary; providers store data in its intended structure, not as atoms.

Index

Constants

View Source 
const (
	DistanceL2           = shared.DistanceL2
	DistanceCosine       = shared.DistanceCosine
	DistanceInnerProduct = shared.DistanceInnerProduct
)

Distance metric constants.

Variables

View Source 
var (
	ErrNotFound             = shared.ErrNotFound
	ErrDuplicate            = shared.ErrDuplicate
	ErrConflict             = shared.ErrConflict
	ErrConstraint           = shared.ErrConstraint
	ErrInvalidKey           = shared.ErrInvalidKey
	ErrReadOnly             = shared.ErrReadOnly
	ErrTableExists          = shared.ErrTableExists
	ErrTableNotFound        = shared.ErrTableNotFound
	ErrTTLNotSupported      = shared.ErrTTLNotSupported
	ErrDimensionMismatch    = shared.ErrDimensionMismatch
	ErrInvalidVector        = shared.ErrInvalidVector
	ErrIndexNotReady        = shared.ErrIndexNotReady
	ErrInvalidQuery         = shared.ErrInvalidQuery
	ErrOperatorNotSupported = shared.ErrOperatorNotSupported
	ErrFilterNotSupported   = shared.ErrFilterNotSupported
	ErrNoPrimaryKey         = shared.ErrNoPrimaryKey
	ErrMultiplePrimaryKeys  = shared.ErrMultiplePrimaryKeys
)

Semantic errors for storage operations (re-exported from internal/shared).

View Source 
var (
	// QueryAll returns all records from the table.
	QueryAll = edamame.NewQueryStatement("query", "Query all records", edamame.QuerySpec{})

	// CountAll counts all records in the table.
	CountAll = edamame.NewAggregateStatement("count", "Count all records", edamame.AggCount, edamame.AggregateSpec{})
)

Default statements for common operations.

Functions

This section is empty.

Types

type AfterDelete 

type AfterDelete interface {
	AfterDelete(ctx context.Context) error
}

AfterDelete is called after a record has been successfully deleted. Invoked on a zero-value T (no loaded state). Return an error to signal a post-delete failure.

type AfterLoad 

type AfterLoad interface {
	AfterLoad(ctx context.Context) error
}

AfterLoad is called after T has been loaded and decoded. Return an error to signal a post-load invariant failure.

type AfterSave 

type AfterSave interface {
	AfterSave(ctx context.Context) error
}

AfterSave is called after T has been successfully persisted. Return an error to signal a post-save invariant failure.

type AtomicBucket 

type AtomicBucket interface {
	// Spec returns the atom spec describing the payload T structure.
	Spec() atom.Spec

	// Get retrieves the blob at key with atomized payload.
	// Returns ErrNotFound if the key does not exist.
	Get(ctx context.Context, key string) (*AtomicObject, error)

	// Put stores an object with atomized payload at key.
	Put(ctx context.Context, key string, obj *AtomicObject) error

	// Delete removes the blob at key.
	// Returns ErrNotFound if the key does not exist.
	Delete(ctx context.Context, key string) error

	// Exists checks whether a key exists.
	Exists(ctx context.Context, key string) (bool, error)
}

AtomicBucket defines atom-based blob storage operations. atomix.Bucket[T] satisfies this interface, enabling type-agnostic access for framework internals (field-level encryption, pipelines, etc.).

type AtomicDatabase 

type AtomicDatabase interface {
	// Table returns the table name this provider manages.
	Table() string

	// Spec returns the atom spec describing the table's structure.
	Spec() atom.Spec

	// Get retrieves the record at key as an Atom.
	// Returns ErrNotFound if the key does not exist.
	Get(ctx context.Context, key string) (*atom.Atom, error)

	// Set stores an Atom at key (insert or update).
	Set(ctx context.Context, key string, data *atom.Atom) error

	// Delete removes the record at key.
	// Returns ErrNotFound if the key does not exist.
	Delete(ctx context.Context, key string) error

	// Exists checks whether a record exists at key.
	Exists(ctx context.Context, key string) (bool, error)

	// ExecQuery executes a query statement and returns atoms.
	ExecQuery(ctx context.Context, stmt edamame.QueryStatement, params map[string]any) ([]*atom.Atom, error)

	// ExecSelect executes a select statement and returns an atom.
	ExecSelect(ctx context.Context, stmt edamame.SelectStatement, params map[string]any) (*atom.Atom, error)
}

AtomicDatabase defines atom-based storage operations for a single table. atomix.Database[T] satisfies this interface, enabling type-agnostic access for framework internals (field-level encryption, pipelines, etc.).

type AtomicDocument 

type AtomicDocument = shared.AtomicDocument

AtomicDocument holds a search document with atomized content. Used by AtomicSearch for type-agnostic access to document data.

type AtomicIndex 

type AtomicIndex interface {
	// Spec returns the atom spec describing the metadata type's structure.
	Spec() atom.Spec

	// Get retrieves the vector at ID with atomized metadata.
	// Returns ErrNotFound if the ID does not exist.
	Get(ctx context.Context, id uuid.UUID) (*AtomicVector, error)

	// Upsert stores a vector with atomized metadata.
	Upsert(ctx context.Context, id uuid.UUID, vector []float32, metadata *atom.Atom) error

	// Delete removes the vector at ID.
	// Returns ErrNotFound if the ID does not exist.
	Delete(ctx context.Context, id uuid.UUID) error

	// Exists checks whether an ID exists.
	Exists(ctx context.Context, id uuid.UUID) (bool, error)

	// Search performs similarity search returning atomized results.
	Search(ctx context.Context, vector []float32, k int, filter *atom.Atom) ([]AtomicVector, error)

	// Query performs similarity search with vecna filter support.
	Query(ctx context.Context, vector []float32, k int, filter *vecna.Filter) ([]AtomicVector, error)

	// Filter returns vectors matching the metadata filter without similarity search.
	Filter(ctx context.Context, filter *vecna.Filter, limit int) ([]AtomicVector, error)
}

AtomicIndex defines atom-based vector storage operations. atomix.Index[T] satisfies this interface, enabling type-agnostic access for framework internals (field-level encryption, pipelines, etc.).

type AtomicObject 

type AtomicObject = shared.AtomicObject

AtomicObject holds blob metadata with an atomized payload. Used by AtomicBucket for type-agnostic access to blob data.

type AtomicSearch 

type AtomicSearch interface {
	// Index returns the index name this provider manages.
	Index() string

	// Spec returns the atom spec describing the document type's structure.
	Spec() atom.Spec

	// Get retrieves the document at ID with atomized content.
	// Returns ErrNotFound if the ID does not exist.
	Get(ctx context.Context, id string) (*AtomicDocument, error)

	// Index stores a document with atomized content.
	IndexDoc(ctx context.Context, id string, doc *atom.Atom) error

	// Delete removes the document at ID.
	// Returns ErrNotFound if the ID does not exist.
	Delete(ctx context.Context, id string) error

	// Exists checks whether a document ID exists.
	Exists(ctx context.Context, id string) (bool, error)

	// Search performs a search returning atomized results.
	Search(ctx context.Context, search *lucene.Search) ([]AtomicDocument, error)
}

AtomicSearch defines atom-based search operations. atomix.Search[T] satisfies this interface, enabling type-agnostic access for framework internals (field-level encryption, pipelines, etc.).

type AtomicStore 

type AtomicStore interface {
	// Spec returns the atom spec describing the stored type's structure.
	Spec() atom.Spec

	// Get retrieves the value at key as an Atom.
	// Returns ErrNotFound if the key does not exist.
	Get(ctx context.Context, key string) (*atom.Atom, error)

	// Set stores an Atom at key with optional TTL.
	// TTL of 0 means no expiration.
	Set(ctx context.Context, key string, data *atom.Atom, ttl time.Duration) error

	// Delete removes the value at key.
	// Returns ErrNotFound if the key does not exist.
	Delete(ctx context.Context, key string) error

	// Exists checks whether a key exists.
	Exists(ctx context.Context, key string) (bool, error)
}

AtomicStore defines atom-based key-value storage operations. atomix.Store[T] satisfies this interface, enabling type-agnostic access for framework internals (field-level encryption, pipelines, etc.).

type AtomicVector 

type AtomicVector = shared.AtomicVector

AtomicVector holds vector data with an atomized metadata payload. Used by AtomicIndex for type-agnostic access to vector data.

type BeforeDelete 

type BeforeDelete interface {
	BeforeDelete(ctx context.Context) error
}

BeforeDelete is called before deleting a record. Invoked on a zero-value T (no loaded state). Return an error to abort the operation.

type BeforeSave 

type BeforeSave interface {
	BeforeSave(ctx context.Context) error
}

BeforeSave is called before persisting T. Return an error to abort the operation.

type Bucket 

type Bucket[T any] struct {
	// contains filtered or unexported fields
}

Bucket provides type-safe blob storage operations for T. Wraps a BucketProvider, handling serialization of Object[T] to/from bytes.

func NewBucket 

func NewBucket[T any](provider BucketProvider) *Bucket[T]

NewBucket creates a Bucket for type T backed by the given provider. Uses JSON codec by default.

func NewBucketWithCodec 

func NewBucketWithCodec[T any](provider BucketProvider, codec Codec) *Bucket[T]

NewBucketWithCodec creates a Bucket for type T with a custom codec.

func (*Bucket[T]) Atomic 

func (b *Bucket[T]) Atomic() *atomix.Bucket[T]

Atomic returns an atom-based view of this bucket. The returned atomix.Bucket satisfies the AtomicBucket interface. The instance is created once and cached for subsequent calls. Panics if T is not atomizable (a programmer error).

func (*Bucket[T]) Delete 

func (b *Bucket[T]) Delete(ctx context.Context, key string) error

Delete removes the object at key.

func (*Bucket[T]) Exists 

func (b *Bucket[T]) Exists(ctx context.Context, key string) (bool, error)

Exists checks whether a key exists.

func (*Bucket[T]) Get 

func (b *Bucket[T]) Get(ctx context.Context, key string) (*Object[T], error)

Get retrieves the object at key.

func (*Bucket[T]) List 

func (b *Bucket[T]) List(ctx context.Context, prefix string, limit int) ([]ObjectInfo, error)

List returns object info for keys matching the given prefix. Limit of 0 means no limit.

func (*Bucket[T]) Put 

func (b *Bucket[T]) Put(ctx context.Context, obj *Object[T]) error

Put stores an object at key.

type BucketProvider 

type BucketProvider interface {
	// Get retrieves the blob at key.
	// Returns ErrNotFound if the key does not exist.
	Get(ctx context.Context, key string) ([]byte, *ObjectInfo, error)

	// Put stores data at key with associated metadata.
	Put(ctx context.Context, key string, data []byte, info *ObjectInfo) error

	// Delete removes the blob at key.
	// Returns ErrNotFound if the key does not exist.
	Delete(ctx context.Context, key string) error

	// Exists checks whether a key exists.
	Exists(ctx context.Context, key string) (bool, error)

	// List returns object info for keys matching the given prefix.
	// Limit of 0 means no limit.
	List(ctx context.Context, prefix string, limit int) ([]ObjectInfo, error)
}

BucketProvider defines raw blob storage operations. Implementations (s3, gcs, azure) satisfy this interface.

type Codec 

type Codec interface {
	Encode(v any) ([]byte, error)
	Decode(data []byte, v any) error
}

Codec defines encoding/decoding operations for Store values.

type Database 

type Database[T any] struct {
	// contains filtered or unexported fields
}

Database provides type-safe SQL storage operations for T. Constructed via NewDatabase (executor path) or NewDatabaseFromProvider (provider path). Builder methods (Query, Select, Insert, etc.), Tx variants, Executor(), and Atomic() require the executor path and will panic if called on a provider-backed instance.

func NewDatabase 

func NewDatabase[T any](db *sqlx.DB, table string, renderer astql.Renderer) *Database[T]

NewDatabase creates a Database for type T. The primary key column is derived from the struct field tagged with constraints:"primarykey". Panics if T is not a valid struct or has incorrect primary key tags (programmer error). Use the *Tx method variants (GetTx, SetTx, etc.) for transaction support.

func NewDatabaseFromProvider added in v0.1.14 

func NewDatabaseFromProvider[T any](provider DatabaseProvider, table string) *Database[T]

NewDatabaseFromProvider creates a Database for type T backed by a DatabaseProvider. Uses JSON codec by default. Builder methods (Query, Select, Insert, Modify, Remove, Count), Tx variants, Executor(), and Atomic() are not available and will panic if called.

func NewDatabaseFromProviderWithCodec added in v0.1.14 

func NewDatabaseFromProviderWithCodec[T any](provider DatabaseProvider, table string, codec Codec) *Database[T]

NewDatabaseFromProviderWithCodec creates a Database for type T with a custom codec. Builder methods, Tx variants, Executor(), and Atomic() are not available and will panic if called.

func (*Database[T]) Atomic 

func (d *Database[T]) Atomic() AtomicDatabase

Atomic returns an atom-based view of this database. The returned atomix.Database satisfies the AtomicDatabase interface. The instance is created once and cached for subsequent calls. Panics if T is not atomizable (a programmer error) or if constructed from a provider.

func (*Database[T]) Count 

func (d *Database[T]) Count() *soy.Aggregate[T]

Count returns an aggregate builder for counting records. Panics if the database was constructed from a provider.

func (*Database[T]) Delete 

func (d *Database[T]) Delete(ctx context.Context, key string) error

Delete removes the record at key.

func (*Database[T]) DeleteTx 

func (d *Database[T]) DeleteTx(ctx context.Context, tx *sqlx.Tx, key string) error

DeleteTx removes the record at key within a transaction. Panics if the database was constructed from a provider.

func (*Database[T]) ExecAggregate 

func (d *Database[T]) ExecAggregate(ctx context.Context, stmt edamame.AggregateStatement, params map[string]any) (float64, error)

ExecAggregate executes an aggregate statement.

func (*Database[T]) ExecAggregateTx 

func (d *Database[T]) ExecAggregateTx(ctx context.Context, tx *sqlx.Tx, stmt edamame.AggregateStatement, params map[string]any) (float64, error)

ExecAggregateTx executes an aggregate statement within a transaction. Panics if the database was constructed from a provider.

func (*Database[T]) ExecQuery 

func (d *Database[T]) ExecQuery(ctx context.Context, stmt edamame.QueryStatement, params map[string]any) ([]*T, error)

ExecQuery executes a query statement and returns multiple records.

func (*Database[T]) ExecQueryTx 

func (d *Database[T]) ExecQueryTx(ctx context.Context, tx *sqlx.Tx, stmt edamame.QueryStatement, params map[string]any) ([]*T, error)

ExecQueryTx executes a query statement within a transaction and returns multiple records. Panics if the database was constructed from a provider.

func (*Database[T]) ExecSelect 

func (d *Database[T]) ExecSelect(ctx context.Context, stmt edamame.SelectStatement, params map[string]any) (*T, error)

ExecSelect executes a select statement and returns a single record.

func (*Database[T]) ExecSelectTx 

func (d *Database[T]) ExecSelectTx(ctx context.Context, tx *sqlx.Tx, stmt edamame.SelectStatement, params map[string]any) (*T, error)

ExecSelectTx executes a select statement within a transaction and returns a single record. Panics if the database was constructed from a provider.

func (*Database[T]) ExecUpdate 

func (d *Database[T]) ExecUpdate(ctx context.Context, stmt edamame.UpdateStatement, params map[string]any) (*T, error)

ExecUpdate executes an update statement.

func (*Database[T]) ExecUpdateTx 

func (d *Database[T]) ExecUpdateTx(ctx context.Context, tx *sqlx.Tx, stmt edamame.UpdateStatement, params map[string]any) (*T, error)

ExecUpdateTx executes an update statement within a transaction. Panics if the database was constructed from a provider.

func (*Database[T]) Executor 

func (d *Database[T]) Executor() *edamame.Executor[T]

Executor returns the underlying edamame Executor for advanced query operations. Panics if the database was constructed from a provider.

func (*Database[T]) Exists 

func (d *Database[T]) Exists(ctx context.Context, key string) (bool, error)

Exists checks whether a record exists at key.

func (*Database[T]) ExistsTx 

func (d *Database[T]) ExistsTx(ctx context.Context, tx *sqlx.Tx, key string) (bool, error)

ExistsTx checks whether a record exists at key within a transaction. Panics if the database was constructed from a provider.

func (*Database[T]) Get 

func (d *Database[T]) Get(ctx context.Context, key string) (*T, error)

Get retrieves the record at key as T. Returns ErrNotFound if the key does not exist.

func (*Database[T]) GetTx 

func (d *Database[T]) GetTx(ctx context.Context, tx *sqlx.Tx, key string) (*T, error)

GetTx retrieves the record at key as T within a transaction. Returns ErrNotFound if the key does not exist. Panics if the database was constructed from a provider.

func (*Database[T]) Insert 

func (d *Database[T]) Insert() *soy.Create[T]

Insert returns an insert builder (auto-generates PK). Panics if the database was constructed from a provider.

func (*Database[T]) InsertFull 

func (d *Database[T]) InsertFull() *soy.Create[T]

InsertFull returns an insert builder that includes the PK field. Panics if the database was constructed from a provider.

func (*Database[T]) Modify 

func (d *Database[T]) Modify() *soy.Update[T]

Modify returns an update builder. Panics if the database was constructed from a provider.

func (*Database[T]) Query 

func (d *Database[T]) Query() *soy.Query[T]

Query returns a query builder for fetching multiple records. Panics if the database was constructed from a provider.

func (*Database[T]) Remove 

func (d *Database[T]) Remove() *soy.Delete[T]

Remove returns a delete builder. Panics if the database was constructed from a provider.

func (*Database[T]) Select 

func (d *Database[T]) Select() *soy.Select[T]

Select returns a select builder for fetching a single record. Panics if the database was constructed from a provider.

func (*Database[T]) Set 

func (d *Database[T]) Set(ctx context.Context, _ string, value *T) error

Set stores value at key (insert or update via upsert).

func (*Database[T]) SetTx 

func (d *Database[T]) SetTx(ctx context.Context, tx *sqlx.Tx, _ string, value *T) error

SetTx stores value at key within a transaction (insert or update via upsert). Panics if the database was constructed from a provider.

type DatabaseProvider added in v0.1.14 

type DatabaseProvider interface {
	// Get retrieves the record at key.
	// Returns ErrNotFound if the key does not exist.
	Get(ctx context.Context, key string) ([]byte, error)

	// Set stores value at key (insert or update).
	Set(ctx context.Context, key string, value []byte) error

	// Delete removes the record at key.
	// Returns ErrNotFound if the key does not exist.
	Delete(ctx context.Context, key string) error

	// Exists checks whether a record exists at key.
	Exists(ctx context.Context, key string) (bool, error)

	// ExecQuery executes a named query and returns multiple records as raw bytes.
	ExecQuery(ctx context.Context, stmt edamame.QueryStatement, params map[string]any) ([][]byte, error)

	// ExecSelect executes a named select and returns a single record as raw bytes.
	ExecSelect(ctx context.Context, stmt edamame.SelectStatement, params map[string]any) ([]byte, error)

	// ExecUpdate executes a named update and returns the affected record as raw bytes.
	ExecUpdate(ctx context.Context, stmt edamame.UpdateStatement, params map[string]any) ([]byte, error)

	// ExecAggregate executes an aggregate and returns a scalar.
	ExecAggregate(ctx context.Context, stmt edamame.AggregateStatement, params map[string]any) (float64, error)
}

DatabaseProvider defines raw SQL storage operations. Implementations satisfy this interface to provide a mockable database backend. Use NewDatabaseProvider to create a provider backed by a sqlx.DB connection.

func NewDatabaseProvider added in v0.1.14 

func NewDatabaseProvider[T any](db *sqlx.DB, table string, renderer astql.Renderer) DatabaseProvider

NewDatabaseProvider creates a DatabaseProvider backed by a sqlx.DB connection. The primary key column is derived from the struct field tagged with constraints:"primarykey". Panics if T is not a valid struct or has incorrect primary key tags (programmer error). Lifecycle hooks (OnScan, OnRecord) are registered on the soy instance.

type DistanceMetric 

type DistanceMetric = shared.DistanceMetric

DistanceMetric is re-exported from internal/shared for the public API.

type Document 

type Document[T any] struct {
	ID      string
	Content T
	Score   float64 // Populated on search results
}

Document represents a search document with ID and typed content.

type GobCodec 

type GobCodec struct{}

GobCodec implements Codec using Gob encoding.

func (GobCodec) Decode 

func (GobCodec) Decode(data []byte, v any) error

Decode unmarshals Gob data into v.

func (GobCodec) Encode 

func (GobCodec) Encode(v any) ([]byte, error)

Encode marshals v to Gob bytes.

type Index 

type Index[T any] struct {
	// contains filtered or unexported fields
}

Index provides type-safe vector storage operations with metadata of type T. Wraps a VectorProvider, handling serialization of T to/from map[string]any.

func NewIndex 

func NewIndex[T any](provider VectorProvider) *Index[T]

NewIndex creates an Index for metadata type T backed by the given provider. Uses JSON codec by default.

func NewIndexWithCodec 

func NewIndexWithCodec[T any](provider VectorProvider, codec Codec) *Index[T]

NewIndexWithCodec creates an Index for metadata type T with a custom codec.

func (*Index[T]) Atomic 

func (i *Index[T]) Atomic() *atomix.Index[T]

Atomic returns an atom-based view of this index. The returned atomix.Index satisfies the AtomicIndex interface. The instance is created once and cached for subsequent calls. Panics if T is not atomizable (a programmer error).

func (*Index[T]) Delete 

func (i *Index[T]) Delete(ctx context.Context, id uuid.UUID) error

Delete removes a vector by ID. Returns ErrNotFound if the ID does not exist.

func (*Index[T]) DeleteBatch 

func (i *Index[T]) DeleteBatch(ctx context.Context, ids []uuid.UUID) error

DeleteBatch removes multiple vectors by ID. Non-existent IDs are silently ignored.

func (*Index[T]) Exists 

func (i *Index[T]) Exists(ctx context.Context, id uuid.UUID) (bool, error)

Exists checks whether a vector ID exists.

func (*Index[T]) Filter 

func (i *Index[T]) Filter(ctx context.Context, filter *vecna.Filter, limit int) ([]*Vector[T], error)

Filter returns vectors matching the metadata filter without similarity search. Result ordering is provider-dependent and not guaranteed. Limit of 0 returns all matching vectors. Returns ErrFilterNotSupported if the provider cannot perform metadata-only filtering.

func (*Index[T]) Get 

func (i *Index[T]) Get(ctx context.Context, id uuid.UUID) (*Vector[T], error)

Get retrieves a vector by ID. Returns ErrNotFound if the ID does not exist.

func (*Index[T]) List 

func (i *Index[T]) List(ctx context.Context, limit int) ([]uuid.UUID, error)

List returns vector IDs. Limit of 0 means no limit.

func (*Index[T]) Query 

func (i *Index[T]) Query(ctx context.Context, vector []float32, k int, filter *vecna.Filter) ([]*Vector[T], error)

Query performs similarity search with vecna filter support. Returns ErrInvalidQuery if the filter contains validation errors. Returns ErrOperatorNotSupported if the provider doesn't support an operator.

func (*Index[T]) Search 

func (i *Index[T]) Search(ctx context.Context, vector []float32, k int, filter *T) ([]*Vector[T], error)

Search performs similarity search and returns the k nearest neighbors. filter is optional metadata filtering (nil means no filter).

func (*Index[T]) Upsert 

func (i *Index[T]) Upsert(ctx context.Context, id uuid.UUID, vector []float32, metadata *T) error

Upsert stores or updates a vector with associated metadata. If the ID exists, the vector and metadata are replaced.

func (*Index[T]) UpsertBatch 

func (i *Index[T]) UpsertBatch(ctx context.Context, vectors []Vector[T]) error

UpsertBatch stores or updates multiple vectors.

type JSONCodec 

type JSONCodec struct{}

JSONCodec implements Codec using JSON encoding.

func (JSONCodec) Decode 

func (JSONCodec) Decode(data []byte, v any) error

Decode unmarshals JSON data into v.

func (JSONCodec) Encode 

func (JSONCodec) Encode(v any) ([]byte, error)

Encode marshals v to JSON bytes.

type Object 

type Object[T any] struct {
	Key         string            `json:"key" atom:"key"`
	ContentType string            `json:"content_type" atom:"content_type"`
	Size        int64             `json:"size" atom:"size"`
	ETag        string            `json:"etag,omitempty" atom:"etag"`
	Metadata    map[string]string `json:"metadata,omitempty" atom:"metadata"`
	Data        T                 `json:"data" atom:"data"`
}

Object wraps payload T with blob metadata for atomization. The entire structure is atomizable, enabling field-level operations on both metadata and payload.

type ObjectInfo 

type ObjectInfo = shared.ObjectInfo

ObjectInfo is re-exported from internal/shared for the public API. 

type Search[T any] struct {
	// contains filtered or unexported fields
}

Search provides type-safe search operations for documents of type T. Wraps a SearchProvider, handling serialization of T to/from bytes.

func NewSearch 

func NewSearch[T any](provider SearchProvider, index string) *Search[T]

NewSearch creates a Search for type T backed by the given provider. Uses JSON codec by default.

func NewSearchWithCodec 

func NewSearchWithCodec[T any](provider SearchProvider, index string, codec Codec) *Search[T]

NewSearchWithCodec creates a Search for type T with a custom codec.

func (*Search[T]) Atomic 

func (s *Search[T]) Atomic() *atomix.Search[T]

Atomic returns an atom-based view of this search index. The instance is created once and cached for subsequent calls. Panics if T is not atomizable (a programmer error).

func (*Search[T]) Count 

func (s *Search[T]) Count(ctx context.Context, query lucene.Query) (int64, error)

Count returns the number of documents matching the query.

func (*Search[T]) Delete 

func (s *Search[T]) Delete(ctx context.Context, id string) error

Delete removes a document by ID. Returns ErrNotFound if the ID does not exist.

func (*Search[T]) DeleteBatch 

func (s *Search[T]) DeleteBatch(ctx context.Context, ids []string) error

DeleteBatch removes multiple documents by ID. Non-existent IDs are silently ignored.

func (*Search[T]) Execute 

func (s *Search[T]) Execute(ctx context.Context, search *lucene.Search) (*SearchResult[T], error)

Execute performs a search using the provided search request.

func (*Search[T]) Exists 

func (s *Search[T]) Exists(ctx context.Context, id string) (bool, error)

Exists checks whether a document ID exists.

func (*Search[T]) Get 

func (s *Search[T]) Get(ctx context.Context, id string) (*Document[T], error)

Get retrieves a document by ID. Returns ErrNotFound if the ID does not exist.

func (*Search[T]) Index 

func (s *Search[T]) Index(ctx context.Context, id string, doc *T) error

Index stores a document with the given ID. If the ID exists, the document is replaced.

func (*Search[T]) IndexBatch 

func (s *Search[T]) IndexBatch(ctx context.Context, docs map[string]*T) error

IndexBatch stores multiple documents.

func (*Search[T]) Query 

func (s *Search[T]) Query() *lucene.Builder[T]

Query returns the type-safe query builder for constructing searches. The builder validates field names against the schema of T.

func (*Search[T]) Refresh 

func (s *Search[T]) Refresh(ctx context.Context) error

Refresh makes recent operations visible for search. Use sparingly as it can impact performance.

type SearchHit 

type SearchHit = shared.SearchHit

SearchHit represents a single search result.

type SearchProvider 

type SearchProvider interface {
	// Index stores a document with the given ID.
	// If the ID exists, the document is replaced.
	Index(ctx context.Context, index, id string, doc []byte) error

	// IndexBatch stores multiple documents.
	IndexBatch(ctx context.Context, index string, docs map[string][]byte) error

	// Get retrieves a document by ID.
	// Returns ErrNotFound if the ID does not exist.
	Get(ctx context.Context, index, id string) ([]byte, error)

	// Delete removes a document by ID.
	// Returns ErrNotFound if the ID does not exist.
	Delete(ctx context.Context, index, id string) error

	// DeleteBatch removes multiple documents by ID.
	// Non-existent IDs are silently ignored.
	DeleteBatch(ctx context.Context, index string, ids []string) error

	// Exists checks whether a document ID exists.
	Exists(ctx context.Context, index, id string) (bool, error)

	// Search performs a search using the provided search request.
	Search(ctx context.Context, index string, search *lucene.Search) (*SearchResponse, error)

	// Count returns the number of documents matching the query.
	Count(ctx context.Context, index string, query lucene.Query) (int64, error)

	// Refresh makes recent operations visible for search.
	Refresh(ctx context.Context, index string) error
}

SearchProvider defines raw search/document storage operations. Implementations (opensearch, elasticsearch) satisfy this interface.

type SearchResponse 

type SearchResponse = shared.SearchResponse

SearchResponse represents the response from a search operation.

type SearchResult 

type SearchResult[T any] struct {
	Hits     []*Document[T]
	Total    int64
	MaxScore float64
	// Aggregations are returned as raw JSON for flexibility.
	Aggregations map[string]any
}

SearchResult contains search response data.

type Store 

type Store[T any] struct {
	// contains filtered or unexported fields
}

Store provides type-safe key-value storage operations for T. Wraps a StoreProvider, handling serialization of T to/from bytes.

func NewStore 

func NewStore[T any](provider StoreProvider) *Store[T]

NewStore creates a Store for type T backed by the given provider. Uses JSON codec by default.

func NewStoreWithCodec 

func NewStoreWithCodec[T any](provider StoreProvider, codec Codec) *Store[T]

NewStoreWithCodec creates a Store for type T with a custom codec.

func (*Store[T]) Atomic 

func (s *Store[T]) Atomic() *atomix.Store[T]

Atomic returns an atom-based view of this store. The returned atomix.Store satisfies the AtomicStore interface. The instance is created once and cached for subsequent calls. Panics if T is not atomizable (a programmer error).

func (*Store[T]) Delete 

func (s *Store[T]) Delete(ctx context.Context, key string) error

Delete removes the value at key.

func (*Store[T]) Exists 

func (s *Store[T]) Exists(ctx context.Context, key string) (bool, error)

Exists checks whether a key exists.

func (*Store[T]) Get 

func (s *Store[T]) Get(ctx context.Context, key string) (*T, error)

Get retrieves the value at key as T.

func (*Store[T]) GetBatch 

func (s *Store[T]) GetBatch(ctx context.Context, keys []string) (map[string]*T, error)

GetBatch retrieves multiple values by key. Missing keys are omitted from the result.

func (*Store[T]) List 

func (s *Store[T]) List(ctx context.Context, prefix string, limit int) ([]string, error)

List returns keys matching the given prefix. Limit of 0 means no limit.

func (*Store[T]) Set 

func (s *Store[T]) Set(ctx context.Context, key string, value *T, ttl time.Duration) error

Set stores value at key with optional TTL. TTL of 0 means no expiration.

func (*Store[T]) SetBatch 

func (s *Store[T]) SetBatch(ctx context.Context, items map[string]*T, ttl time.Duration) error

SetBatch stores multiple key-value pairs with optional TTL. TTL of 0 means no expiration.

type StoreProvider 

type StoreProvider interface {
	// Get retrieves the value at key.
	// Returns ErrNotFound if the key does not exist.
	Get(ctx context.Context, key string) ([]byte, error)

	// Set stores value at key with optional TTL.
	// TTL of 0 means no expiration.
	Set(ctx context.Context, key string, value []byte, ttl time.Duration) error

	// Delete removes the value at key.
	// Returns ErrNotFound if the key does not exist.
	Delete(ctx context.Context, key string) error

	// Exists checks whether a key exists.
	Exists(ctx context.Context, key string) (bool, error)

	// List returns keys matching the given prefix.
	// Limit of 0 means no limit.
	List(ctx context.Context, prefix string, limit int) ([]string, error)

	// GetBatch retrieves multiple values by key.
	// Missing keys are omitted from the result (no error).
	GetBatch(ctx context.Context, keys []string) (map[string][]byte, error)

	// SetBatch stores multiple key-value pairs with optional TTL.
	// TTL of 0 means no expiration.
	SetBatch(ctx context.Context, items map[string][]byte, ttl time.Duration) error
}

StoreProvider defines raw key-value storage operations. Implementations (redis, badger, bolt) satisfy this interface.

type Vector 

type Vector[T any] struct {
	ID       uuid.UUID `json:"id" atom:"id"`
	Vector   []float32 `json:"vector" atom:"vector"`
	Score    float32   `json:"score,omitempty" atom:"score"`
	Metadata T         `json:"metadata" atom:"metadata"`
}

Vector wraps payload T (metadata) with vector data for atomization. The entire structure is atomizable, enabling field-level operations on both vector data and metadata.

type VectorInfo 

type VectorInfo = shared.VectorInfo

VectorInfo is re-exported from internal/shared for the public API.

type VectorProvider 

type VectorProvider interface {
	// Upsert stores or updates a vector with associated metadata.
	// If the ID exists, the vector and metadata are replaced.
	Upsert(ctx context.Context, id uuid.UUID, vector []float32, metadata []byte) error

	// UpsertBatch stores or updates multiple vectors.
	UpsertBatch(ctx context.Context, vectors []VectorRecord) error

	// Get retrieves a vector by ID.
	// Returns ErrNotFound if the ID does not exist.
	Get(ctx context.Context, id uuid.UUID) ([]float32, *VectorInfo, error)

	// Delete removes a vector by ID.
	// Returns ErrNotFound if the ID does not exist.
	Delete(ctx context.Context, id uuid.UUID) error

	// DeleteBatch removes multiple vectors by ID.
	// Non-existent IDs are silently ignored.
	DeleteBatch(ctx context.Context, ids []uuid.UUID) error

	// Search performs similarity search and returns the k nearest neighbors.
	// filter is optional metadata filtering (nil means no filter).
	Search(ctx context.Context, vector []float32, k int, filter map[string]any) ([]VectorResult, error)

	// Query performs similarity search with vecna filter support.
	// Returns ErrInvalidQuery if the filter contains validation errors.
	// Returns ErrOperatorNotSupported if the provider doesn't support an operator.
	Query(ctx context.Context, vector []float32, k int, filter *vecna.Filter) ([]VectorResult, error)

	// Filter returns vectors matching the metadata filter without similarity search.
	// Result ordering is provider-dependent and not guaranteed by the interface.
	// Limit of 0 returns all matching vectors.
	// Returns ErrFilterNotSupported if the provider cannot perform metadata-only filtering.
	Filter(ctx context.Context, filter *vecna.Filter, limit int) ([]VectorResult, error)

	// List returns vector IDs.
	// Limit of 0 means no limit.
	List(ctx context.Context, limit int) ([]uuid.UUID, error)

	// Exists checks whether a vector ID exists.
	Exists(ctx context.Context, id uuid.UUID) (bool, error)
}

VectorProvider defines raw vector storage operations. Implementations (pinecone, weaviate, milvus, qdrant) satisfy this interface.

type VectorRecord 

type VectorRecord = shared.VectorRecord

VectorRecord is re-exported from internal/shared for the public API.

type VectorResult 

type VectorResult = shared.VectorResult

VectorResult is re-exported from internal/shared for the public API.

Source Files

View all Source files

Directories

Path Synopsis
internal
atomix
Package atomix provides atom-based storage wrappers for grub.
 Package atomix provides atom-based storage wrappers for grub.
mockdb
Package mockdb provides a mock SQL driver for testing query generation.
 Package mockdb provides a mock SQL driver for testing query generation.
shared
Package shared contains canonical type definitions shared across grub.
 Package shared contains canonical type definitions shared across grub.
minio module
opensearch module
postgres module
redis module

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.