rostam

README

Rostam

A high-performance vector database and sub-microsecond key-value store in a single Go engine — run it as a standalone server, replicate it across a Raft cluster, or embed it directly in your binary with no server at all.

At matched recall it serves ~2× the queries of Milvus and pgvector and ~4× Qdrant, with the fastest load in the set (1M × 768d in 282 s) and the highest recall measured. → How it compares

Three ways to run it, one engine:

Standalone server	REST, gRPC and a binary TCP protocol. Talk to it from Python or any language.
Replicated cluster	Per-shard Raft, online resharding, backups to S3, RBAC/JWT/mTLS.
Embedded library	Import it into a Go binary. No server, no cgo required.

Agent memory over MCP

rostam-server mcp runs Rostam as an MCP stdio server, giving Claude Code, Claude Desktop, Cursor, or any MCP client persistent agent memory and vector-DB tools. No daemon, no cloud account, and no embedding API key required — memory works out of the box on built-in BM25, and pointing it at an OpenAI-compatible endpoint upgrades recall to hybrid dense+BM25.

claude mcp add rostam -- rostam-server mcp

→ MCP server for the full tool reference, client config snippets, and embedder configuration.

LLM response caching

rostam-server llm-proxy is an OpenAI-compatible caching reverse proxy: change one line (base_url="http://localhost:8484/v1") and an eligible repeated chat-completions request — cacheable shape, matching scope, and a prompt that hits the cache — is answered from a local semantic cache instead of hitting the upstream API. Exact byte-identical matching works out of the box with no embedding key; pointing it at an OpenAI-compatible embedder upgrades it to matching near-duplicate prompts too.

rostam-server llm-proxy

→ LLM caching proxy for flags, cache-scoping rules, and what's never cached.

RAG in a box

rostam-server rag chunks and indexes your documents, then answers questions from them with cited sources — no separate vector-store setup or ingestion pipeline required:

rostam-server rag ingest ./docs
rostam-server rag ask "How does the LLM proxy decide what's cacheable?"

Retrieval works out of the box on BM25 full-text search; pointing it at an OpenAI-compatible embedder upgrades it to dense+BM25 hybrid fusion by default (-no-hybrid selects pure dense search). rag query does retrieval alone, with no LLM required.

→ RAG CLI for flags, embedded-vs-remote mode, and supported file types.

And two engines, usable together or entirely on their own:

• a vector search engine (vector/) — HNSW, IVF and Vamana indexes, SQ8/binary/PQ quantization, hybrid dense+sparse fusion, BM25 full text, and metadata filtering with an exact filter-first query planner that does not fall off a recall cliff. Depends on nothing else in the repo.
• a key-value store — an in-process session cache, a replicated store, or a TCP server, all behind one Store interface, with server-side custom ops and sandboxed WASM stored procedures.

Both are built for the latency-sensitive end of the spectrum: reads are zero-copy and lock-light, the hot paths are allocation-free, and the distance kernels are hand-vectorized.

Status: Beta. Actively developed and tested (race-clean, benchmarked). APIs may still change ahead of a 1.0 release.

📚 docs.rostamlabs.com — quickstart, concepts, guides, and full API references (HTTP, gRPC, Go, Python). The site is built from the docs/ tree in this repository (mkdocs.yml), so it can also be read here or served locally.

---

How it compares

Under VectorDBBench — a neutral, third-party harness — on Cohere-1M (768d, cosine). Six engines, 27 cases, one continuous session, with HNSW m=16 / ef_construction=200 / k=100 pinned identically on every engine and ef swept.

Read at matched recall, which is the only like-for-like comparison — engines land at different recall for the same ef, and a segmented engine runs that ef against every segment, so equal ef is not equal work:

Matched recall	Rostam	vs Milvus	vs pgvector	vs Weaviate	vs Qdrant	vs Redis
0.95	3,161 QPS	1.84×	1.83×	3.25×	—	6.54×
0.97	2,642 QPS	2.02×	2.18×	3.03×	4.16×	7.53×
0.98	2,088 QPS	1.90×	2.28×	2.67×	3.98×	7.90×
0.99	1,471 QPS	1.78×	—	—	3.55×	—

The lead is stable across the whole curve, so it does not depend on picking a favourable operating point. Rostam also loads fastest (282 s for 1M × 768d) while spending the least total CPU of any multi-core engine in the set, and reaches the highest recall measured (0.9978).

Two caveats that bound all of this, and they are not footnotes: every number is same-session, because this hardware drifts up to 42% between sessions on unchanged code; and the benchmark client shares the box, which penalises the fastest engine hardest — so the throughput figures are floors, not ceilings. Floors in one more way: the sweep runs over Rostam's JSON wire, and at the ef=300 point its binary query framing measures +16.8% QPS and −19% single-client p99 on the same box and corpus. That is one point on the curve, not a uniform uplift — the wire saves a fixed cost per query, so its share shrinks as the search itself gets more expensive.

Full per-ef curves, per-engine CPU accounting, the filter case, four paired A/B controls and the complete methodology (including where the data cuts against Rostam) are in rostam-bench/vectordbbench.

And the key-value side

Rostam is two engines, so the KV half gets measured the same way — seven engines over the wire, one machine, one session:

GET ops/s	Rostam	Memcached	Dragonfly	Aerospike	Valkey	Redis	KeyDB
64 conns	726.1k	682.9k	532.0k	510.6k	239.1k	228.3k	206.3k
256 conns	732.4k	681.7k	551.6k	516.9k	227.9k	219.6k	208.8k

p99 at 512 connections: Rostam 2.02 ms, best of the seven (Memcached 2.32, Aerospike 2.75, Redis 4.69). Memcached wins at 8 connections; Rostam pulls ahead as concurrency rises.

Replicated writes (RF=2) against Aerospike CE 8.1 at matched commit semantics — both acking only after the replica has the write:

RF=2, replica-ack	8 conns	32 conns	128 conns	p50 @128
Rostam PB	35.6k	80.9k	113.6k	0.99 ms
Aerospike CE	38.0k	57.0k	85.9k	1.35 ms
Redis 7 (WAIT 1)	28.7k	43.2k	42.9k	2.95 ms
Valkey 8 (WAIT 1)	28.0k	42.2k	45.9k	2.72 ms

Rostam's lead is a scaling property. Aerospike wins at 8 connections — and has the better tail there — while Rostam overtakes at 32 and leads +42% / +32% at 32/128, with the better median from 32 upward. The Redis-protocol engines flatten above 32 connections, which is a per-instance ceiling. Relaxing Rostam and Aerospike to ack-at-master keeps the shape (157.4k against 125.7k at 128).

Measured twice on the same box four days apart; Rostam reproduced within ~1–4% at the anchor points.

Over a real network on separate nodes it tightens to near-parity, with Rostam's tail still ahead (p99 ~8 ms vs 17–22 ms). Both runs, the full posture matrix, and why a single load generator invalidates this comparison entirely are in rostam-bench/netkv.

---

Two ways to use it. Run the server and talk to it over REST, gRPC or a binary TCP protocol from any language — or embed the Go library directly, with no server in the picture. Start below; the Go embedding path is further down.

Install

# Prebuilt binary — verifies the release checksum before installing
curl -fsSL https://raw.githubusercontent.com/rostamlabs/rostam/main/install.sh | sh

# ...to a system path instead (prompts for sudo; ROSTAM_NO_SUDO refuses)
curl -fsSL https://raw.githubusercontent.com/rostamlabs/rostam/main/install.sh \
  | ROSTAM_INSTALL_DIR=/usr/local/bin sh

# Container (amd64 + arm64)
docker run -p 8080:8080 -e ROSTAM_API_KEY=secret ghcr.io/rostamlabs/rostam:latest

# Go toolchain
go install github.com/rostamlabs/rostam/cmd/rostam-server@latest   # the server
go get github.com/rostamlabs/rostam                                # the library

# Python client
pip install rostam-client

Prefer not to pipe a script into a shell? Download the archive and checksums.txt from the latest release, then verify before extracting — that is all the script does, plus picking the right file for your platform:

sha256sum -c checksums.txt --ignore-missing        # Linux
shasum -a 256 -c checksums.txt --ignore-missing    # macOS

The release binaries are pure-Go builds: everything works except WASM stored procedures, which return wasm: stored procedures require a cgo build. The container image always includes them. go install includes them only when it builds with cgo — a native build (cgo defaults off when cross-compiling) on a machine with a C compiler.

Local in-process embeddings are pure Go (rembed) — no cgo, no ONNX Runtime, nothing to install — so they are built into every binary and image. Just point at a model:

docker run -p 8080:8080 -e ROSTAM_API_KEY=secret \
  -e ROSTAM_EMBED_LOCAL=minilm-l6-v2 -e REMBED_CACHE=/models -v rostam-models:/models \
  ghcr.io/rostamlabs/rostam:latest

Weights download from the Hugging Face Hub on first use into REMBED_CACHE (default the OS user cache dir); point it at a mounted volume — as above — to persist them across restarts. ROSTAM_EMBED_LOCAL also accepts any Hugging Face org/model id beyond the built-in catalog.

See Local embeddings for the model catalog and configuration.

Quick start — run the server

go run ./cmd/rostam-server -http 127.0.0.1:8080 -data ./data

With no authenticator configured the server refuses to bind a reachable address rather than serve an open datastore to the network — so a bare :8080 will not start. To listen beyond loopback, give it auth (-api-key or -keys-file), or pass -insecure to run open deliberately.

curl -s localhost:8080/v1/collections \
  -d '{"name":"docs","config":{"dim":4,"metric":"cosine"}}'
curl -s localhost:8080/v1/collections/docs/points \
  -d '{"id":1,"vector":[0.1,0.2,0.3,0.4],"content":"hello rostam","upsert":true}'
curl -s localhost:8080/v1/collections/docs/points/search \
  -d '{"query":[0.1,0.2,0.3,0.4],"k":3}'

Or in a container

The image binds 0.0.0.0, so it requires authentication — pass the token by environment variable, which keeps it out of the process table and out of docker inspect:

docker build -f cmd/rostam-server/Dockerfile -t rostam-server .
docker run -p 8080:8080 -e ROSTAM_API_KEY=secret rostam-server

curl -s localhost:8080/v1/collections -H 'Authorization: Bearer secret' \
  -d '{"name":"docs","config":{"dim":4,"metric":"cosine"}}'

GET /v1/health and /v1/ready stay auth-exempt, so orchestrator probes work without the token. For a deliberately open dev container, append -insecure.

Clustering (per-shard Raft, online resharding), auth (RBAC/JWT/mTLS), TLS, backups to S3, and Prometheus metrics are flag-driven. → Running the server · Clustering · Security · HTTP API reference

Quick start — Python

pip install rostam-client   # stdlib-only; optional [langchain] [llamaindex] [haystack]

One class, Rostam(target), picks the transport from the target string: an http(s):// URL speaks REST, a tcp://host:port (or bare host:port) speaks the native binary protocol. Vector methods are identical either way:

from rostam import Rostam, filters as f

c = Rostam("http://localhost:8080")   # REST, port 8080
c.create_collection("docs", dim=768, full_text=True)
vec = [0.0] * 768  # replace with a real embedding
c.upsert("docs", 1, vec, content="hello", metadata={"tenant": "acme"})

# Dense kNN, hybrid dense+sparse, or BM25 full text — one call each.
hits = c.hybrid_text("docs", dense=vec, text="hello", k=5)

# Metadata filtering goes through the exact filter-first planner, so a
# selective filter does not degrade recall.
hits = c.search("docs", vec, k=10, filter=f.eq("tenant", "acme"))

Point the same class at the -tcp port instead for the native binary protocol — same flat vector API, plus r.kv.* for the key-value store, which has no REST surface:

r = Rostam("tcp://localhost:7000")    # or Rostam("localhost:7000") — bare host:port defaults to TCP

r.create_collection("docs", dim=768, full_text=True)
vec = [0.0] * 768  # replace with a real embedding
r.upsert("docs", 1, vec, content="hello", metadata={"tenant": "acme"})
hits = r.hybrid_text("docs", dense=vec, text="hello", k=5)

r.kv.put("user:42", b'{"coins":100}', ttl_ms=300_000)   # TCP-only
r.kv.incr("views:42", 1)                                  # atomic; missing key counts as 0

Working against one collection? r.collection(name) binds it so you stop repeating the name (mirrors the Go client's client.Collection):

docs = r.collection("docs")
docs.upsert(1, vec, content="hello")
hits = docs.hybrid_text(dense=vec, text="hello", k=5)

Bulk ingest (bulk_stage + bulk_build, HTTP-only) ships vectors as raw f32 over a binary wire rather than JSON text, which is what makes large loads fast — a 1M × 768d load runs in 282 s. → Python client docs

Other languages talk to the same server over REST or gRPC.

---

Quick start — vector search (Go library)

Embedding the engine in a Go process: no server, no cgo, no other dependencies:

import "github.com/rostamlabs/rostam/vector"

col, _ := vector.NewCollection("docs", vector.Config{
    Dim:    768,
    Metric: vector.Cosine,
    Quant:  vector.QuantSQ8, // 4× smaller, ~98% recall retained
})
defer col.Close()

// Insert is create-only (ErrDuplicateID on a live id); Upsert replaces.
_ = col.Insert(1, embedding, 0, vector.Metadata{
    "tenant": vector.NewString("acme"),
}, nil)

// Exact, fast filtered search — the payload index narrows to tenant=acme.
hits, _ := col.SearchFiltered(query, 10, vector.Filter{
    Op: vector.FilterEq, Field: "tenant", Value: vector.NewString("acme"),
})

// Diversified retrieval for RAG:
diverse, _ := col.SearchMMR(query, 10, vector.MMROpts{Lambda: 0.5})

Beyond kNN: hybrid dense+sparse fusion (RRF/weighted/DBSF), BM25 full-text, recommendation (± examples), discovery, group-by, scroll with order-by, IVF and Vamana indexes, binary/PQ quantization with mmap-resident codes, per-collection quotas and TTL. → Vector engine docs

Quick start — key-value store (Go library)

import (
    "github.com/rostamlabs/rostam"
    "github.com/rostamlabs/rostam/ops"
)

reg := ops.NewRegistry()
_ = ops.RegisterBuiltins(reg) // get / put / del / incr / expire / vector ops (Ops is required)
store, err := rostam.NewDirect(rostam.DirectConfig{Ops: reg /* DataDir: "..." */})
if err != nil {
    log.Fatal(err)
}
defer store.Close()

_ = store.Put(ctx, []byte("user:42"), []byte(`{"coins":100}`), 5*time.Minute)
v, _ := store.Get(ctx, []byte("user:42"))

// Atomic, server-side updates — one round trip, serialized per shard:
_, _ = store.Call(ctx, "incr", ops.EncodeIncrArgs([]byte("views:42"), 1))
_, _ = store.Call(ctx, "expire", ops.EncodeExpireArgs([]byte("user:42"), time.Hour))

Swap the constructor for a different backend — same Store interface:

Constructor	Backend	When
rostam.NewDirect	in-process, no Raft	single node, library, fastest
rostam.NewEmbedded	in-process + per-shard Raft	replicated / multi-node durability
rostam.NewClient	TCP client to a remote cluster	talk to a running server

You can also register your own Go functions as server-side ops (atomic read-modify-write under the shard lock, no CAS loops) or ship sandboxed, fuel-capped WASM procedures over the wire to a running cluster. → KV docs · Custom ops · WASM

---

Architecture

        vector/   ← standalone vector engine (usable on its own)
        HNSW · IVF · Vamana · GPU · quantization (SQ8/BQ1/PQ/PRQ, mmap) ·
        hybrid sparse+dense · BM25 · payload index + planner ·
        MMR/recommend/discover · tenants/auth/quotas

        rostam.Store  (Direct │ Embedded │ Client)      ← unified KV facade
              │
   ┌──────────┼───────────────┬──────────────────┐
 cache/     shard/+raft/     server/+client/     wasm/
 slab pool  per-shard        TCP protocol +      wasmtime UDF
 TTL, mmap  Raft + ops       smart routing       runtime
 warm-start registry

vector/'s only in-repo dependencies are the stdlib-only objstore/ package and its own analysis subpackage — vendor those three as a pure vector library. → Architecture

Micro-benchmarks

The head-to-head comparisons are at the top. These are in-process numbers for both engines, plus accuracy properties that hold independently of the machine.

Accuracy properties — these are properties of the algorithms, not of the machine, so they hold anywhere:

	Result
SQ8 quantization (4× smaller)	recall@10 ≈ 0.98 of exact
Binary quantization (32× smaller)	recall@10 ≈ 0.96 of exact (rescored)
Selective filtered search	exact filter-first path — no recall cliff
int8 (SQ8) distance kernel, AVX2 vs scalar	~2.9–3.3× (768d–1536d)
int8 distance kernel, VNNI vs AVX2	~1.3×
float32 distance kernels, AVX2 vs scalar	~12–14×

Latencies (make bench, in-process unless noted):

	Result
KV Get (hit), Direct	~29 ns/op
KV Put, Direct	~240 ns/op
KV Get/Put, Embedded (Raft, no-sync)	~222 ns / ~12.7 µs
Search, SIFT-1M, recall@10 0.968	173 µs p50 / 60.4k QPS
KV Get/Put over TCP loopback (Direct server)	~1.7 µs / ~1.8 µs

Measured on the same 12-core AMD EPYC Genoa the VectorDBBench comparison above runs on, so the two are directly comparable. The KV figures are parallel benchmarks at GOMAXPROCS=12 — per-op cost scales with core count, so a wider machine reports a lower number for identical code. The loopback row is the one exception: that box's NIC has a single queue and would measure the adapter rather than the storage path, so it comes from a multi-queue host. Re-run make bench on your own machine before sizing anything.

Reproducible head-to-head comparisons against Redis, Aerospike, Qdrant, Milvus, pgvector, and in-memory Go caches live in the separate rostam-bench repo, so this module stays dependency-light. → Performance

Examples

• examples/semantic-search — end-to-end RAG pipeline: OpenAI embeddings → upsert → dense vs hybrid search.
• examples/filtered-recall-cliff — why the filter-first planner exists, measured.

Development

Requires Go 1.26+. mmap persistence is Linux and 64-bit Windows (windows/amd64; a Windows shard file is fully allocated at creation, where a Linux one starts sparse); the AVX2 kernels are Linux/amd64. Everything else has a portable fallback. The full build needs cgo (wasmtime); the vector/cache/ops packages build with CGO_ENABLED=0.

Two optional build tags add cgo-only functionality, compiled out by default:

Tag	Enables	Requires
-tags cuda	GPU exact-KNN index	cgo + a CUDA toolchain

make test    # tests           make race   # race detector
make bench   # benchmarks      make all    # lint + tests + race + bench

→ Development · CONTRIBUTING.md · SECURITY.md

License

Apache License 2.0 (Apache-2.0), © 2026 RostamLabs.

Rostam is open source. Use it, embed it, modify it, run it in production, offer it as a service. Apache-2.0 also grants a patent licence explicitly — one that terminates if you sue someone claiming Rostam infringes a patent.

Redistributing it asks a little more than attribution: ship a copy of the licence and NOTICE, keep the notices already in the files, and mark the files you changed. LICENSE is the authoritative text — the paragraph above is a summary, not terms.

"Rostam" and the Rostam logo are trademarks of RostamLabs; the licence grants no trademark rights.

Documentation

Overview

Package rostam is the public library entry point. It exposes a Store interface backed by either an in-process cluster.Node (Embedded mode) or a networked client.Client (Client mode). Callers code against the interface and switch backends with one constructor.

Example (Embedded)

Example_embedded shows the minimum needed to bring up a single-node in-process Rostam and round-trip a value.

package main

import (
	"context"
	"fmt"
	"log"
	"os"
	"time"

	"github.com/rostamlabs/rostam"
	"github.com/rostamlabs/rostam/ops"
)

func main() {
	reg := ops.NewRegistry()
	if err := ops.RegisterBuiltins(reg); err != nil {
		log.Fatal(err)
	}

	dir, err := os.MkdirTemp("", "rostam-example-*")
	if err != nil {
		log.Fatal(err)
	}
	defer func() { _ = os.RemoveAll(dir) }()

	store, err := rostam.NewEmbedded(rostam.EmbeddedConfig{
		NodeID:    "demo",
		DataDir:   dir,
		NumShards: 1,
		Bootstrap: true,
		Ops:       reg,
	})
	if err != nil {
		log.Fatal(err)
	}
	defer func() { _ = store.Close() }()

	// Wait for leader; production code should bound this and handle the
	// timeout case explicitly.
	deadline := time.Now().Add(5 * time.Second)
	for !store.IsLeader([]byte("k")) && time.Now().Before(deadline) {
		time.Sleep(10 * time.Millisecond)
	}

	ctx := context.Background()
	_ = store.Put(ctx, []byte("k"), []byte("v"), 0)
	got, _ := store.Get(ctx, []byte("k"))
	fmt.Printf("got=%q\n", got)
}

Output:
got="v"

Example (RegisterOp)

Example_registerOp shows how to register a custom atomic read-modify-write op and call it through the Store interface. The same pattern works in both Embedded and Client mode.

package main

import (
	"errors"
	"fmt"

	"github.com/rostamlabs/rostam/cache"
	"github.com/rostamlabs/rostam/ops"
)

func main() {
	reg := ops.NewRegistry()
	_ = ops.RegisterBuiltins(reg)

	// A trivial RMW handler: count visits per key. The args blob IS the
	// key for routing purposes.
	_ = reg.RegisterRoutable("visit_inc", ops.OpReadWrite,
		func(tx *ops.TxContext, args []byte) ([]byte, error) {
			raw, err := tx.Get(args)
			var count uint64
			if err == nil && len(raw) == 8 {
				for i, b := range raw {
					count |= uint64(b) << (8 * i)
				}
			} else if err != nil && !errors.Is(err, cache.ErrNotFound) {
				return nil, err
			}
			count++
			buf := make([]byte, 8)
			for i := 0; i < 8; i++ {
				buf[i] = byte((count >> (8 * i)) & 0xff) //nolint:gosec // masked to one byte
			}
			return buf, tx.Put(args, buf, 0)
		},
		func(args []byte) ([]byte, bool) { return args, true },
	)

	fmt.Println("visit_inc registered")
}

Output:
visit_inc registered

Index

• Constants
• Variables
• func SetMetaReadIndexReadTimeout(d time.Duration) (restore func())
• func SetReshardCutoverGateTimeout(d time.Duration) (restore func())
• func SetReshardDrainGrace(d time.Duration) (restore func())
• func WCWire(opName string, innerArgs []byte, opts WriteOpts) (string, []byte)
• type AliasAction
• type BatchGetPoint
• type CacheConfig
• type ClientConfig
• type DirectConfig
• type DirectServer
  • func NewDirectServer(addr string, cfg DirectConfig) (*DirectServer, error)
  • func (s *DirectServer) Addr() string
  • func (s *DirectServer) Close() error
• type Embedded
• type EmbeddedConfig
• type FanMeta
• type FanoutDispatcher
  • func NewFanoutDispatcher(e *Embedded, inner InnerDispatcher) *FanoutDispatcher
  • func (f *FanoutDispatcher) Partitioned(coll string) (int, uint32, bool)
• type FusionMethod
• type GRPCServer
  • func NewGRPCServer(addr string, cfg DirectConfig) (*GRPCServer, error)
  • func (s *GRPCServer) Addr() string
  • func (s *GRPCServer) Close() error
• type HTTPServer
  • func NewHTTPServer(addr string, cfg DirectConfig) (*HTTPServer, error)
  • func (s *HTTPServer) Addr() string
  • func (s *HTTPServer) Close() error
• type InnerDispatcher
• type MVBatchGetPoint
• type MVHybridOpts
• type MVScrollOpts
• type MultiResult
• type MultiSearchOpts
• type MultiVectorConfig
• type NamedBatchGetPoint
• type NamedHybridOpts
• type NamedScrollOpts
• type NamedSearchOpts
• type NamedVectorParams
• type PartitionCatalog
• type Peer
• type ReadOpts
• type RebalanceResult
  • func Reconfigure(ctx context.Context, serverAddrs []string, target []Peer, rf int) (RebalanceResult, error)
• type ReshardState
• type Server
  • func NewServer(cfg ServerConfig) (*Server, error)
  • func (s *Server) Close() error
  • func (s *Server) ClusterNode() *cluster.Node
  • func (s *Server) GRPCAddr() string
  • func (s *Server) HTTPAddr() string
  • func (s *Server) Store() io.Closer
  • func (s *Server) TCPAddr() string
  • func (s *Server) VectorStore() *vector.CollectionStore
• type ServerConfig
• type Store
  • func NewClient(cfg ClientConfig) (Store, error)
  • func NewDirect(cfg DirectConfig) (Store, error)
  • func NewEmbedded(cfg EmbeddedConfig) (Store, error)
• type VectorConfig
• type VectorDocument
• type VectorFilter
• type VectorGroup
• type VectorGroupOpts
• type VectorHybridOpts
• type VectorInsertOpts
• type VectorMetadata
• type VectorResult
• type VectorScrollOpts
• type VectorSearchOpts
• type VectorSparse
• type WASMRegistration
• type WriteOpts

Examples

• Package (Embedded)
• Package (RegisterOp)

Constants

const (
	FusionRRF      = vector.FusionRRF
	FusionWeighted = vector.FusionWeighted
	FusionDBSF     = vector.FusionDBSF
)

Fusion method constants, re-exported from the vector package.

Variables

var (
	// ErrAliasTargetMissing: a create action's target collection does not exist.
	ErrAliasTargetMissing = errors.New("rostam: alias target collection does not exist")
	// ErrAliasShadowsCollection: an alias name collides with an existing real
	// collection (or, on create-collection, the new name collides with an alias).
	ErrAliasShadowsCollection = errors.New("rostam: alias name shadows an existing collection")
	// ErrAliasTargetIsAlias: a create action's target is itself an alias (only one
	// level of indirection is allowed — targets must be real collections).
	ErrAliasTargetIsAlias = errors.New("rostam: alias target is itself an alias")
	// ErrAliasReservedChar: an alias name contains a reserved '#'/'@' character
	// (those are reserved for physical partition / generation names).
	ErrAliasReservedChar = errors.New("rostam: alias name must not contain reserved characters '#' or '@'")
)

Alias-management sentinel errors. These surface from the alias coordinator ops (AliasBatch/CreateAlias) and mirror the existing vector error sentinels so the transports can map them to 400 / InvalidArgument.

IMPORTANT: the "rostam: alias " prefix shared by all four messages is load-bearing. httpapi.statusForError and grpcapi.grpcError match this prefix via strings.Contains to route alias validation errors to 400 / InvalidArgument (the sentinels live in the root package and cannot be imported by the transports without an import cycle). Do NOT change the prefix without updating both transport matchers — a reworded message silently falls through to 500 / Internal.

var ErrKeyAdminUnavailable = errors.New("rostam: online key-admin unavailable (no keys registry configured; start with -keys-file)")

ErrKeyAdminUnavailable is returned by the keys coordinator virtual-ops when no *vector.KeyRegistry is wired into the server (open/dev mode, or the static -api-key authenticator, which has no mutable registry). It is fail-loud: the keys ops never silently no-op when there is nothing to mutate.

var ErrNotFound = errors.New("rostam: not found")

ErrNotFound is returned when a key is absent or expired.

var ErrNotLeader = errors.New("rostam: not leader")

ErrNotLeader is returned when a write reaches a non-leader and (Networked mode only) the retry budget is exhausted.

var ErrPartitionedUnsupported = errors.New("rostam: operation not supported on a partitioned collection (Partitions>1)")

ErrPartitionedUnsupported is the loud-fail sentinel for an operation that is not wired for cross-shard fan-out being invoked on a partitioned collection (Partitions>1): on such a collection the data lives in the physical partition collections and the logical name is empty, so routing the op by the logical name would silently return zero results (or mutate nothing). No op currently returns this — every vector op now fans out across partitions — so it is reserved for future partition-unsupported ops. Unpartitioned collections (Partitions<=1) are unaffected.

Functions

func SetMetaReadIndexReadTimeout

func SetMetaReadIndexReadTimeout(d time.Duration) (restore func())

SetMetaReadIndexReadTimeout overrides metaReadIndexReadTimeout with restore.

func SetReshardCutoverGateTimeout

func SetReshardCutoverGateTimeout(d time.Duration) (restore func())

SetReshardCutoverGateTimeout overrides reshardCutoverGateTimeout with restore.

func SetReshardDrainGrace

func SetReshardDrainGrace(d time.Duration) (restore func())

SetReshardDrainGrace overrides reshardDrainGrace, returning a restore func so the internal read of the unexported var is unchanged after the test.

func WCWire

func WCWire(opName string, innerArgs []byte, opts WriteOpts) (string, []byte)

WCWire forwards to the unexported wcWire write-consistency wire decision so the inttest package can unit-test the plain-vs-envelope branch without a transport.

Types

type AliasAction

type AliasAction struct {
	Alias     string
	Canonical string
	Delete    bool
}

AliasAction is one mutation in an atomic alias batch at the embedded layer. It mirrors cluster.AliasAction (the meta-Raft wire type) so the partitionCatalog interface stays decoupled from the cluster package — metaCatalog bridges the two, exactly as ReshardState bridges to cluster.ReshardEntry. Names are canonicalized by the catalog before they reach meta-Raft. Delete=true removes the alias; otherwise Alias is created/overwritten to point at Canonical.

type BatchGetPoint

type BatchGetPoint struct {
	ID      uint64
	Vec     []float32
	Meta    VectorMetadata
	TTL     time.Duration
	Sparse  *VectorSparse
	Version uint64 // per-point CAS version (>=1); 0 on a backend that does not carry it
}

BatchGetPoint is one present point returned by VectorGetBatch: its id plus the SAME projected fields a single VectorGet carries (vector, payload, remaining TTL, sparse lane). Vec/Meta/Sparse follow the with_vector / with_payload projection requested at fetch time. Unlike VectorDocument (a search-hit alias with Distance/Score/Content) this carries the raw point projection and its id — a batch caller must know which id each point belongs to. Absent ids are NOT represented here; they appear in VectorGetBatch's separate missing slice.

type CacheConfig

type CacheConfig struct {
	// NumShardsPerNode controls how many independent Raft groups this
	// node hosts. Defaults to 64 when zero (matches cluster.Config).
	NumShardsPerNode int

	// Durable, when true, runs an msync ticker so dirty mmap
	// pages flush every MsyncIntervalMs. Requires NumShardsPerNode > 0
	// and a non-empty DataDir on EmbeddedConfig.
	Durable bool

	// Mlock pins the mmap region into RAM. Requires ulimit -l to cover
	// the total size; failure logs and continues without mlock.
	Mlock bool

	// MsyncIntervalMs is the flush interval for the Durable ticker.
	// Defaults to 100 when zero. Ignored when Durable is false.
	MsyncIntervalMs int

	// TTLSweepIntervalMs controls how often each shard's background sweeper
	// actively reaps expired TTL keys to reclaim capacity, independent of whether
	// they are ever read again (lazy-on-read expiry always returns an expired key
	// as not-found regardless of this). Zero keeps the library default (1000ms); a
	// NEGATIVE value disables active reaping entirely, leaving only lazy-on-read
	// expiry (and, for persistent shards, cold compaction at the next open). The
	// interval is a memory-reclaim-latency vs CPU-churn tradeoff, not a correctness
	// knob: a slower sweep lets expired bytes linger longer, which on a write-heavy
	// replicated heap shard raises the chance of hitting the capacity cap between
	// sweeps.
	TTLSweepIntervalMs int

	// DisableColdCompaction turns OFF the live-only rewrite of each persistent
	// shard's pages file at open. Default false (compaction ON), which is what a
	// persistent shard needs: it is the only thing that reclaims the ghost page
	// bytes left behind by overwritten and expired keys. This is the operational
	// escape hatch if that rewrite ever misbehaves — see cache.Config's field.
	DisableColdCompaction bool

	// MaxMemoryBytes bounds TOTAL cache memory for this node across every
	// shard. Zero means derive it from the host (a fraction of system RAM);
	// see cachebudget.go. The per-shard cap and page size are derived from
	// this, so the bound no longer moves when NumShardsPerNode changes.
	//
	// This is an upper bound, not a reservation: pages are allocated lazily.
	// It matters because Put is append-only — the lock-free read path freezes
	// retired pages — so a write-heavy node climbs toward this cap even when
	// the live key set is small, and only starts recycling once it arrives.
	// Set it below what the host can spare, or the process dies before the
	// ring-buffer eviction it depends on ever runs.
	//
	// It bounds CACHE PAGES, not process RSS. Pages are the live heap and Go
	// lets the heap reach (1 + GOGC/100) x live before collecting, so plan for
	// RSS ~= MaxMemoryBytes * (1 + GOGC/100) + ~40 MB: ~2x at Go's default
	// GOGC=100, ~1.5x at GOGC=40 (both measured, flat over 56M writes). The
	// multiplier is GC headroom rather than engine overhead; set GOMEMLIMIT to
	// bound RSS independently of it.
	MaxMemoryBytes int64
}

CacheConfig mirrors the cache-layer knobs callers care about. Its fields map one-to-one onto cache.Config; the indirection keeps the public API stable when cache.Config grows internal fields.

type ClientConfig

type ClientConfig struct {
	// Servers is the initial bootstrap list of "host:port" entries.
	// Smart-client topology refresh discovers the rest. Required.
	Servers []string

	// Ops is the caller's op registry mirror. The client uses it for
	// KeyExtractor-based routing — without it, ops fall back to
	// round-robin. Required if any custom op needs per-key routing.
	Ops *ops.Registry

	// MaxConnsPerServer caps the per-server connection pool. Default 8.
	MaxConnsPerServer int

	// MaxNotLeaderHops caps retries on stale topology. Default 5.
	MaxNotLeaderHops int

	// TopologyRefreshInterval polls cluster topology. Default 5s.
	TopologyRefreshInterval time.Duration

	// AuthToken, when non-empty, is sent on every RPC via a protocol-v2 frame
	// prefix. The server's Authenticator hook validates it. Leave empty for
	// legacy (no-auth) deployments — that keeps the wire format on v1.
	AuthToken string

	// TLSConfig, when non-nil, dials every server over TLS instead of plaintext.
	// Build it via tlsutil.ClientTLS(caFile, certFile, keyFile, serverName): set
	// RootCAs to verify the server, and (for mTLS) a client cert/key — the cert CN
	// then becomes the principal when no AuthToken is set. nil ⇒ plaintext default.
	TLSConfig *tls.Config
}

ClientConfig configures a networked Rostam client.

type DirectConfig

type DirectConfig struct {
	// DataDir is the root directory for the cache's mmap file. Empty
	// means heap mode (no persistence).
	DataDir string

	// Ops is the caller's op registry. MUST include ops.RegisterBuiltins
	// plus any caller-registered ops. Required.
	Ops *ops.Registry

	// Cache configures the cache layer (mmap knobs).
	Cache CacheConfig

	// Authenticator, when non-nil, gates every request on all transports. It is
	// the unified RBAC authorizer (authz.Authenticator): it receives an
	// authz.AuthRequest{Token, Op, Args} (token from the protocol-v2 frame /
	// Bearer header / gRPC metadata — empty for v1 clients) and returns true to
	// allow. nil = no auth (legacy/open mode).
	//
	// Build it from a vector.KeyRegistry with granular per-collection scopes:
	//
	//	reg, _ := vector.OpenKeyRegistry(filepath.Join(dir, "auth", "keys.json"))
	//	auth := authz.NewRBACAuthenticator(reg, opsReg, internalToken)
	Authenticator server.Authenticator
}

DirectConfig configures a no-replication in-process Store. Use this when you need persistence (via mmap) but explicitly do NOT need Raft replication — single-node deployments. Writes bypass the Raft log entirely; durability comes from the mmap header (if Cache.Durable is set) and Cache.Close's final msync.

All registered ops (read-only AND read-write) execute directly against the cache inside the shard's write lock. There is no applied-index, no log, no quorum.

type DirectServer

type DirectServer struct {
	// contains filtered or unexported fields
}

DirectServer is a TCP server backed by a no-Raft Direct store. Use it when you want a Rostam cache reachable over the network without paying for replication — e.g. a per-host cache process that application code connects to via NewClient.

func NewDirectServer

func NewDirectServer(addr string, cfg DirectConfig) (*DirectServer, error)

NewDirectServer constructs a Direct-backed cache and binds a TCP server to addr. Use "127.0.0.1:0" to get an OS-assigned port; read it back via Addr().

Callers connect with NewClient pointing at the returned Addr(). Close stops the listener, drains in-flight requests, and closes the cache (flushing the mmap when Durable is set).

func (*DirectServer) Addr

func (s *DirectServer) Addr() string

Addr returns the bound TCP address (useful when addr was ":0").

func (*DirectServer) Close

func (s *DirectServer) Close() error

Close stops the TCP server and the underlying Direct store. Idempotent.

type Embedded

type Embedded = embedded

Type aliases (identical underlying type ⇒ s.(*rostam.Embedded) works with zero rename).

type EmbeddedConfig

type EmbeddedConfig struct {
	// NodeID is the unique identifier for this node in the Raft cluster.
	NodeID string

	// DataDir is the base directory for Raft logs, snapshots, and mmap files.
	DataDir string

	// NumShards is the number of independent Raft shards. Defaults to 64
	// when zero. See cluster.Config.NumShards for the tuning rationale (fewer
	// groups = higher write throughput on commodity nodes; raise for many
	// large-core nodes). Fixed at creation.
	NumShards int

	// ReplicationFactor is how many nodes host each shard. 0 (default) or
	// >= len(Peers) means full replication (every node hosts every shard); a
	// smaller value partitions shards across the cluster — each node stores only
	// its shards and forwards ops for others to an owner.
	ReplicationFactor int

	// Peers is the static cluster membership list. nil or empty means
	// single-node mode.
	Peers []Peer

	// Bootstrap controls whether the node bootstraps itself as a fresh
	// single-node cluster. Set true on first start only.
	Bootstrap bool

	// Ops is the registry of named operations. Must be non-nil.
	Ops *ops.Registry

	// Cache configures cache-layer behaviour (durability, mlock, msync).
	Cache CacheConfig

	// WriteTimeout is the EFFECTIVE client-facing TCP response WriteTimeout,
	// threaded down from ServerConfig so the replicated shard's online-compaction
	// alias-drain fence (cache AliasQuarantine = 2*WriteTimeout) tracks the real
	// deadline instead of a hardcoded mirror. 0 ⇒ the shard falls back to its
	// built-in default (matching server.Config's own WriteTimeout default). Set by
	// rostam.NewServer; a direct NewEmbedded embedder may set it to match whatever
	// WriteTimeout it gives its own transport.
	WriteTimeout time.Duration

	// EnableOnlineCompaction opts every replicated mmap reject-writes shard into
	// online relocating compaction with quarantine-then-reset recycle
	// (cache/compact_online.go → cache.Config.OnlineCompaction). Off by default; a
	// no-op on heap / single-node / ringbuf shards. It is ONLY memory-safe when every
	// read is released within AliasQuarantine — i.e. all reads flow through the
	// WriteTimeout-bounded server transport. An embedder that retains a raw
	// `Store.Get`/`Node.Call` alias past the fence MUST NOT enable it (those readers
	// must copy). Threaded down from rostam.ServerConfig.EnableOnlineCompaction; see
	// its doc for the full contract.
	EnableOnlineCompaction bool

	// RaftAddr is this node's multiplexed Raft transport endpoint.
	// Required when len(Peers) > 1.
	RaftAddr string

	// RaftTransport selects the inter-node Raft transport: "" or "mux" (default)
	// uses the per-group NetworkTransport over the shared TCP listener; "fabric"
	// uses the multiplexed batching transport (raft/fabric). Flag-gated so the
	// default path is unchanged.
	RaftTransport string

	// RaftHeartbeatMs overrides the Raft heartbeat interval in milliseconds.
	// Zero uses the shard package default.
	RaftHeartbeatMs int

	// RaftElectionMs overrides the Raft election timeout in milliseconds.
	// Zero uses the shard package default.
	RaftElectionMs int

	// NoSync disables fsync on Raft log writes. Improves throughput in
	// testing; do not use in production.
	NoSync bool

	// VolatileLog puts data shards' Raft logs fully in memory (no write()
	// syscall); durability comes only from replication. The meta group stays
	// durable. See raft.Config.VolatileLog for the fresh-rejoin safety contract.
	VolatileLog bool

	// RaftLogLevel controls how loud the embedded hashicorp/raft is:
	// "TRACE"|"DEBUG"|"INFO"|"WARN"|"ERROR"|"OFF". Empty means INFO. Set
	// "ERROR" in tests and benchmarks that build clusters in a loop — `go test`
	// merges the test binary's stdout and stderr, so raft's per-election output
	// otherwise interleaves with the results.
	RaftLogLevel string

	// PersistentVectors makes vector collections mmap-backed (off-heap) on every
	// node. Raft stays the durability authority (vectors are wiped at startup and
	// repopulated from the Raft snapshot/log); this only changes the in-memory
	// layout, trading heap/GC pressure for the OS page cache. Recommended for
	// large vector datasets.
	PersistentVectors bool

	// PBFrontierStampInterval bounds how often the PB applied frontier is persisted
	// into the cache header. 0 selects the shard default (1s).
	//
	// It is exposed here because the cost is a full-region msync PER SHARD per
	// tick, so it is the knob that governs write TAIL latency in PB mode — and it
	// was previously unreachable: shard.Config carried it, but nothing plumbed it,
	// so no embedded caller or operator could change it. Lowering it tightens a
	// restarted node's catch-up delta at the cost of that tail; raising it does the
	// reverse. Neither direction affects correctness (see shard.Config's note: a
	// staler watermark only under-reports, and log matching turns an under-report
	// into a true-prefix catch-up or a clean divergence reject).
	//
	// PBFrontierStampEvery is the OPT-IN write-count trigger, off by default
	// because an msync costs O(mapped region) rather than O(bytes changed) — see
	// shard's defaultPBFrontierStampEvery for the measured penalty.
	PBFrontierStampInterval time.Duration
	PBFrontierStampEvery    int

	// InternalToken is the inter-node service credential presented by every
	// forwarding/admin client this node opens to a peer. The destination node's
	// RBAC authorizer treats it as the superuser service principal so an inter-node
	// forward passes auth. REQUIRED when the cluster runs with RBAC enabled
	// (otherwise a write forwarded from a non-leader node arrives token-less and is
	// denied). Empty = no token (correct for nil-auth / open clusters). Inter-node
	// traffic is plaintext this round, so this token is the only inter-node auth.
	InternalToken string

	// InterNodeTLS is the TLS CLIENT config for the inter-node forwarding dial,
	// threaded straight into cluster.Config.InterNodeTLS. Set it (alongside client
	// TLS) so peerClient dials TLS-wrapped peer ports over TLS, verifying each peer's
	// server cert against the CA. nil ⇒ plaintext inter-node dial (default; zero cost
	// when client TLS is off). AUTH is still the internal token. See
	// cluster.Config.InterNodeTLS.
	InterNodeTLS *tls.Config

	// InterNodeServerTLS is the TLS SERVER config that wraps the inter-node
	// REPLICATION listeners (Raft mux/fabric + PB), threaded straight into
	// cluster.Config.InterNodeServerTLS. Set it (alongside InterNodeTLS) so the
	// replication ports are mTLS-authenticated, not just the forwarding dial. nil ⇒
	// plaintext replication listeners (default; byte-identical to today). See
	// cluster.Config.InterNodeServerTLS.
	InterNodeServerTLS *tls.Config

	// NodeCNAllowlist is the OPT-IN per-node mTLS identity allowlist, threaded
	// straight into cluster.Config.NodeCNAllowlist (the inter-node CLIENT peer-CN
	// verify). Empty/nil = OFF = byte-identical (no callback attached). See
	// cluster.Config.NodeCNAllowlist.
	NodeCNAllowlist map[string]bool

	// ReplicationMode selects the cluster-level data-plane replication engine,
	// threaded straight into cluster.Config.ReplicationMode: "" or "raft"
	// (default) uses per-shard Raft groups, byte-identical to today. "pb"
	// selects EXPERIMENTAL primary-backup/ISR replication (shard.ReplicationModePB)
	// for every shard — a static cluster only (no automatic failover yet; see
	// shard/pbisr/BENCHMARK.md for the measured comparison). Requires
	// MinISR >= 1 and every Peer's PBAddr set.
	ReplicationMode string

	// MinISR is the minimum in-sync-replica count required by "pb" mode (must be
	// >= 1 when ReplicationMode == "pb"). Unused in "raft"/"" mode. Threaded
	// straight into cluster.Config.MinISR.
	MinISR int

	// PBCommitPrimary selects commit-on-primary durability for "pb" mode
	// (threaded into cluster.Config.PBCommitPrimary): the primary acks on local
	// apply and replicates asynchronously. DURABILITY DOWNGRADE — an acked write
	// can be lost if the primary dies before a backup received it. Default false
	// waits for the full ISR. Unused in "raft"/"" mode.
	PBCommitPrimary bool

	// PBAutoFailover enables automatic primary-backup failover for "pb"
	// mode (threaded into cluster.Config.PBAutoFailover): each primary commits a
	// liveness beacon, the meta leader promotes an ISR survivor when one goes
	// silent, and the ISR shrink/grow drivers un-wedge and re-open shards. Without
	// it a PB shard whose primary dies stays DOWN until an operator intervenes.
	// Off (false) is byte-identical to the static pre-Plan-4 cluster: no beacon
	// reaches the meta-Raft log and no epoch is ever bumped automatically.
	//
	// NOTE the default differs by entry point: rostam-server defaults
	// -pb-auto-failover to TRUE (both pre-default-on gates pass — see
	// shard/pbisr/DESIGN.md), whereas this field's Go zero value
	// is false, so a direct library embedder must opt in explicitly. Unused in
	// "raft"/"" mode.
	PBAutoFailover bool

	// WASMBlobRetention enables WASM blob retirement (threaded straight into
	// cluster.Config.WASMBlobRetention): how long a module blob that nothing on
	// this node references — a superseded version, or a __wasm_blob_put__ orphan —
	// is kept before its file is deleted.
	//
	// ZERO (THE DEFAULT, AND THE DEFAULT ON rostam-server TOO) DISABLES IT: no
	// sweeper runs and nothing is ever removed. Read cluster.Config's field
	// documentation and cluster/wasm_blob_retire.go before setting it — the value
	// is an assertion about how far behind a replica may fall, not a tuning knob,
	// and getting it wrong parks a lagging replica until an operator supplies the
	// bytes by hand with __wasm_blob_put__.
	WASMBlobRetention time.Duration
}

EmbeddedConfig configures an in-process Rostam node.

type FanMeta

type FanMeta struct {
	Degraded bool  // true when some partitions were unreachable in Partial mode — results are incomplete
	Missing  []int // partition indices skipped (sorted); nil if none
}

FanMeta reports cross-shard fan-out completeness for a partitioned read. On a single-partition or non-clustered backend it is the zero value (not degraded).

type FanoutDispatcher

type FanoutDispatcher = fanoutDispatcher

Type aliases (identical underlying type ⇒ s.(*rostam.Embedded) works with zero rename).

func NewFanoutDispatcher

func NewFanoutDispatcher(e *Embedded, inner InnerDispatcher) *FanoutDispatcher

NewFanoutDispatcher forwards to the unexported newFanoutDispatcher constructor.

func (*FanoutDispatcher) Partitioned

func (f *FanoutDispatcher) Partitioned(coll string) (int, uint32, bool)

Partitioned exposes the unexported fanoutDispatcher.partitioned routing lookup (returns P, gen, ok for a collection) for the inttest fan-out tests.

type FusionMethod

type FusionMethod = vector.FusionMethod

FusionMethod selects dense/sparse fusion in a hybrid search. Alias of vector.FusionMethod.

type GRPCServer

type GRPCServer struct {
	// contains filtered or unexported fields
}

GRPCServer is a Direct-backed store exposed over gRPC (see the grpcapi package for the service). It is a single-transport convenience over NewServer; for a store reachable over several transports at once (HTTP + gRPC + TCP sharing one cache) use NewServer with multiple addresses.

func NewGRPCServer

func NewGRPCServer(addr string, cfg DirectConfig) (*GRPCServer, error)

NewGRPCServer constructs a Direct-backed cache and serves the gRPC API on addr. Use "127.0.0.1:0" for an OS-assigned port; read it back via Addr(). cfg.Authenticator, if set, gates every RPC by the "authorization" metadata bearer token + op name (Health is exempt). Close stops the server and store.

func (*GRPCServer) Addr

func (s *GRPCServer) Addr() string

Addr returns the bound gRPC address (useful when addr was ":0").

func (*GRPCServer) Close

func (s *GRPCServer) Close() error

Close gracefully stops the gRPC server and closes the underlying store. Idempotent.

type HTTPServer

type HTTPServer struct {
	// contains filtered or unexported fields
}

HTTPServer is a Direct-backed store exposed over a REST/JSON HTTP API (see the httpapi package for the route surface). It is a single-transport convenience over NewServer; for a store reachable over several transports at once (HTTP + gRPC + TCP sharing one cache) use NewServer with multiple addresses.

func NewHTTPServer

func NewHTTPServer(addr string, cfg DirectConfig) (*HTTPServer, error)

NewHTTPServer constructs a Direct-backed cache and serves the REST API on addr. Use "127.0.0.1:0" for an OS-assigned port; read it back via Addr(). cfg.Authenticator, if set, gates every request by bearer token + op name (health is exempt). Close stops the server and the underlying store.

func (*HTTPServer) Addr

func (s *HTTPServer) Addr() string

Addr returns the bound HTTP address (useful when addr was ":0").

func (*HTTPServer) Close

func (s *HTTPServer) Close() error

Close shuts down the HTTP server and the underlying Direct store. Idempotent.

type InnerDispatcher

type InnerDispatcher = innerDispatcher

Type aliases (identical underlying type ⇒ s.(*rostam.Embedded) works with zero rename).

type MVBatchGetPoint

type MVBatchGetPoint struct {
	ID      uint64
	Tokens  [][]float32
	Meta    VectorMetadata
	Version uint64 // per-document CAS version (>=1); 0 on a backend that does not carry it
}

MVBatchGetPoint is one present point returned by VectorMVGetBatch: its id plus the SAME projected fields a single VectorMVGet carries (the token matrix + payload). Tokens/Meta follow the with_vector / with_payload projection requested at fetch time. Absent ids are NOT represented here; they appear in VectorMVGetBatch's separate missing slice. The MV clone of NamedBatchGetPoint (a token matrix, NO ttl and no sparse lane).

type MVHybridOpts

type MVHybridOpts struct {
	Filter  VectorFilter // shared-payload predicate applied to BOTH lanes; zero = no filter
	Method  FusionMethod // FusionRRF (default), FusionWeighted, or FusionDBSF
	Alpha   float64      // weighted only: MaxSim-lane weight in [0,1] (0 → 0.5 default)
	RRFK    int          // RRF constant; 0 = default 60
	DenseK  int          // MaxSim-lane candidate pool; 0 = max(k, 50)
	SparseK int          // sparse-lane candidate pool; 0 = max(k, 50)

	// ReadConsistency / OnPartitionUnavailable mirror MultiSearchOpts: 0 = AnyReplica
	// / Partial (default). Linearizable arms the meta + per-shard barriers; Fail errors
	// if any partition is unreachable during fan-out.
	ReadConsistency        uint8
	OnPartitionUnavailable uint8

	// MaxStaleness bounds replica lag (raft entries) behind the leader's
	// committed frontier; in effect ONLY when ReadConsistency==3 (BoundedStaleness).
	MaxStaleness uint64
}

MVHybridOpts carries the settings for an MV cross-modality hybrid search (VectorMVHybridSearch): the per-doc MaxSim (late-interaction dense) lane fused with the per-doc sparse lane. The MV-family analogue of NamedHybridOpts; the MV token query matrix and the sparse query ride as top-level args. DenseK sizes the MaxSim lane, SparseK the sparse lane.

type MVScrollOpts

type MVScrollOpts struct {
	// ReadConsistency controls which replicas may serve the scroll.
	// 0 = AnyReplica (default); 1 = LeaderOnly; 2 = Linearizable.
	ReadConsistency uint8

	// OnPartitionUnavailable controls cross-shard fan-out behaviour when a
	// partition is unreachable. 0 = Partial (default); 1 = Fail.
	OnPartitionUnavailable uint8

	// MaxStaleness bounds replica lag (raft entries) behind the leader's
	// committed frontier; in effect ONLY when ReadConsistency==3 (BoundedStaleness).
	MaxStaleness uint64

	// OrderBy, when non-nil, paginates the MV scroll by an arbitrary NUMERIC or
	// DATETIME payload field (Qdrant-style order_by) instead of the default
	// id-ascending order: the result set is globally ordered by the field's
	// (value, id) total order (ASC or DESC), missing/non-numeric-field points are
	// EXCLUDED, and the cursor is then a v2 (value, id) resume token. nil = the
	// id-ascending scroll (zero-overhead, v1 cursor). Mirrors VectorScrollOpts.OrderBy.
	OrderBy *vector.OrderBy
}

MVScrollOpts carries optional per-scroll settings for a multi-vector scroll (VectorMVScrollExt). The cursor stays its own VectorMVScrollExt parameter (mirrors the scroll signature); only the consistency knobs live here. Mirrors NamedScrollOpts / VectorScrollOpts.

type MultiResult

type MultiResult = vector.MultiResult

MultiResult is one scored document from a multi-vector search. Alias of vector.MultiResult.

type MultiSearchOpts

type MultiSearchOpts = vector.MultiSearchOpts

MultiSearchOpts tunes a multi-vector search. Alias of vector.MultiSearchOpts.

type MultiVectorConfig

type MultiVectorConfig = vector.MultiVectorConfig

MultiVectorConfig configures a late-interaction collection. Alias of vector.MultiVectorConfig.

type NamedBatchGetPoint

type NamedBatchGetPoint struct {
	ID      uint64
	Vectors map[string][]float32
	Meta    VectorMetadata
	TTL     time.Duration
	Version uint64 // per-point CAS version (>=1); 0 on a backend that does not carry it
}

NamedBatchGetPoint is one present point returned by VectorNamedGetBatch: its id plus the SAME projected fields a single VectorNamedGet carries (the per-space vectors map, shared payload, remaining TTL). Vectors/Meta follow the with_vector / with_payload projection requested at fetch time. Absent ids are NOT represented here; they appear in VectorNamedGetBatch's separate missing slice. The named clone of BatchGetPoint (a vectors MAP + ttl, no sparse lane).

type NamedHybridOpts

type NamedHybridOpts struct {
	Filter  VectorFilter // shared-payload predicate applied to BOTH lanes; zero = no filter
	Method  FusionMethod // FusionRRF (default), FusionWeighted, or FusionDBSF
	Alpha   float64      // weighted only: dense weight in [0,1] (0 → 0.5 default)
	RRFK    int          // RRF constant; 0 = default 60
	DenseK  int          // dense-lane candidate pool; 0 = max(k, 50)
	SparseK int          // sparse-lane candidate pool; 0 = max(k, 50)

	// ReadConsistency / OnPartitionUnavailable mirror NamedSearchOpts: 0 = AnyReplica
	// / Partial (default). Linearizable arms the meta + per-shard barriers; Fail errors
	// if any partition is unreachable during fan-out.
	ReadConsistency        uint8
	OnPartitionUnavailable uint8

	// MaxStaleness bounds replica lag (raft entries) behind the leader's
	// committed frontier; in effect ONLY when ReadConsistency==3 (BoundedStaleness).
	MaxStaleness uint64
}

NamedHybridOpts carries the settings for a named cross-space hybrid search (VectorNamedHybridSearch): one dense named space fused with one sparse named space. It is the named-family analogue of VectorHybridOpts (the dense single- vector hybrid), minus the sparse query (which is a top-level arg here, alongside the dense query and the two space names).

type NamedScrollOpts

type NamedScrollOpts struct {
	// ReadConsistency controls which replicas may serve the scroll.
	// 0 = AnyReplica (default); 1 = LeaderOnly; 2 = Linearizable.
	ReadConsistency uint8

	// OnPartitionUnavailable controls cross-shard fan-out behaviour when a
	// partition is unreachable. 0 = Partial (default); 1 = Fail.
	OnPartitionUnavailable uint8

	// MaxStaleness bounds replica lag (raft entries) behind the leader's
	// committed frontier; in effect ONLY when ReadConsistency==3 (BoundedStaleness).
	MaxStaleness uint64

	// OrderBy, when non-nil, paginates the named scroll by an arbitrary NUMERIC or
	// DATETIME payload field (Qdrant-style order_by) instead of the default
	// id-ascending order: the result set is globally ordered by the field's
	// (value, id) total order (ASC or DESC), missing/non-numeric-field points are
	// EXCLUDED, and the cursor is then a v2 (value, id) resume token. nil = the
	// id-ascending scroll (zero-overhead, v1 cursor). Mirrors VectorScrollOpts.OrderBy.
	OrderBy *vector.OrderBy
}

NamedScrollOpts carries optional per-scroll settings for a named-vector scroll (VectorNamedScrollExt). The cursor stays its own VectorNamedScrollExt parameter (mirrors the scroll signature); only the consistency knobs live here.

type NamedSearchOpts

type NamedSearchOpts struct {
	Filter VectorFilter // zero = no filter (predicate over the shared payload)

	// ReadConsistency controls which replicas may serve the query.
	// 0 = AnyReplica (default, fastest); 1 = LeaderOnly; 2 = Linearizable.
	// Linearizable arms the meta readIndex barrier on the coordinator and the
	// per-shard data barrier on every partition. Ignored for a single-node
	// engine / unpartitioned collection.
	ReadConsistency uint8

	// OnPartitionUnavailable controls what happens when a shard partition is
	// unreachable during a cross-shard fan-out.
	// 0 = Partial (default, return results from available shards);
	// 1 = Fail (return an error if any partition is unavailable).
	OnPartitionUnavailable uint8

	// MaxStaleness bounds replica lag (raft entries) behind the leader's
	// committed frontier; in effect ONLY when ReadConsistency==3 (BoundedStaleness).
	MaxStaleness uint64
}

NamedSearchOpts carries optional per-search settings for a named-vector KNN search (VectorNamedSearchExt / VectorNamedSearchDocsExt). Mirrors VectorSearchOpts / MultiSearchOpts.

type NamedVectorParams

type NamedVectorParams = vector.NamedVectorParams

NamedVectorParams is the per-named-space index configuration of a named-vector collection. Alias of vector.NamedVectorParams.

type PartitionCatalog

type PartitionCatalog = partitionCatalog

Type aliases (identical underlying type ⇒ s.(*rostam.Embedded) works with zero rename).

type Peer

type Peer struct {
	NodeID     string // unique identifier in the Raft cluster
	RaftAddr   string // host:port for inter-node Raft transport
	ServerAddr string // host:port for the TCP server (used by Client mode)

	// PBAddr is this node's primary-backup (pbisr) NetTransport listen
	// endpoint, e.g. "10.0.0.1:7200". Required only when
	// EmbeddedConfig.ReplicationMode is "pb"; left empty in "raft" mode
	// (unused there). Mirrors cluster.Peer.PBAddr.
	PBAddr string
}

Peer describes one node in the cluster membership list. Exported so callers do not have to import the internal cluster package.

type ReadOpts

type ReadOpts struct {
	// ReadConsistency controls which replicas may serve the read.
	// 0 = AnyReplica (default, fastest); 1 = LeaderOnly; 2 = Linearizable
	// (readIndex barrier — read-your-writes).
	ReadConsistency uint8

	// OnPartitionUnavailable controls cross-shard fan-out behaviour when a shard
	// partition is unreachable. 0 = Partial (default); 1 = Fail. A single-id get
	// routes to ONE partition, so this matters only on the get_config catalog read
	// fan-out; it is threaded for symmetry with the search opts.
	OnPartitionUnavailable uint8

	// MaxStaleness is the max raft-entry lag the serving replica may have behind
	// the leader's committed frontier, in effect ONLY when ReadConsistency==3
	// (BoundedStaleness). Zero is a valid bound (serve only a fully-caught-up
	// replica). Ignored for every other ReadConsistency level.
	MaxStaleness uint64
}

ReadOpts carries read-consistency settings for the point-get + get_config reads (VectorGetExt / VectorNamedGetExt / VectorMVGetExt and the *GetConfigExt variants). It mirrors the ReadConsistency / OnPartitionUnavailable knobs the search/scroll opts carry, so a Linearizable point-get arms the shard readIndex barrier (and a Linearizable get_config also arms the meta-catalog barrier), symmetric with search/scroll. The zero value is AnyReplica / Partial — the legacy behaviour, so the non-Ext methods delegate with a zero ReadOpts and stay byte/behaviour-identical.

type RebalanceResult

type RebalanceResult = cluster.RebalanceResult

RebalanceResult summarizes a triggered online rebalance: how many shard moves the plan contained and how they resolved. Alias of cluster.RebalanceResult.

func Reconfigure

func Reconfigure(ctx context.Context, serverAddrs []string, target []Peer, rf int) (RebalanceResult, error)

Reconfigure connects to a running cluster (any of serverAddrs) and triggers an online rebalance to the target member set + replication factor, returning when it completes. This is the client side of the operator surface — the same action `rostam-server -reconfigure` performs.

target is the desired membership: to grow, include the new node(s); to decommission, omit the departing node(s) (they keep running and forwarding until their shards have re-homed). rf is the target replication factor (0 or >= len(target) means full replication). The call blocks until the rebalance finishes; use a context deadline sized to the data volume being moved.

type ReshardState

type ReshardState struct {
	Status uint8
	OldP   int
	OldGen uint32
	NewP   int
	NewGen uint32
}

ReshardState is the embedded-level view of a collection's online-reshard status. Status 0 = Stable (no reshard); 1 = Resharding (a live repartition is dual-writing to the new gen). OldP/OldGen mirror the live PartitionsGen at reshard-begin (the read source of truth during the reshard); NewP/NewGen are the target gen being copied into. The zero value is Stable.

Exported because the dual-write routing and the reshard orchestrator consume it across embedded.go, and a future reshard-progress query API may surface it on the public Store interface.

type Server

type Server struct {
	// contains filtered or unexported fields
}

Server is a single store (Direct or replicated) fronted by one or more network transports (see ServerConfig). Close shuts every transport down and closes the underlying store once.

func NewServer

func NewServer(cfg ServerConfig) (*Server, error)

NewServer builds the store (Direct, or replicated when cfg.Cluster is set) and serves it over every transport whose address is set in cfg. Use "127.0.0.1:0" for an OS-assigned port and read the bound address back via HTTPAddr/GRPCAddr/TCPAddr. cfg.Authenticator, if set, gates every transport with the same func(token, op) bool.

func (*Server) Close

func (s *Server) Close() error

Close stops every enabled transport and closes the underlying store once. Idempotent.

func (*Server) ClusterNode

func (s *Server) ClusterNode() *cluster.Node

ClusterNode returns the cluster.Node backing this server, or nil when the backend is not a cluster (single-node Direct). It is the cluster analog of VectorStore(): the handle the out-of-band per-node backup/restore driver (cmd/rostam-server) uses to snapshot every owned shard (cache + vectors via the shard FSM snapshot) and the MetaRaft catalog — the state a cluster deployment partitions across shards, which VectorStore() cannot reach.

func (*Server) GRPCAddr

func (s *Server) GRPCAddr() string

GRPCAddr returns the bound gRPC address, or "" if gRPC is disabled.

func (*Server) HTTPAddr

func (s *Server) HTTPAddr() string

HTTPAddr returns the bound REST address, or "" if HTTP is disabled.

func (*Server) Store

func (s *Server) Store() io.Closer

Store exposes the unexported Server.store backing store for the inttest package, so a cluster integration test can type-assert it to *Embedded. Returns the io.Closer the Server holds (a *embedded for replicated/cluster servers).

func (*Server) TCPAddr

func (s *Server) TCPAddr() string

TCPAddr returns the bound binary-TCP address, or "" if TCP is disabled.

It must consult BOTH transports. The epoll server is stored in epollSrv instead of tcpSrv, and epoll is the default for single-node — so checking only tcpSrv reported "no TCP transport" for the configuration most servers run. The startup log iterates these accessors, which meant `-tcp` came up, accepted connections, and was never announced: an operator reading the log had no way to confirm the listener existed.

func (*Server) VectorStore

func (s *Server) VectorStore() *vector.CollectionStore

VectorStore returns the single-node vector CollectionStore backing this server, or nil when the backend has no directly-reachable store. It is the handle the out-of-band backup driver (cmd/rostam-server) uses to enumerate + snapshot collections without going through a transport.

Only the single-node Direct backend exposes its store this way (its *directStore.vectors). In cluster mode the vectors are partitioned across Raft shards and Raft is the durability authority, so there is no single store to back up here — that returns nil and cluster backup is a documented follow-up (drive it from the Raft FSM snapshot, not this accessor).

type ServerConfig

type ServerConfig struct {
	DirectConfig                 // single-node store config (used when Cluster == nil)
	Cluster      *EmbeddedConfig // when set, run a replicated (Raft) node instead of Direct
	HTTPAddr     string          // REST/JSON listen address; "" = disabled
	GRPCAddr     string          // gRPC listen address; "" = disabled
	TCPAddr      string          // binary TCP listen address; "" = disabled

	// EpollTCP selects the epoll/event-loop TCP transport (server.EpollServer)
	// instead of goroutine-per-connection. PLAINTEXT ONLY — ignored (falls back
	// to the goroutine server) when TLSConfig is set, so it is always safe to
	// enable. EpollLoops sets the event-loop count (0 = GOMAXPROCS default).
	//
	// rostam-server passes true by DEFAULT (-epoll=false opts out). The win is
	// conditional on CORE PRESSURE, not universal: when the server competes for
	// cores (a small or shared box, or the load generator co-located on it),
	// goroutine-per-connection scheduler churn costs throughput and epoll avoids
	// it — ~1.4x at 8 connections on an 8-core co-located box. When the server
	// has its own dedicated cores, the churn is absorbed and epoll is within
	// noise of the goroutine server at every concurrency (measured: 16 dedicated
	// cores, epoll and goroutine both ~720k at 64+ conns). It is never slower, so
	// it is a safe default that helps constrained deployments for free. Both
	// transports share the same 5-minute idle-connection timeout, so enabling it
	// does not change connection lifetime semantics.
	//
	// The zero value is false, so EMBEDDERS still opt in explicitly — flipping a
	// zero-valued bool would silently switch every existing caller's transport.
	EpollTCP   bool
	EpollLoops int

	// EnableOnlineCompaction opts every REPLICATED mmap reject-writes shard into
	// ONLINE relocating compaction with quarantine-then-reset recycle
	// (cache/compact_online.go): while the process runs, the TTL sweeper relocates
	// the live entries out of fragmented pages, RETIRES the emptied source extents,
	// and — once the alias-drain fence has elapsed — RESETS those extents back into
	// writable space, so a persistent replicated shard reclaims ghost page bytes
	// WHILE running instead of only at restart (cold compaction). Off by default.
	//
	// SAFETY CONTRACT — READ BEFORE ENABLING. Recycling overwrites retired mmap page
	// bytes after AliasQuarantine (= 2*WriteTimeout) has elapsed. It is ONLY
	// memory-safe when every read of the shard is released within that window — i.e.
	// all reads flow through the SERVER TRANSPORT, whose WriteTimeout bounds the
	// zero-copy response alias. A reject-writes/mmap Get returns a []byte that ALIASES
	// live mmap page bytes; the server response writer's write deadline is the only
	// thing that bounds how long that alias can escape. Do NOT enable this if ANY
	// in-process caller retains a raw cache alias past AliasQuarantine — e.g. an
	// embedder that holds a `Store.Get` / `Node.Call` result (both return the alias
	// verbatim) beyond the fence; such readers MUST copy the value out promptly. When
	// unset the whole feature is a no-op (nothing relocates or recycles), exactly as
	// before this flag existed. Threaded down to every replicated shard's cache
	// (AliasQuarantine derived from WriteTimeout, enforced fail-closed in shard.New).
	EnableOnlineCompaction bool

	// WriteTimeout bounds how long a single TCP response write+flush may block on a
	// stalled client before the connection is aborted (server.Config.WriteTimeout).
	// 0 selects the server's default (30s). It is a CORRECTNESS bound, not just
	// slow-loris hygiene: a reject-writes/mmap response payload is a zero-copy alias
	// into live mmap page bytes, and the online page-recycle fence
	// (cache.AliasQuarantine) must outlast the maximum alias hold. This one value is
	// the SINGLE SOURCE OF TRUTH: NewServer passes it to the TCP transport AND threads
	// it down to every replicated shard's cache (AliasQuarantine = 2*WriteTimeout,
	// enforced fail-closed in shard.New), so the deadline and the fence can never
	// drift apart. Applies to the goroutine-per-connection TCP server; the epoll
	// transport copies each payload synchronously in its event loop (no queued hold).
	WriteTimeout time.Duration

	// TLSConfig, when non-nil, enables TLS on ALL THREE client-facing transports
	// (HTTP, gRPC, TCP) using a single pre-built *tls.Config — typically built once
	// via tlsutil.ServerTLS(cert, key, ca, requireClientCert). nil ⇒ every
	// transport serves PLAINTEXT exactly as before (the default; byte-identical to
	// the pre-TLS path).
	//
	// When the config sets ClientAuth=RequireAndVerifyClientCert (mTLS), a client
	// presenting no cert or a cert not chaining to the config's ClientCAs is
	// rejected at the TLS handshake, before any app logic. The verified client-cert
	// CN is then threaded into authz.AuthRequest.ClientCN so a cert-only client
	// authorizes via its CN's registry scopes.
	//
	// SCOPE: this covers only the client-facing transports. The cluster's
	// inter-node peerClient and the Raft mux stay PLAINTEXT this round (a
	// documented follow-up); they are not derived from this field.
	TLSConfig *tls.Config

	// InterNodeTLS is the TLS CLIENT config for the cluster's inter-node forwarding
	// dial (peerClient). When TLSConfig wraps the client-facing transports of a
	// multi-node cluster, the TCP port that inter-node forwarding dials is itself
	// TLS-wrapped, so the inter-node dial must be TLS too or forwarded ops EOF at the
	// peer's handshake. Set this (typically tlsutil.ClientTLS(ca, cert, key, "") with
	// the node's own cert/key as the client cert; ServerName is set per-peer by
	// peerClient) so the inter-node dial verifies each peer's server cert against the
	// CA and presents a node client cert when the peer requires mTLS.
	//
	// nil ⇒ plaintext inter-node dial (the default; zero cost when client TLS is
	// off). AUTH is still the internal token; this only encrypts the transport.
	// Threaded into cfg.Cluster.InterNodeTLS in cluster mode (ignored when Cluster is
	// nil — single-node has no inter-node dial).
	InterNodeTLS *tls.Config

	// InterNodeServerTLS is the TLS SERVER config that wraps this cluster's
	// inter-node REPLICATION listeners (Raft mux/fabric + PB), threaded into
	// cfg.Cluster.InterNodeServerTLS in cluster mode (ignored when Cluster is nil).
	// It is the server-side counterpart of InterNodeTLS: without it the replication
	// ports stay plaintext even when TLSConfig wraps the client-facing ports, so a
	// hardened deployment should set BOTH. nil ⇒ plaintext replication listeners
	// (default; byte-identical to today). See cluster.Config.InterNodeServerTLS.
	InterNodeServerTLS *tls.Config

	// NodeCNAllowlist is the OPT-IN per-node mTLS identity allowlist, threaded into
	// cfg.Cluster.NodeCNAllowlist in cluster mode (ignored when Cluster is nil).
	// Empty/nil = OFF = byte-identical. See cluster.Config.NodeCNAllowlist.
	NodeCNAllowlist map[string]bool

	// KeyRegistry, when non-nil, is the SAME *vector.KeyRegistry the Authenticator
	// reads — wired into the dispatcher so the online key-admin coordinator
	// virtual-ops (__keys_add__/__keys_revoke__/__keys_list__) mutate/list it
	// directly. Sharing the one instance means concurrent auth-reads and admin
	// writes serialize on the registry's RWMutex, and an add/revoke takes effect on
	// the serving registry immediately (no restart) and persists via the registry's
	// atomic keys-file flush.
	//
	// nil ⇒ the keys ops are still reachable on every transport but fail loud with
	// ErrKeyAdminUnavailable (open/dev mode, or the static -api-key authenticator,
	// which has no mutable registry). The ops are admin-gated by the normal
	// authorize path (authz classifies the three op names as admin).
	//
	// v1 is per-node-local: in a multi-node cluster a keys mutation applies only to
	// the receiving node's registry (documented; cluster-wide meta-Raft propagation
	// is a v2 follow-up).
	KeyRegistry *vector.KeyRegistry

	// Admin, when non-nil, backs the OPT-IN object-storage admin REST endpoints
	// (backup-now / list-backups / evict / restore). It is built by the cmd layer
	// over the single-node CollectionStore + the shared objstore client, and is
	// admin-scope-gated by the normal authorize path. nil ⇒ those routes return 412
	// (object storage not configured) after the auth check. Ignored when HTTP is
	// disabled. Passed through verbatim to httpapi.Handler.
	Admin httpapi.AdminBackend

	// AccessLog, when non-nil (enabled), turns on the OPT-IN per-request access log
	// on EVERY transport: HTTP wraps the handler with a request-id + access-log
	// middleware, gRPC chains an access-log unary interceptor, and the TCP servers
	// emit one line per dispatched frame. Each line carries a request-id, the op,
	// status, latency, a REDACTED principal (token fingerprint or cert CN, never
	// the raw token), and bytes. nil ⇒ off (the default): no middleware/interceptor
	// is installed and the dispatch hot path is byte-identical to before, so the
	// feature costs nothing when unused. Built by the cmd layer from -access-log.
	AccessLog *rlog.AccessLog
}

ServerConfig configures a unified multi-transport server: one store exposed simultaneously over any combination of REST/JSON (HTTPAddr), gRPC (GRPCAddr), and the binary TCP protocol (TCPAddr). Leave an address empty to disable that transport; at least one must be set. All enabled transports dispatch into the same store — three front doors onto one store, not separate stores.

By default the store is single-node (Direct). Set Cluster to run a replicated Raft-backed node instead; writes replicate across the cluster and vectors are partitioned across shards by collection name.

type Store

type Store interface {
	// Get reads a value. Local read; never goes through Raft. Returns
	// ErrNotFound if absent or expired.
	Get(ctx context.Context, key []byte) ([]byte, error)

	// GetInto is the allocation-light variant of Get: the value is copied into
	// dst (reusing its capacity when large enough) and the resulting slice is
	// returned. With a reused dst the Networked path is a zero-allocation read
	// (pooled request args, no defensive response copy). Same ErrNotFound
	// semantics as Get. The returned slice may alias dst; do not retain the
	// argument after the call.
	//
	// Note: on the Networked backend GetInto goes through the request-response
	// path even when PipelineDepth > 0 (pipelining applies to Get, not the
	// zero-copy CallFunc path GetInto uses) — trade pipelining throughput for
	// the per-call allocation win accordingly.
	GetInto(ctx context.Context, key, dst []byte) ([]byte, error)

	// Put writes through Raft with the given TTL. Returns ErrNotLeader
	// if the backing node (Embedded) or the client's exhausted retry
	// budget (Networked) cannot reach the shard leader.
	Put(ctx context.Context, key, value []byte, ttl time.Duration) error

	// PutBatch writes many key/value pairs, batching per shard so each shard
	// takes one Raft log entry (fsync/round-trip/apply) per chunk instead of one
	// per key — the bulk-insert fast path. Same durability/consistency as Put;
	// ErrNotLeader semantics propagate as with Put.
	PutBatch(ctx context.Context, entries []ops.PutEntry) error

	// Del removes a key. Returns (true, nil) if the entry existed and
	// was deleted, (false, nil) if absent, (false, err) on failure.
	Del(ctx context.Context, key []byte) (bool, error)

	// Call invokes a registered op by name with caller-encoded args.
	// Read-only ops execute locally; read-write ops go through Raft.
	Call(ctx context.Context, op string, args []byte) ([]byte, error)

	// IsLeader reports whether the backing node owns leadership for the
	// shard hashing to key. Embedded: authoritative. Networked:
	// best-effort from the topology cache.
	IsLeader(key []byte) bool

	// LeaderAddr returns the current leader's address for key's shard,
	// or "" if unknown.
	LeaderAddr(key []byte) string

	// Close releases all resources. Idempotent.
	Close() error

	// RegisterWASM compiles and registers a WASM module as a named op.
	// On Embedded stores this round-trips through Raft, broadcasting the
	// registration into every shard group's log. On Direct stores the
	// module is registered in-process only (no replication). On networked
	// stores the call is forwarded to the leader via __register_wasm__.
	//
	// On the replicated paths a successful return means every shard group
	// COMMITTED the registration, not that the op is invocable everywhere yet:
	// each node opens its route gate per shard group as it applies the entry,
	// so an invocation may briefly return a transient, retryable error. See
	// client.Client.RegisterWASM for the full contract.
	//
	// UPDATING A LIVE MODULE'S BYTES IS SUPPORTED on the replicated paths, since
	// per-group version binding shipped. The version a group executes is bound to
	// that group's own log, so an update cannot leave two replicas of one group
	// executing the same committed entry with different bytes — which is what made
	// it unsafe before, when the executing version was node-wide while
	// registrations commit per shard group.
	//
	// ONLY THE BYTES MAY CHANGE. Kind is frozen at first registration and a change
	// to it is still refused with cluster.ErrWASMUpdateUnsupported — register the
	// new contract under a NEW op name. The freeze is not a simplification: Kind is
	// read on the propose side to decide whether an invocation replicates at all,
	// so a per-group Kind cannot be resolved before the group is known. The key
	// extractor is the other field that cannot be per group — it is what COMPUTES
	// the group index — and it is CONSTANT rather than frozen: every WASM op uses
	// the same one, so there is nothing to change (ops.WASMKeyExtractorHandle).
	// An identical re-registration remains accepted — that is the idempotent retry
	// the partial-broadcast recovery path depends on. Direct stores reject a
	// re-registration for an unrelated reason (a duplicate op name); see
	// directStore.RegisterWASM.
	//
	// pushReport IS PART OF THE RESULT, NOT DIAGNOSTICS. On the replicated paths,
	// before the registration enters any log the receiving node pushes the
	// module's BYTES to every member it can reach and requires a compile verdict
	// from each one that answers; a member that refuses fails this call. The
	// report is empty when every member acked (and always on Direct stores, which
	// have no peers). When it is not empty it NAMES the members that rendered no
	// verdict — unreachable, or on a build that does not know the push op — and
	// those are exactly the members that do not hold the bytes and will have to
	// fetch them on demand. Ignoring it is choosing not to know which nodes are one
	// step from being unable to run the op.
	RegisterWASM(ctx context.Context, r WASMRegistration, module []byte) (pushReport string, err error)

	// VectorInsert adds a vector with id under the named collection. CREATE-ONLY:
	// a live id is rejected with vector.ErrDuplicateID (the id must be deleted
	// first, or use VectorUpsert to replace / VectorInsertIfAbsent to no-op).
	// Note: unlike the dense family, VectorNamedInsert and VectorMVAdd REPLACE an
	// existing id rather than erroring.
	VectorInsert(ctx context.Context, collection string, id uint64, vec []float32, opts ...WriteOpts) error

	// VectorSearch returns the k nearest neighbors of query in the named collection.
	// Convenience form; use VectorSearchExt to observe FanMeta.
	VectorSearch(ctx context.Context, collection string, query []float32, k int) ([]VectorResult, error)

	// VectorSearchInto is the allocation-light variant of VectorSearch: results
	// are appended into dst (reused when its capacity allows), returning the
	// populated slice. Pass a reused dst from a hot query loop to avoid the
	// per-search result-slice allocation. Direct (in-process) stores hit the
	// engine's zero-alloc SearchInto directly; networked stores decode the wire
	// response straight into dst with no defensive copy. Plain kNN (no filter).
	// Convenience form; use VectorSearchExt to observe FanMeta.
	VectorSearchInto(ctx context.Context, collection string, query []float32, k int, dst []VectorResult) ([]VectorResult, error)

	// VectorDelete tombstones the vector with id in the named collection. Returns
	// true if the vector was present.
	VectorDelete(ctx context.Context, collection string, id uint64, opts ...WriteOpts) (bool, error)

	// VectorInsertIfAbsent inserts vec under id ONLY if id is not currently live,
	// reporting whether it inserted (false = no-op because id was already live).
	// ATOMIC: the liveness check and the insert run in a single op (one engine
	// write-lock critical section, serialized on the partition's Raft log) — there
	// is no check-then-act gap, so when it races a concurrent upsert on the same id
	// the live value always wins (it never clobbers). LIVENESS: an id is live iff
	// present and neither tombstoned (deleted) nor TTL-expired; a dead slot counts
	// as absent and is resurrected with the new value. The online-copy primitive
	// that closes Race A (value clobber); the copy MUST use this, never a plain upsert.
	VectorInsertIfAbsent(ctx context.Context, collection string, id uint64, vec []float32, opts VectorInsertOpts) (bool, error)

	// VectorExists reports whether id is currently live in the named collection,
	// using the same liveness definition as search admission: tombstoned (deleted)
	// and TTL-expired ids are NOT live, and a never-inserted id is absent. O(1)
	// idMap probe (no scan). The cheap liveness probe the online-copy resurrection
	// guard uses to re-check the source generation after an insert-if-absent (Race B).
	VectorExists(ctx context.Context, collection string, id uint64) (bool, error)

	// CreateCollection registers a new vector collection with the given configuration.
	CreateCollection(ctx context.Context, name string, cfg VectorConfig) error

	// VectorInsertExt adds a vector with optional TTL and metadata. CREATE-ONLY,
	// like VectorInsert: a live id is rejected with vector.ErrDuplicateID (delete
	// first, or use VectorUpsert / VectorInsertIfAbsent).
	VectorInsertExt(ctx context.Context, collection string, id uint64, vec []float32, opts VectorInsertOpts) error

	// VectorSearchExt returns the k nearest neighbors matching the optional filter.
	// The returned FanMeta reports cross-shard fan-out completeness (degraded when
	// a partition was unreachable in Partial mode); it is the zero value on a
	// single-partition or non-clustered backend.
	VectorSearchExt(ctx context.Context, collection string, query []float32, k int, opts VectorSearchOpts) ([]VectorResult, FanMeta, error)

	// VectorHybridSearch fuses a dense KNN lane and a sparse lane into the top-k.
	// An empty opts.Sparse degrades to pure dense; a nil dense query is pure sparse.
	// The returned FanMeta reports cross-shard fan-out completeness (see VectorSearchExt).
	VectorHybridSearch(ctx context.Context, collection string, dense []float32, k int, opts VectorHybridOpts) ([]VectorResult, FanMeta, error)

	// VectorQuery runs the unified Query API (vector_query): a root leaf plus N
	// single-level prefetch leaves combined by FUSION (RRF/Weighted/DBSF over the
	// prefetch lanes) or RERANK (the root re-scores the union of the prefetch
	// candidates). specBytes is the marshaled pb.QuerySpec carried on the wire;
	// spec is the decoded engine spec the coordinator uses for the fan-out
	// fusion/rerank merge. Returns the final top-k + FanMeta (cross-shard
	// completeness; see VectorSearchExt). The dense-family Qdrant-parity query op.
	VectorQuery(ctx context.Context, collection string, specBytes []byte, spec vector.QuerySpec, opts ReadOpts) ([]VectorResult, FanMeta, error)

	// VectorQueryGrouped runs the GROUPED Query API (vector_query with a non-empty
	// spec.GroupBy): the SAME root leaf + N prefetch leaves combined by FUSION/RERANK,
	// but the final ordered candidate pool is collapsed by the GroupBy metadata field
	// into the top-k GROUPS (best member first) with up to spec.GroupSize hits each —
	// the Query API generalization of VectorSearchGroups. Grouping is a deterministic
	// post-process over the EXACT global ordered pool (the same merge VectorQuery uses),
	// so P>1==P1 for both modes. Dense-only in v1 (named/MV grouped query fails loud).
	VectorQueryGrouped(ctx context.Context, collection string, specBytes []byte, spec vector.QuerySpec, opts ReadOpts) ([]VectorGroup, FanMeta, error)

	// VectorUpsert inserts or replaces a record (vector + document content +
	// metadata) by id — the RAG-store write path. opts carries optional
	// TTL/metadata/sparse. Replacing an id updates its vector and content.
	VectorUpsert(ctx context.Context, collection string, id uint64, vec []float32, content string, opts VectorInsertOpts) error

	// VectorSearchDocs runs a filtered KNN search and returns each hit with its
	// stored content and metadata in one call. The returned FanMeta reports
	// cross-shard fan-out completeness (see VectorSearchExt).
	VectorSearchDocs(ctx context.Context, collection string, query []float32, k int, opts VectorSearchOpts) ([]VectorDocument, FanMeta, error)

	// VectorSearchText runs a BM25 full-text search over the collection's indexed
	// $content and returns each hit enriched with content + metadata (like
	// VectorSearchDocs). The query is RAW text — the server tokenizes + BM25-scores
	// it (the caller ships no tokens). Requires a collection created with FullText
	// (else ErrFullTextDisabled). Under partitioning IDF is per-shard-local, so the
	// scores are APPROXIMATE (query_then_fetch); see VectorSearchExt for FanMeta.
	VectorSearchText(ctx context.Context, collection string, query string, k int, opts VectorSearchOpts) ([]VectorDocument, FanMeta, error)

	// VectorHybridText fuses a dense KNN lane with a BM25 full-text lane into the
	// top-k. The text lane is RAW query text analyzed server-side (no sparse query
	// rides the wire). opts mirrors VectorHybridOpts (fusion method/alpha/per-lane
	// pools/filter). Requires a FullText collection (else ErrFullTextDisabled). The
	// returned FanMeta reports cross-shard completeness (see VectorSearchExt). Under
	// partitioning the text lane's IDF is per-shard-local (approximate).
	VectorHybridText(ctx context.Context, collection string, dense []float32, query string, k int, opts VectorHybridOpts) ([]VectorResult, FanMeta, error)

	// VectorDeleteByFilter deletes all records matching filter (e.g. every chunk
	// of a document) and returns the count removed. A zero filter is rejected.
	VectorDeleteByFilter(ctx context.Context, collection string, filter VectorFilter) (int, error)

	// VectorSearchGroups runs a group-by-document search: it collapses KNN hits
	// sharing the opts.GroupBy metadata value into groups, returning the top-k
	// groups (best member first) with up to opts.GroupSize hits each. The RAG
	// "top-k distinct documents" retrieval primitive. The returned FanMeta reports
	// cross-shard fan-out completeness (see VectorSearchExt).
	VectorSearchGroups(ctx context.Context, collection string, query []float32, k int, opts VectorGroupOpts) ([]VectorGroup, FanMeta, error)

	// VectorMVCreateCollection registers a late-interaction (multi-vector /
	// ColBERT MaxSim) collection. Multi-vector collections are in-memory only.
	VectorMVCreateCollection(ctx context.Context, name string, cfg MultiVectorConfig) error

	// VectorMVDropCollection removes a multi-vector collection.
	VectorMVDropCollection(ctx context.Context, name string) error

	// VectorMVAdd inserts or replaces a document's token vectors (each length
	// cfg.Dim) in a multi-vector collection. meta is optional.
	VectorMVAdd(ctx context.Context, name string, docID uint64, tokens [][]float32, meta VectorMetadata, opts ...WriteOpts) error

	// VectorMVSearch runs a MaxSim late-interaction search, returning the top-k
	// documents ranked by descending score. The returned FanMeta reports
	// cross-shard fan-out completeness (see VectorSearchExt).
	VectorMVSearch(ctx context.Context, name string, query [][]float32, k int, opts MultiSearchOpts) ([]MultiResult, FanMeta, error)

	// VectorMVHybridSearch fuses an MV collection's MaxSim (late-interaction dense)
	// lane and its per-doc sparse lane into the top-k (cross-modality hybrid). query
	// is the MV token query matrix (empty ⇒ sparse-only); sparseQ is the doc-sparse
	// query (zero ⇒ MaxSim-only). opts carries the fusion method/params, the optional
	// shared-payload filter (applied to BOTH lanes), and read-consistency / partition
	// opts. A Linearizable read arms the meta + per-shard barriers. Returns the fused
	// top-k (id + fusion score). The MV analogue of VectorNamedHybridSearch.
	VectorMVHybridSearch(ctx context.Context, name string, query [][]float32, sparseQ VectorSparse, k int, opts MVHybridOpts) ([]VectorResult, error)

	// VectorMVHybridSearchExt is VectorMVHybridSearch with the cross-partition
	// fan-out completeness exposed as FanMeta (mirroring VectorHybridSearch). Under
	// the default OnPartitionUnavailable=Partial an unreachable partition yields
	// FanMeta{Degraded:true, Missing:...} instead of a silently incomplete top-k; a
	// single-node/unpartitioned backend never fans out and reports a zero FanMeta.
	VectorMVHybridSearchExt(ctx context.Context, name string, query [][]float32, sparseQ VectorSparse, k int, opts MVHybridOpts) ([]VectorResult, FanMeta, error)

	// VectorMVQuery runs the unified Query API (vector_mv_query) against a
	// MULTI-VECTOR collection: a root leaf plus N single-level prefetch leaves where
	// every leaf is an MV node (a MaxSim late-interaction lane and/or the doc-level
	// sparse field), combined by FUSION (RRF/Weighted/DBSF over the prefetch lanes) or
	// RERANK (the root re-scores the union of the prefetch candidates). Both MV lanes
	// are score-descending, so the coordinator folds via FuseScoreLanes (the
	// orientation-aware merge). specBytes is the marshaled pb.QuerySpec carried on the
	// wire; spec is the decoded engine spec the coordinator uses for the fan-out
	// fusion/rerank merge. Returns the final top-k + FanMeta (cross-shard
	// completeness; see VectorSearchExt). The MV-family analogue of VectorNamedQuery.
	VectorMVQuery(ctx context.Context, name string, specBytes []byte, spec vector.QuerySpec, opts ReadOpts) ([]VectorResult, FanMeta, error)

	// VectorMVDelete removes a document from a multi-vector collection, returning
	// whether it existed.
	VectorMVDelete(ctx context.Context, name string, docID uint64, opts ...WriteOpts) (bool, error)

	// VectorMVAddIfAbsent adds a document ONLY if docID is not already present,
	// reporting whether it inserted (false = no-op because docID was live). ATOMIC
	// (single op, Raft-serialized, no check-then-act gap) so a copy's add-if-absent
	// racing a concurrent replace-Add never clobbers the live document. LIVENESS:
	// the MV index has no tombstones/TTL, so docID is live iff it has a token-set
	// entry; a deleted docID counts as absent. The MV online-copy primitive that
	// closes Race A. Mirrors VectorInsertIfAbsent for the multi-vector path.
	VectorMVAddIfAbsent(ctx context.Context, name string, docID uint64, tokens [][]float32, meta VectorMetadata) (bool, error)

	// VectorMVExists reports whether docID is currently live in a multi-vector
	// collection (O(1) map probe). The MV resurrection-guard liveness check (Race B).
	VectorMVExists(ctx context.Context, name string, docID uint64) (bool, error)

	// VectorScroll lists live documents matching filter (zero filter = all),
	// enriched with content + metadata, up to limit (0 = no cap). A query-less
	// listing primitive (used by framework adapters for enumerate/count/filter).
	// The returned FanMeta reports cross-shard fan-out completeness (see VectorSearchExt).
	//
	// Cursor pagination (resume-after-id): opts.Cursor is the opaque token from the
	// previous page (empty = first page); the returned nextCursor is the token for
	// the next page (empty = exhausted, no more pages). Scroll is deterministic
	// id-ASCENDING globally; a no-cursor limit-capped scroll returns the smallest-id
	// `limit` documents. A malformed cursor surfaces ops.ErrBadScrollCursor.
	VectorScroll(ctx context.Context, collection string, filter VectorFilter, limit int, opts VectorScrollOpts) (docs []VectorDocument, meta FanMeta, nextCursor string, err error)

	// VectorResplit changes a partitioned collection's partition count by building a
	// NEW generation of physical partitions, streaming every vector into it re-hashed
	// by PartitionOf(id, newP), atomically flipping the catalog to {newP, gen+1}, then
	// dropping the old generation. OFFLINE: the caller MUST quiesce writes first — a
	// concurrent write during resplit may land in the old generation and be lost.
	VectorResplit(ctx context.Context, collection string, newP int) error

	// VectorResplitCleanup drops physical partitions left behind by a failed resplit:
	// every partition whose generation is not the collection's current live generation.
	// Safe to call any time; idempotent; returns the number of partitions dropped.
	// Best-effort discovery via bounded probe (the system has no collection enumeration),
	// so an orphan generation with more than the probe bound of partitions, or a wide
	// internal gap from a partial drop, may leave a tail (benign storage leak; re-runnable).
	VectorResplitCleanup(ctx context.Context, collection string) (int, error)

	// VectorMVResplit changes a partitioned multi-vector collection's partition count
	// by building a NEW generation of physical partitions, streaming every document
	// into it re-hashed by PartitionOf(id, newP), atomically flipping the catalog to
	// {newP, gen+1}, then dropping the old generation. OFFLINE: the caller MUST quiesce
	// writes first — a concurrent write during resplit may land in the old generation
	// and be lost. Mirrors VectorResplit for the dense path.
	VectorMVResplit(ctx context.Context, collection string, newP int) error

	// VectorMVResplitCleanup drops physical partitions left behind by a failed MV
	// resplit: every partition whose generation is not the collection's current live
	// generation. Safe to call any time; idempotent; returns the number of partitions
	// dropped. Best-effort discovery via bounded probe (the system has no collection
	// enumeration), so an orphan generation with more than the probe bound of
	// partitions, or a wide internal gap from a partial drop, may leave a tail (benign
	// storage leak; re-runnable). Mirrors VectorResplitCleanup for the dense path.
	VectorMVResplitCleanup(ctx context.Context, collection string) (int, error)

	// VectorReshard repartitions a dense collection LIVE (online) — both reads AND
	// writes stay up for the entire operation; there is no quiesce and no recreate.
	// It builds a NEW generation (gen+1) of physical partitions and runs a
	// dual-write + background-copy state machine:
	//
	//   - While resharding, every user point write (insert/upsert/delete) is
	//     dual-written to BOTH the old (read source-of-truth) gen and the new gen,
	//     so the new gen converges to live data without losing concurrent writes.
	//   - The copy uses atomic insert-IF-ABSENT (never plain upsert) so it can never
	//     clobber a newer concurrent write (value-clobber race), and a per-record
	//     resurrection guard (re-check old, delete-from-new if gone) so it can never
	//     resurrect a concurrently-deleted id (delete-resurrection race).
	//   - CUTOVER is a single atomic catalog flip to {newP, gen+1}; reads move to
	//     the new gen there. That flip is the single POINT OF NO RETURN.
	//
	// Resumable: if the coordinator dies mid-reshard the collection is left in the
	// Resharding state (reads still served from the old gen, dual-write still on,
	// status durable). Re-invoking VectorReshard with the SAME newP resumes and
	// converges (the copy is idempotent); a different in-flight target is refused.
	//
	// Abort window: VectorReshardAbort is valid only BEFORE cutover (see below).
	//
	// Cost: dual-write doubles write amplification for the reshard's duration only.
	// The background copy streams (bounded memory) and is throttleable so it cannot
	// saturate the cluster. newP must be in [2, 65536] and != the current P; the
	// collection must already be partitioned (P>1). The offline VectorResplit
	// (quiesced bulk path) remains available and is unaffected.
	VectorReshard(ctx context.Context, collection string, newP int) error

	// VectorReshardAbort cancels an in-progress online reshard and restores the
	// collection to its old generation. It is valid ONLY before cutover — while the
	// live generation is still the old one. It clears the Resharding status (turning
	// off dual-write) and drops the new-gen partitions; the collection is fully
	// intact on the old gen because reads never left it and dual-writes to it were
	// the source of truth. After cutover the reshard is committed and abort returns
	// an error (run a new reshard to revert). Errors if no reshard is in progress.
	VectorReshardAbort(ctx context.Context, collection string) error

	// VectorMVReshard repartitions a multi-vector collection LIVE (online) — the
	// multi-vector mirror of VectorReshard with identical semantics. Both reads AND
	// writes stay up for the whole operation (no quiesce, no recreate); it builds a
	// NEW generation (gen+1) and runs the same dual-write + background-copy state
	// machine:
	//
	//   - While resharding, every MV point write (add/delete) is dual-written to
	//     BOTH the old (read source-of-truth) gen and the new gen, so the new gen
	//     converges to live data without losing concurrent writes.
	//   - The copy uses atomic mv-add-IF-ABSENT (never plain add) so it can never
	//     clobber a newer concurrent write (value-clobber race), threading the FULL
	//     token matrix + metadata; a per-doc resurrection guard (re-check old,
	//     delete-from-new if gone) prevents resurrecting a concurrently-deleted doc.
	//   - CUTOVER is a single atomic catalog flip to {newP, gen+1}; that flip is the
	//     single POINT OF NO RETURN.
	//
	// Resumable: re-invoking with the SAME newP resumes a crashed reshard and
	// converges (the copy is idempotent); a different in-flight target is refused.
	// Abort window: VectorMVReshardAbort is valid only BEFORE cutover. Cost:
	// dual-write doubles write amplification for the reshard's duration only; the
	// copy streams (bounded memory) and is throttleable. newP must be in [2, 65536]
	// and != the current P; the collection must already be partitioned (P>1). The
	// offline VectorMVResplit (quiesced bulk path) remains available and unaffected.
	VectorMVReshard(ctx context.Context, collection string, newP int) error

	// VectorMVReshardAbort cancels an in-progress multi-vector online reshard and
	// restores the collection to its old generation — the MV mirror of
	// VectorReshardAbort. Valid ONLY before cutover (while the live generation is
	// still the old one): it clears the Resharding status (turning off dual-write)
	// and drops the new-gen partitions; the collection is fully intact on the old
	// gen because reads never left it and dual-writes to it were the source of truth.
	// After cutover the reshard is committed and abort returns an error (run a new
	// reshard to revert). Errors if no reshard is in progress.
	VectorMVReshardAbort(ctx context.Context, collection string) error

	// CreateAlias creates (or overwrites — upsert) an alias name that resolves to a
	// real (canonical) collection, so data-plane ops on the alias transparently
	// route to the target. Validation: the target collection must EXIST; the alias
	// name must not shadow an existing real collection nor contain reserved
	// '#'/'@' characters; the target must not itself be an alias (one level only).
	// Alias management is a coordinator op (meta-Raft metadata, NOT shard-routed).
	CreateAlias(ctx context.Context, alias, collection string) error

	// DeleteAlias removes an alias. An absent alias is a no-op.
	DeleteAlias(ctx context.Context, alias string) error

	// AliasBatch atomically applies a batch of alias mutations (create/delete) in
	// ONE meta-Raft log entry — an atomic swap {delete prod, create prod→v2}
	// repoints with no undefined window. The WHOLE batch is validated before
	// commit; any invalid create rejects the entire batch (nothing applied).
	AliasBatch(ctx context.Context, actions []AliasAction) error

	// ListAliases returns the alias→collection map (a local read, no consensus).
	// When collection != "" the result is filtered to aliases targeting it.
	ListAliases(ctx context.Context, collection string) (map[string]string, error)

	// VectorNamedCreateCollection registers a named-vector (Qdrant-style
	// multi-vector-space) collection: a MAP of named dense vector spaces, each its
	// own HNSW index, all sharing ONE per-point payload + point-id namespace. The
	// config maps each space name to its per-space index params. At least one space
	// is required; names must be non-empty and reserved-char-free. Named collections
	// are in-memory only (durable via Raft snapshot, not WAL). partitions is the
	// collection-level partition count (0 or 1 = single-partition; >1 splits the
	// collection across shards via cross-shard fan-out, like dense/MV — every
	// physical partition is a named collection with the same spaces config and a
	// point's id maps to exactly one partition).
	VectorNamedCreateCollection(ctx context.Context, name string, cfg map[string]NamedVectorParams, partitions int) error

	// VectorNamedDropCollection removes a named-vector collection (all sub-indexes
	// + the shared per-point store).
	VectorNamedDropCollection(ctx context.Context, name string) error

	// VectorNamedInsert upserts point id into a named-vector collection: a map of
	// named vectors (each name must be a configured space; each vec's length must
	// equal that space's Dim — fail loud on either), a SHARED per-point payload, and
	// a point-level ttl. A point may omit some configured spaces; re-inserting an id
	// replaces the vectors for the provided spaces + the payload + ttl.
	VectorNamedInsert(ctx context.Context, name string, id uint64, vectors map[string][]float32, payload VectorMetadata, ttl time.Duration, opts ...WriteOpts) error

	// VectorNamedSearch runs a filtered KNN search against the named space
	// (vectorName must be a configured space — fail loud). The optional filter is
	// predicate-evaluated against the SHARED per-point payload. Returns the top-k
	// point ids + distances. Back-compat convenience for VectorNamedSearchExt with
	// default (AnyReplica) consistency.
	VectorNamedSearch(ctx context.Context, name, vectorName string, query []float32, k int, filter VectorFilter) ([]VectorResult, error)

	// VectorNamedSearchExt is VectorNamedSearch with read-consistency / partition
	// opts (opts.Filter carries the optional payload predicate). A Linearizable
	// read arms the meta + per-shard barriers; AnyReplica/LeaderOnly stay
	// zero-overhead.
	VectorNamedSearchExt(ctx context.Context, name, vectorName string, query []float32, k int, opts NamedSearchOpts) ([]VectorResult, error)

	// VectorNamedSearchDocs is VectorNamedSearch returning each hit enriched with
	// the SHARED per-point payload (the named spaces store no per-arena content).
	// Back-compat convenience for VectorNamedSearchDocsExt.
	VectorNamedSearchDocs(ctx context.Context, name, vectorName string, query []float32, k int, filter VectorFilter) ([]VectorDocument, error)

	// VectorNamedSearchDocsExt is VectorNamedSearchDocs with read-consistency /
	// partition opts (opts.Filter carries the optional payload predicate).
	VectorNamedSearchDocsExt(ctx context.Context, name, vectorName string, query []float32, k int, opts NamedSearchOpts) ([]VectorDocument, error)

	// VectorNamedSparseSearch runs a sparse-dot-product top-k search against a
	// SPARSE named space (space must be a configured SPARSE space — fail loud;
	// ErrSpaceModalityMismatch for a dense space). The optional filter is
	// predicate-evaluated against the SHARED per-point payload. Returns the top-k
	// point ids + scores (descending by sparse dot product). Back-compat convenience
	// for VectorNamedSparseSearchExt with default (AnyReplica) consistency.
	VectorNamedSparseSearch(ctx context.Context, name, space string, query VectorSparse, k int, filter VectorFilter) ([]VectorResult, error)

	// VectorNamedSparseSearchExt is VectorNamedSparseSearch with read-consistency /
	// partition opts (opts.Filter carries the optional payload predicate). A
	// Linearizable read arms the meta + per-shard barriers.
	VectorNamedSparseSearchExt(ctx context.Context, name, space string, query VectorSparse, k int, opts NamedSearchOpts) ([]VectorResult, error)

	// VectorNamedHybridSearch fuses a DENSE named space and a SPARSE named space
	// into the top-k (cross-space hybrid). denseSpace must be a configured dense
	// space and sparseSpace a configured sparse space (fail loud:
	// ErrSpaceModalityMismatch / ErrUnknownVectorName). An empty dense query degrades
	// to the sparse lane only; an empty sparse query degrades to the dense lane only
	// (mirror the dense hybrid). opts carries the fusion method/params, the optional
	// shared-payload filter (applied to BOTH lanes), and read-consistency / partition
	// opts. A Linearizable read arms the meta + per-shard barriers. Returns the fused
	// top-k (id + dense distance + fusion score).
	VectorNamedHybridSearch(ctx context.Context, name, denseSpace string, denseQ []float32, sparseSpace string, sparseQ VectorSparse, k int, opts NamedHybridOpts) ([]VectorResult, error)

	// VectorNamedHybridSearchExt is VectorNamedHybridSearch with the cross-partition
	// fan-out completeness exposed as FanMeta (mirroring VectorHybridSearch). Under
	// the default OnPartitionUnavailable=Partial an unreachable partition yields
	// FanMeta{Degraded:true, Missing:...} instead of a silently incomplete top-k; a
	// single-node/unpartitioned backend never fans out and reports a zero FanMeta.
	VectorNamedHybridSearchExt(ctx context.Context, name, denseSpace string, denseQ []float32, sparseSpace string, sparseQ VectorSparse, k int, opts NamedHybridOpts) ([]VectorResult, FanMeta, error)

	// VectorNamedQuery runs the unified Query API (vector_named_query) against a
	// NAMED collection: a root leaf plus N single-level prefetch leaves where EVERY
	// leaf targets a configured named SPACE (dense or sparse), combined by FUSION
	// (RRF/Weighted/DBSF over the prefetch lanes — N>2 multi-space fusion is the
	// distinctive named-family value) or RERANK (the root re-scores the union of the
	// prefetch candidates). specBytes is the marshaled pb.QuerySpec carried on the
	// wire; spec is the decoded engine spec the coordinator uses for the fan-out
	// fusion/rerank merge. Returns the final top-k + FanMeta (cross-shard
	// completeness; see VectorSearchExt). The named-family analogue of VectorQuery.
	VectorNamedQuery(ctx context.Context, name string, specBytes []byte, spec vector.QuerySpec, opts ReadOpts) ([]VectorResult, FanMeta, error)

	// VectorNamedDelete removes point id from EVERY named space + the shared
	// payload/ttl, returning whether it existed.
	VectorNamedDelete(ctx context.Context, name string, id uint64, opts ...WriteOpts) (bool, error)

	// VectorNamedScroll lists live points (+ shared payload) matching filter (zero
	// filter = all), up to limit (0 = no cap). Payload-only (no vectors).
	//
	// Cursor pagination (resume-after-id): cursor is the opaque token from the
	// previous page (empty = first page); the returned nextCursor is the token for
	// the next page (empty = exhausted). Scroll is deterministic id-ASCENDING
	// globally. A malformed cursor surfaces ops.ErrBadScrollCursor. Back-compat
	// convenience for VectorNamedScrollExt with default consistency.
	VectorNamedScroll(ctx context.Context, name string, filter VectorFilter, limit int, cursor string) (docs []VectorDocument, nextCursor string, err error)

	// VectorNamedScrollExt is VectorNamedScroll with read-consistency / partition
	// opts. The cursor stays its own parameter.
	VectorNamedScrollExt(ctx context.Context, name string, filter VectorFilter, limit int, cursor string, opts NamedScrollOpts) (docs []VectorDocument, nextCursor string, err error)

	// VectorNamedGetConfig returns the configured named spaces of a named-vector
	// collection (the introspection accessor).
	VectorNamedGetConfig(ctx context.Context, name string) (map[string]NamedVectorParams, error)

	// VectorNamedGetConfigExt is VectorNamedGetConfig with read-consistency opts. A
	// Linearizable read arms the meta-catalog read barrier (resolveCollectionForRead)
	// so the returned config reflects a just-created / just-reconfigured collection,
	// and routes the catalog read to the owning shard leader. The zero ReadOpts is
	// AnyReplica — behaviour-identical to VectorNamedGetConfig.
	VectorNamedGetConfigExt(ctx context.Context, name string, opts ReadOpts) (map[string]NamedVectorParams, error)

	// VectorGet retrieves a dense point by id: its (cosine-normalized, if the
	// metric is cosine) vector, payload, remaining TTL, and sparse lane. withVector
	// / withPayload gate the vector and the payload+sparse projections (pass both
	// true for the common "fetch everything" case). found is false for an
	// absent/tombstoned/TTL-expired point — a not-found FLAG, NOT an error, so a
	// point-op routed to one partition treats "not here" as expected.
	VectorGet(ctx context.Context, collection string, id uint64, withVector, withPayload bool) (found bool, vec []float32, meta VectorMetadata, ttl time.Duration, sparse *VectorSparse, err error)

	// VectorGetExt is VectorGet with read-consistency opts. A Linearizable read
	// (opts.ReadConsistency == 2) routes the single-id point-get to the owning
	// partition's Raft leader and arms the shard readIndex barrier (read-your-writes).
	// The zero ReadOpts is AnyReplica — byte/behaviour-identical to VectorGet.
	VectorGetExt(ctx context.Context, collection string, id uint64, withVector, withPayload bool, opts ReadOpts) (found bool, vec []float32, meta VectorMetadata, ttl time.Duration, sparse *VectorSparse, err error)

	// VectorGetBatch retrieves MANY dense points by id in ONE op. It returns the
	// PRESENT points (each carrying its id + the with_vector / with_payload
	// projection) plus the missing ids (absent / tombstoned / TTL-expired). A
	// partial miss is NORMAL, never an error — mirroring single VectorGet's
	// not-found FLAG. On a partitioned collection the ids are grouped by their
	// owning partition and each partition is asked ONLY for its owned subset
	// (concurrently), then the results are merged. Duplicate ids are deduped (a
	// repeated id is fetched once and appears once). points and missing are both
	// sorted ascending by id (deterministic). An empty ids list yields empty
	// points + empty missing. Like VectorGet this is AnyReplica (no read
	// consistency). An unreachable partition fails the whole batch (fail-loud).
	VectorGetBatch(ctx context.Context, collection string, ids []uint64, withVector, withPayload bool) (points []BatchGetPoint, missing []uint64, err error)

	// VectorSetPayload merges patch into the point's existing payload (patch keys
	// overwrite/add, other keys retained), reindexing the dense payload index and
	// WAL-logging the result. Does NOT change the vector or TTL. applied is false
	// (NOT an error) when the point is absent/tombstoned/expired. keyTTLMs is an
	// optional per-key payload TTL map (key -> RELATIVE ms; the engine computes the
	// absolute deadline); nil/empty = no per-key TTL.
	VectorSetPayload(ctx context.Context, collection string, id uint64, patch VectorMetadata, keyTTLMs map[string]int64, opts ...WriteOpts) (applied bool, err error)

	// VectorOverwritePayload replaces the point's entire payload with meta (nil =
	// clear). applied is false (not an error) for an absent point. keyTTLMs sets the
	// per-key payload TTL for the new payload (relative ms; engine computes the
	// absolute deadline); nil/empty = no per-key TTL.
	VectorOverwritePayload(ctx context.Context, collection string, id uint64, meta VectorMetadata, keyTTLMs map[string]int64, opts ...WriteOpts) (applied bool, err error)

	// VectorDeletePayloadKeys removes the listed keys from the point's payload
	// (absent keys = no-op). applied is false (not an error) for an absent point.
	VectorDeletePayloadKeys(ctx context.Context, collection string, id uint64, keys []string, opts ...WriteOpts) (applied bool, err error)

	// VectorClearPayload removes the point's entire payload. applied is false (not
	// an error) for an absent point.
	VectorClearPayload(ctx context.Context, collection string, id uint64, opts ...WriteOpts) (applied bool, err error)

	// VectorNamedGet retrieves a named-vector point by id: its per-space vectors
	// (map[name][]float32; omitted spaces absent), shared payload, and remaining
	// TTL. found is false (not an error) for an absent/expired point. See VectorGet.
	VectorNamedGet(ctx context.Context, name string, id uint64, withVector, withPayload bool) (found bool, vectors map[string][]float32, payload VectorMetadata, ttl time.Duration, err error)

	// VectorNamedGetExt is VectorNamedGet with read-consistency opts. A
	// Linearizable read routes to the owning partition's leader and arms the shard
	// readIndex barrier. The zero ReadOpts is byte/behaviour-identical to
	// VectorNamedGet.
	VectorNamedGetExt(ctx context.Context, name string, id uint64, withVector, withPayload bool, opts ReadOpts) (found bool, vectors map[string][]float32, payload VectorMetadata, ttl time.Duration, err error)

	// VectorNamedGetBatch retrieves MANY named-vector points by id in ONE op. It
	// returns the PRESENT points (each carrying its id + the per-space vectors map +
	// shared payload + remaining TTL, gated by with_vector / with_payload) plus the
	// missing ids (absent / expired). A partial miss is NORMAL, never an error —
	// mirroring single VectorNamedGet's not-found FLAG. On a partitioned collection
	// the ids are grouped by their owning partition and each partition is asked ONLY
	// for its owned subset (concurrently), then merged. Duplicate ids are deduped.
	// points and missing are both sorted ascending by id (deterministic). An empty
	// ids list yields empty points + missing. Like VectorNamedGet this is AnyReplica
	// (no read consistency). An unreachable partition fails the batch (fail-loud).
	// The named clone of VectorGetBatch.
	VectorNamedGetBatch(ctx context.Context, collection string, ids []uint64, withVector, withPayload bool) (points []NamedBatchGetPoint, missing []uint64, err error)

	// VectorNamedSetPayload merges patch into id's SHARED payload (no reindex — the
	// named family has no payload index). keyTTLMs is an optional per-key payload
	// TTL map (key -> RELATIVE ms; the engine computes the absolute deadline, stored
	// in the named snapshot); nil/empty = no per-key TTL. applied false (not an
	// error) for absent.
	VectorNamedSetPayload(ctx context.Context, name string, id uint64, patch VectorMetadata, keyTTLMs map[string]int64, opts ...WriteOpts) (applied bool, err error)

	// VectorNamedOverwritePayload replaces id's entire shared payload. keyTTLMs sets
	// the per-key payload TTL for the new payload (relative ms; engine computes the
	// absolute deadline); nil/empty = no per-key TTL. applied false (not an error)
	// for absent.
	VectorNamedOverwritePayload(ctx context.Context, name string, id uint64, meta VectorMetadata, keyTTLMs map[string]int64, opts ...WriteOpts) (applied bool, err error)

	// VectorNamedDeletePayloadKeys removes the listed keys from id's shared payload.
	// applied false (not an error) for absent.
	VectorNamedDeletePayloadKeys(ctx context.Context, name string, id uint64, keys []string, opts ...WriteOpts) (applied bool, err error)

	// VectorNamedClearPayload removes id's entire shared payload. applied false (not
	// an error) for absent.
	VectorNamedClearPayload(ctx context.Context, name string, id uint64, opts ...WriteOpts) (applied bool, err error)

	// VectorMVGet retrieves a multi-vector document by id: its token matrix
	// ([][]float32) and payload. found is false (not an error) for an absent
	// document (the MV index has no tombstones/TTL). See VectorGet.
	VectorMVGet(ctx context.Context, name string, docID uint64, withVector, withPayload bool) (found bool, tokens [][]float32, payload VectorMetadata, err error)

	// VectorMVGetExt is VectorMVGet with read-consistency opts. A Linearizable read
	// routes to the owning partition's leader and arms the shard readIndex barrier.
	// The zero ReadOpts is byte/behaviour-identical to VectorMVGet.
	VectorMVGetExt(ctx context.Context, name string, docID uint64, withVector, withPayload bool, opts ReadOpts) (found bool, tokens [][]float32, payload VectorMetadata, err error)

	// VectorMVGetBatch retrieves MANY multi-vector documents by id in ONE op. It
	// returns the PRESENT points (each carrying its id + the token matrix +
	// payload, gated by with_vector / with_payload) plus the missing ids (absent).
	// A partial miss is NORMAL, never an error — mirroring single VectorMVGet's
	// not-found FLAG. On a partitioned collection the ids are grouped by their
	// owning partition and each partition is asked ONLY for its owned subset
	// (concurrently), then merged. Duplicate ids are deduped. points and missing
	// are both sorted ascending by id (deterministic). An empty ids list yields
	// empty points + missing. Like VectorMVGet this is AnyReplica (no read
	// consistency). An unreachable partition fails the batch (fail-loud). MV has NO
	// ttl. The MV clone of VectorNamedGetBatch.
	VectorMVGetBatch(ctx context.Context, collection string, ids []uint64, withVector, withPayload bool) (points []MVBatchGetPoint, missing []uint64, err error)

	// VectorMVScroll lists live multi-vector documents (id + payload, no token
	// vectors) matching filter (zero filter = all), up to limit (0 = no cap). A
	// query-less listing primitive for the MV family (the dense/named scroll
	// mirror; vector_mv_get is the path for token matrices). The returned FanMeta
	// reports cross-shard fan-out completeness (see VectorScroll).
	//
	// Cursor pagination (resume-after-id): cursor is the opaque token from the
	// previous page (empty = first page); the returned nextCursor is the token for
	// the next page (empty = exhausted). Scroll is deterministic id-ASCENDING
	// globally; a no-cursor limit-capped scroll returns the smallest-id `limit`
	// documents. A malformed cursor surfaces ops.ErrBadScrollCursor. Back-compat
	// convenience for VectorMVScrollExt with default consistency.
	VectorMVScroll(ctx context.Context, name string, filter VectorFilter, limit int, cursor string) (docs []VectorDocument, meta FanMeta, nextCursor string, err error)

	// VectorMVScrollExt is VectorMVScroll with read-consistency / partition opts. A
	// Linearizable scroll arms the meta readIndex barrier on the coordinator and
	// the per-shard data barrier on every partition; rc rides every per-partition
	// arg. The cursor stays its own parameter.
	VectorMVScrollExt(ctx context.Context, name string, filter VectorFilter, limit int, cursor string, opts MVScrollOpts) (docs []VectorDocument, meta FanMeta, nextCursor string, err error)

	// VectorMVSetPayload merges patch into docID's payload (no reindex). keyTTLMs is
	// an optional per-key payload TTL map (key -> RELATIVE ms; the engine computes
	// the absolute deadline, stored in the MV snapshot); nil/empty = no per-key TTL.
	// applied false (not an error) for an absent document.
	VectorMVSetPayload(ctx context.Context, name string, docID uint64, patch VectorMetadata, keyTTLMs map[string]int64, opts ...WriteOpts) (applied bool, err error)

	// VectorMVOverwritePayload replaces docID's entire payload. keyTTLMs sets the
	// per-key payload TTL for the new payload (relative ms; engine computes the
	// absolute deadline); nil/empty = no per-key TTL. applied false (not an error)
	// for an absent document.
	VectorMVOverwritePayload(ctx context.Context, name string, docID uint64, meta VectorMetadata, keyTTLMs map[string]int64, opts ...WriteOpts) (applied bool, err error)

	// VectorMVDeletePayloadKeys removes the listed keys from docID's payload. applied
	// false (not an error) for an absent document.
	VectorMVDeletePayloadKeys(ctx context.Context, name string, docID uint64, keys []string, opts ...WriteOpts) (applied bool, err error)

	// VectorMVClearPayload removes docID's entire payload. applied false (not an
	// error) for an absent document.
	VectorMVClearPayload(ctx context.Context, name string, docID uint64, opts ...WriteOpts) (applied bool, err error)
}

Store is the unified interface implemented by both backends. All methods are safe for concurrent use.

func NewClient

func NewClient(cfg ClientConfig) (Store, error)

NewClient constructs a networked Rostam client and returns a Store backed by it.

func NewDirect

func NewDirect(cfg DirectConfig) (Store, error)

NewDirect constructs an in-process Store backed by a single cache.Cache, with no Raft layer. Writes are ~30× faster than NewEmbedded because no log entry is created, no FSM dispatch happens, and no applied-index bookkeeping runs.

Use NewEmbedded when you need replication (multi-node clusters). Use NewDirect when you have a single-node deployment and want the cache layer's raw write speed.

func NewEmbedded

func NewEmbedded(cfg EmbeddedConfig) (Store, error)

type VectorConfig

type VectorConfig = vector.Config

VectorConfig configures a vector collection. Alias of vector.Config.

type VectorDocument

type VectorDocument = vector.Document

VectorDocument is a SearchDocs hit: a result enriched with its stored content and metadata. Alias of vector.Document.

type VectorFilter

type VectorFilter = vector.Filter

VectorFilter is a metadata predicate tree for filtered search. Alias of vector.Filter.

type VectorGroup

type VectorGroup = vector.Group

VectorGroup is one group of a group-by-document search: a shared key and its best hits. Alias of vector.Group.

type VectorGroupOpts

type VectorGroupOpts = vector.GroupOpts

VectorGroupOpts configures a group-by-document search. Alias of vector.GroupOpts.

type VectorHybridOpts

type VectorHybridOpts struct {
	Sparse  VectorSparse // query sparse vector; zero = dense-only
	Filter  VectorFilter // metadata predicate; zero = no filter
	Method  FusionMethod // FusionRRF (default), FusionWeighted, or FusionDBSF
	Alpha   float64      // weighted only: dense weight in [0,1]
	RRFK    int          // RRF constant; 0 = default 60
	DenseK  int          // dense-lane candidate pool; 0 = max(k, 50)
	SparseK int          // sparse-lane candidate pool; 0 = max(k, 50)

	// ReadConsistency controls which replicas may serve the query.
	// 0 = AnyReplica (default, fastest); 1 = LeaderOnly (best-effort, no barrier);
	// 2 = Linearizable (readIndex barrier — read-your-writes); 3 = BoundedStaleness
	// (any-replica read within MaxStaleness raft entries of the leader).
	// Applies only to the clustered backend when Partitions > 1; ignored otherwise.
	ReadConsistency uint8

	// OnPartitionUnavailable controls what happens when a shard partition is
	// unreachable during a cross-shard fan-out.
	// 0 = Partial (default, return results from available shards);
	// 1 = Fail (return an error if any partition is unavailable).
	// Applies only to the clustered backend when Partitions > 1; ignored otherwise.
	OnPartitionUnavailable uint8

	// MaxStaleness bounds replica lag (raft entries) behind the leader's
	// committed frontier; in effect ONLY when ReadConsistency==3 (BoundedStaleness).
	MaxStaleness uint64

	// GlobalIDF opts into the BM25 global-DF (dfs_query_then_fetch) two-phase text
	// lane: a partitioned (P>1) VectorHybridText gathers + sums per-shard corpus
	// stats into GLOBAL stats and re-scores the BM25 text lane with the SAME IDF so
	// the fused result matches a single-node corpus. The DENSE lane is unaffected.
	// Default false ⇒ the per-shard-local text lane (today's behavior), byte-
	// identical wire. Ignored for an unpartitioned collection.
	GlobalIDF bool
}

VectorHybridOpts carries the settings for a hybrid (dense + sparse) search.

type VectorInsertOpts

type VectorInsertOpts struct {
	TTL      time.Duration  // 0 = no expiry
	Metadata VectorMetadata // nil = no metadata
	Sparse   VectorSparse   // zero = no sparse lane

	// KeyTTLMs is an OPTIONAL per-key payload TTL map (payload key -> RELATIVE
	// ms). At insert/upsert the engine computes the ABSOLUTE deadline now+ttl for
	// each key (mirroring set_payload) and lazily drops the key once its deadline
	// passes, while the point itself lives on. nil/empty = no per-key TTL (the
	// zero-overhead, byte-identical wire path).
	KeyTTLMs map[string]int64

	// WriteOpts carries the write-consistency knobs (WriteConsistencyFactor,
	// Wait). Embedded so the zero value preserves today's behavior. The same
	// knobs will be added to the delete/payload write paths in a later task;
	// those methods do not yet take an opts struct, so there is nothing else to
	// embed into today.
	WriteOpts
}

VectorInsertOpts carries optional per-insert settings.

type VectorMetadata

type VectorMetadata = vector.Metadata

VectorMetadata is per-vector attribute data. Alias of vector.Metadata.

type VectorResult

type VectorResult = vector.Result

VectorResult is one entry in a VectorSearch result list. Alias of vector.Result.

type VectorScrollOpts

type VectorScrollOpts struct {
	// Cursor is the opaque resume-after-id pagination token from the previous
	// page (ops.EncodeScrollCursor). Empty = the first page (no lower bound,
	// id 0 included). The next page returns ids strictly greater than the cursor's
	// id, globally id-ascending. A malformed cursor fails loud (ops.ErrBadScrollCursor).
	Cursor string

	// ReadConsistency controls which replicas may serve the query.
	// 0 = AnyReplica (default, fastest); 1 = LeaderOnly (best-effort, no barrier);
	// 2 = Linearizable (readIndex barrier — read-your-writes); 3 = BoundedStaleness
	// (any-replica read within MaxStaleness raft entries of the leader).
	// Applies only to the clustered backend when Partitions > 1; ignored otherwise.
	ReadConsistency uint8

	// OnPartitionUnavailable controls what happens when a shard partition is
	// unreachable during a cross-shard fan-out.
	// 0 = Partial (default, return results from available shards);
	// 1 = Fail (return an error if any partition is unavailable).
	// Applies only to the clustered backend when Partitions > 1; ignored otherwise.
	OnPartitionUnavailable uint8

	// MaxStaleness bounds replica lag (raft entries) behind the leader's
	// committed frontier; in effect ONLY when ReadConsistency==3 (BoundedStaleness).
	MaxStaleness uint64

	// OrderBy, when non-nil, paginates the scroll by an arbitrary NUMERIC or
	// DATETIME payload field (Qdrant-style order_by) instead of the default
	// id-ascending order: the result set is globally ordered by the field's
	// (value, id) total order (ASC or DESC), points whose order field is
	// missing/non-numeric are EXCLUDED, and Cursor is then a v2 (value, id) resume
	// token (ops.EncodeScrollCursorOrder). nil = today's id-ascending scroll
	// (zero-overhead, v1 cursor). A v2 cursor with no OrderBy — or a v1/mismatched
	// cursor with an OrderBy — is rejected loud (ops.ErrCursorOrderMismatch).
	OrderBy *vector.OrderBy
}

VectorScrollOpts carries optional per-scroll settings. Scroll has no query tuning of its own; these are the cross-shard routing knobs.

type VectorSearchOpts

type VectorSearchOpts struct {
	Filter VectorFilter // zero = no filter

	// ReadConsistency controls which replicas may serve the query.
	// 0 = AnyReplica (default, fastest); 1 = LeaderOnly (best-effort, no barrier);
	// 2 = Linearizable (readIndex barrier — read-your-writes); 3 = BoundedStaleness
	// (any-replica read within MaxStaleness raft entries of the leader).
	// Applies only to the clustered backend when Partitions > 1; ignored otherwise.
	ReadConsistency uint8

	// OnPartitionUnavailable controls what happens when a shard partition is
	// unreachable during a cross-shard fan-out.
	// 0 = Partial (default, return results from available shards);
	// 1 = Fail (return an error if any partition is unavailable).
	// Applies only to the clustered backend when Partitions > 1; ignored otherwise.
	OnPartitionUnavailable uint8

	// MaxStaleness bounds replica lag (raft entries) behind the leader's
	// committed frontier; in effect ONLY when ReadConsistency==3 (BoundedStaleness).
	MaxStaleness uint64

	// GlobalIDF opts into the BM25 global-DF (dfs_query_then_fetch) two-phase text
	// search: a partitioned (P>1) VectorSearchText first gathers + sums per-shard
	// corpus stats (n/df/avgdl) into GLOBAL stats, then re-scores each shard with the
	// SAME IDF so the merged top-k is the EXACT global ranking (bit-identical to a
	// single-node corpus). Costs one extra round-trip. Default false ⇒ the per-shard-
	// local fast path (today's behavior), byte-identical wire. Ignored for an
	// unpartitioned collection (local corpus IS global) and outside the clustered
	// backend.
	GlobalIDF bool
}

VectorSearchOpts carries optional per-search settings.

type VectorSparse

type VectorSparse = vector.SparseVector

VectorSparse is a sparse vector for hybrid search. Alias of vector.SparseVector.

type WASMRegistration

type WASMRegistration = ops.WASMRegistration

WASMRegistration is a re-export of ops.WASMRegistration so callers do not have to import the internal ops package.

type WriteOpts

type WriteOpts struct {
	WriteConsistencyFactor uint8 // 0 = majority (default, no barrier); >0 = explicit factor (clamped to [1,RF])
	Wait                   *bool // nil = default true; &false = explicit no-barrier; &true = explicit wait

	// ExpectedVersion is the optimistic-CAS precondition: the per-point version the
	// caller expects the point to currently have. When non-nil the write applies
	// ONLY when it matches (0 = expect the point to be absent/new); a mismatch
	// returns vector.ErrVersionConflict with no mutation. nil (the default) = an
	// unconditional write (byte-identical to the pre-CAS wire). Honored by the
	// delete + payload-mutation write paths (insert/upsert carry it via
	// VectorInsertOpts.ExpectedVersion).
	ExpectedVersion *uint64

	// KeyTTLMs is an OPTIONAL per-key payload TTL map (payload key -> RELATIVE ms)
	// for the named-insert / MV-add write paths (dense insert/upsert carry it via
	// VectorInsertOpts.KeyTTLMs instead). At insert/add the engine computes the
	// ABSOLUTE deadline now+ttl for each key (mirroring set_payload) and lazily drops
	// the key once its deadline passes, while the point/document lives on. nil/empty =
	// no per-key TTL (the zero-overhead, byte-identical wire path).
	KeyTTLMs map[string]int64

	// Sparse is an OPTIONAL doc-level sparse vector for the MV-add write path: each
	// multi-vector document MAY carry one doc-level sparse vector alongside its dense
	// token matrix (the MV analogue of a named point's sparse space; consumed by the
	// MV hybrid search in a later task). nil/zero = dense-only (the zero-overhead,
	// byte-identical wire path — no add-wire trailer, no persist block). Ignored by
	// the other write paths.
	Sparse *VectorSparse
}

WriteOpts carries the tunable write-consistency knobs shared by every data-plane write (insert/upsert, delete-by-id, payload mutations, etc.). It is embedded into the per-op opts structs so the zero value (WriteConsistencyFactor==0, Wait==nil) means "today's behavior" — no __wc__ envelope is ever built and no barrier engages (see wcActive).

WriteConsistencyFactor is the number of replicas of the target shard's Raft group that must have applied the write before it returns success. It is clamped downstream to [1, RF]; 0 (unset) means "majority" — the Raft floor — i.e. byte-for-byte today's behavior with no barrier.

Wait is a tri-state bool: nil = default true (block until the factor is met); non-nil false = explicit wait=false (return at majority, skip the >majority barrier — a latency knob, NOT fire-and-forget, since Raft majority is the durability floor); non-nil true = explicit wait=true.