store

Documentation

Overview

Package store defines the pluggable record-store SPI that lets any kit read double as a crawl: with --db set, every emitted record is upserted into a local store before it reaches the output.

The SPI is backend-neutral. A record reaches a Store already marshalled to JSON with its primary key extracted, so a backend never reflects over domain types and the same interface serves SQLite (the built-in default), DuckDB, Postgres, or anything else. Backends register themselves under a URL scheme with Register; callers open one with Open and a DSN.

DSN forms:

x.db                      bare path  -> the default scheme (sqlite)
sqlite:/var/data/x.db     explicit scheme + path
duckdb:x.duckdb           another file backend
postgres://user@host/db   a full URL is passed through verbatim

Index

• func Collection(name string) string
• func ParseDSN(dsn string) (scheme, target string)
• func Register(scheme string, open Opener)
• func Schemes() []string
• func SetDefaultScheme(scheme string)
• type Opener
• type Store
  • func Open(ctx context.Context, dsn string) (Store, error)

Functions

func Collection

func Collection(name string) string

Collection turns a record type name into a safe collection (table) name: snake_case, alphanumerics and underscores only. It is exported so backends and the core agree on the same normalization.

func ParseDSN

func ParseDSN(dsn string) (scheme, target string)

ParseDSN splits a DSN into its scheme and backend target. A URL-style DSN ("scheme://...") keeps the whole string as the target so the URL backend gets everything it needs. A "scheme:target" DSN whose scheme is registered splits on the first colon. Anything else is a bare path under the default scheme.

func Register

func Register(scheme string, open Opener)

Register installs a backend under a scheme. Backends call this from an init function so a blank import wires them in. Registering a scheme twice panics, which surfaces a duplicate-driver bug at startup rather than silently.

func Schemes

func Schemes() []string

Schemes lists the registered backend schemes, sorted.

func SetDefaultScheme

func SetDefaultScheme(scheme string)

SetDefaultScheme changes the scheme used for a bare-path DSN (default "sqlite"). A consumer that prefers another built-in backend can call this.

Types

type Opener

type Opener func(ctx context.Context, target string) (Store, error)

Opener opens a Store from the backend-specific target parsed out of a DSN (the part after the scheme for file backends, or the full DSN for URL backends).

type Store

type Store interface {
	// Upsert writes one record into the named collection, keyed by id. data is
	// the record marshalled to JSON. An empty id means "append" (the backend
	// supplies a key).
	Upsert(ctx context.Context, collection, id string, data []byte) error
	// Close flushes and releases the backend (closing the file/connection).
	Close() error
}

Store persists records. Implementations must be safe for sequential use by a single command run; concurrent use is the caller's responsibility.

func Open

func Open(ctx context.Context, dsn string) (Store, error)

Open resolves the DSN to a backend and opens a Store. It returns a clear error naming the available schemes when the backend is not registered (the common case being a DuckDB or Postgres DSN in a build that did not import the backend).