shardvault

Documentation

Overview

Package shardvault is the sharded, compressed bbolt implementation of the vault Engine (ADR-0025). Records route to one of N independent bbolt files by a hash of bucket and key, each shard living at vault/aa/bb/cc/aabbcc.vlt; values compress with zstd at the fastest level. Each shard is its own failure domain: losing one file loses 1/N of the keyspace, never the store. Cross-shard atomicity is not provided: an Update commits each touched shard independently, so recovery-sensitive callers stage durable intent before destructive writes.

Index

• func Open(dir string, quotaBytes int64, opts ...Option) (*vault.Vault, error)
• func OpenAt(legacyPath string, quotaBytes int64, opts ...Option) (*vault.Vault, error)
• type Option
  • func WithRootRecordRecovery(destination vault.Name, accept func(vault.Key, []byte) bool) Option
  • func WithWordFilter(bucket vault.Name, keyWidth int) Option

Functions

func Open

func Open(dir string, quotaBytes int64, opts ...Option) (*vault.Vault, error)

Open opens (or creates) the sharded vault rooted at dir with the given quota. The shard count derives from the quota and is recorded in the layout manifest on first open; later opens reuse the recorded count.

func OpenAt

func OpenAt(legacyPath string, quotaBytes int64, opts ...Option) (*vault.Vault, error)

OpenAt opens the sharded vault that lives beside the configured single-file storage path. A legacy single-file vault found at the path streams into the sharded layout once (compressing on the way) and is kept as a .migrated.bak file until the operator removes it.

Types

type Option

type Option func(*engine)

Option configures optional engine behavior at open time.

func WithRootRecordRecovery

func WithRootRecordRecovery(
	destination vault.Name,
	accept func(vault.Key, []byte) bool,
) Option

func WithWordFilter

func WithWordFilter(bucket vault.Name, keyWidth int) Option

WithWordFilter enables per-shard binary-fuse membership filters over the term keys of one bucket (PERF-READ-01): a fan-out read seeks only the shards whose filter admits the term prefix, skipping the shards that provably hold no posting for it. bucket names the filtered collection and keyWidth is the term-prefix length; the assembly layer supplies both so the storage engine stays independent of the RWI key layout (ADR-0039).