core

package
v0.3.0 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: May 10, 2026 License: Apache-2.0 Imports: 27 Imported by: 0

Documentation

Overview

Package core is the Scrinium Storage Engine (layer L2).

A self-contained CAS engine: it accepts Artifacts, runs them through a configurable Pipeline, places them on a backend through a Driver, and keeps accounting in a StoreIndex. It operates on cryptographic identifiers (ContentHash, BlobRef, ArtifactID) and has no knowledge of business metadata (Metadata is an opaque json.RawMessage).

The Store contract is split into three interfaces:

  • DataStore — operations on artifacts (Put, Get, Delete, Walk, and so on). The surface seen by client code, decorators, and Curator.
  • AdminStore — administrative API (Unlock, RotateKEK, UpdateConfig). The surface seen by the Store's owner.
  • Store — the union of the two. Returned by InitStore and OpenStore.

DAG: core imports event and driver. It does not import plugin, index, curator, agent, maintenance, or projection.

Index

Constants

View Source 
const (
	EventManifestSaved         = "core.manifest_saved"
	EventArtifactDeleted       = "core.artifact_deleted"
	EventBlobPhysicallyDeleted = "core.blob_physically_deleted"
	EventPackCompacted         = "core.pack_compacted"
	EventCapacityWarning       = "core.capacity_warning"
	EventScrubFailed           = "core.scrub_failed"
	EventStoreDegraded         = "core.store_degraded"
	EventKEKRotated            = "core.kek_rotated"
	EventStaleLeaseTakeover    = "core.stale_lease_takeover"
	EventOrphanScanCompleted   = "core.orphan_scan_completed"
)

Engine event-type constants. Used as the value of event.Event.Type.

Other reserved namespaces and their owners:

  • "agent.*" — agent/events.go
  • "curator.*" — curator/curator.go
  • "index.*" — index/events.go

User code must emit events under its own namespace. The reserved prefixes are enforced by convention; see docs/2. Internals/01 §1.7.

Variables

This section is empty.

Functions

func NewHashRegistry 

func NewHashRegistry() domain.HashRegistry

NewHashRegistry creates an empty hash-algorithm registry. The host application registers factories through Register.

Types

type AdminStore 

type AdminStore interface {
	// State returns the current state of the Store.
	State() domain.StoreState

	// Capabilities returns the bitmask of the underlying driver's
	// abilities. It is stable for the entire lifetime of the Store.
	Capabilities() driver.CapabilityMask

	// Unlock transitions the Store from StateLocked to StateUnlocked.
	// Idempotent: calling it again in StateUnlocked is a no-op.
	Unlock(ctx context.Context) error

	// ExportRecoveryKit returns the current Recovery Kit as bytes.
	// Available only in StateUnlocked and StateDegraded.
	ExportRecoveryKit(ctx context.Context) ([]byte, error)

	// RotateKEK re-encrypts the DEK with a new KEK. The data on disk
	// is not rewritten. The PassphraseProvider configured on the
	// Store is called twice — once for the current passphrase
	// (Reason="unlock", same as Store.Unlock) and once for the
	// replacement (Reason="kek_rotation").
	//
	// After RotateKEK the previous Recovery Kit is invalid; the
	// host is required to obtain a new one through ExportRecoveryKit
	// and persist it before reporting success to the user.
	RotateKEK(ctx context.Context) error

	// SetPassphrase enables encryption on a Store that was
	// initialised with a plaintext DEK. It calls the configured
	// PassphraseProvider once with Reason="set_passphrase" to obtain
	// the new passphrase, derives a KEK, wraps the existing DEK,
	// and persists the descriptor. The data on disk is not
	// rewritten.
	//
	// Refuses with errs.ErrPassphraseAlreadySet when the DEK is
	// already wrapped — use RotateKEK in that case. After
	// SetPassphrase the host MUST persist the freshly-issued
	// Recovery Kit through ExportRecoveryKit.
	SetPassphrase(ctx context.Context) error

	// SetMaintenanceMode transitions the Store into a maintenance
	// mode. Used before running a Maintenance Agent.
	SetMaintenanceMode(ctx context.Context, mode domain.MaintenanceMode) error

	// UpdateConfig updates the mutable parameters of StoreConfig.
	// Immutable parameters cannot be changed — errs.ErrConfigMismatch.
	//
	// Not yet wired: returns errs.ErrNotImplemented in M2. The
	// implementation lands with the configuration-history work in
	// M3.x — by then a new system.config artifact is written, the
	// pointer in system.config/current is bumped atomically, and
	// the active config in memory swaps without restart.
	UpdateConfig(ctx context.Context, cfg domain.StoreConfig) error

	// Config returns a snapshot of the active StoreConfig — the
	// projection persisted in system.config/current, with defaults
	// applied. Read-only; mutations of the returned value have no
	// effect on the running Store.
	Config() domain.StoreConfig

	// ConfigHistory returns the full history of configuration
	// versions.
	ConfigHistory(ctx context.Context) ([]domain.StoreConfig, error)

	// Close releases secrets held by the Store and transitions it
	// to a terminal state. After Close:
	//
	//   - The in-memory DEK is wiped.
	//   - The capability token (if any) is wiped.
	//   - The default StaticKeyResolver (if installed) drops its
	//     internal copy of the DEK.
	//   - The Store's state is set to Locked.
	//   - The Store's StoreIndex is NOT closed: the host owns the
	//     StoreIndex's lifetime (DI contract: see WithStoreIndex)
	//     and must call StoreIndex.Close after this method
	//     returns.
	//
	// Idempotent: a second Close on an already-closed Store
	// returns nil. Operations on a closed Store return an
	// implementation-defined error; do not call Close while reads
	// or writes are in flight.
	//
	// The intended caller is the host application's
	// graceful-shutdown path.
	Close() error
}

AdminStore is the administrative API. It is required by the Store owner: the code that called InitStore / OpenStore. It is not passed to decorators or Curator.

type AgentResult 

type AgentResult struct {
	AgentType   string
	StoreID     string
	StartedAt   time.Time
	CompletedAt time.Time
	Stats       map[string]int64
	Partial     bool // true if the work was interrupted and completed only partially
}

AgentResult is the result of an agent's work (one-shot or one background cycle). Used in EventAgentCompleted and EventAgentCycle.

type ArtifactDeletedPayload 

type ArtifactDeletedPayload struct {
	ArtifactID domain.ArtifactID
}

ArtifactDeletedPayload is the payload of EventArtifactDeleted. Emitted only when the logical deletion actually happens. If the deletion is rejected by retention or policy, the event is not emitted.

type BlobPhysicallyDeletedPayload 

type BlobPhysicallyDeletedPayload struct {
	BlobRef string
}

BlobPhysicallyDeletedPayload is the payload of EventBlobPhysicallyDeleted. Emitted by the GC Agent when the Sweep phase completes (physical removal of the file after the Grace Period).

type CapacityWarningPayload 

type CapacityWarningPayload struct {
	UsedPct float64
}

CapacityWarningPayload is the payload of EventCapacityWarning. UsedPct is the share of used space (0..100).

type DataStore 

type DataStore interface {

	// Put stores an artifact: it runs Payload through the Pipeline,
	// performs deduplication, and writes the blob and the manifest.
	// It returns ArtifactID — the cryptographic hash of the
	// serialised manifest file.
	Put(ctx context.Context, a domain.Artifact, opts domain.PutOptions) (domain.ArtifactID, error)

	// PutBlob writes an anonymous blob without creating a manifest.
	// Not a client method: it is used by level-3 decorators
	// (chunker.Wrapper for writing anonymous chunks).
	PutBlob(ctx context.Context, r io.Reader, blobType domain.BlobType) (domain.ContentHash, error)

	// Get opens an artifact for reading. It returns a ReadHandle —
	// a streaming primitive with lazy resolution of the physical
	// address.
	Get(ctx context.Context, id domain.ArtifactID, opts domain.GetOptions) (ReadHandle, error)

	// Delete performs a logical deletion: it removes the manifest
	// file from disk and decrements ref_count for every related blob
	// in a single StoreIndex transaction. Physical removal is
	// delegated to the GC Agent.
	Delete(ctx context.Context, id domain.ArtifactID) error

	// Verify performs a full integrity check of an artifact: it
	// re-hashes the manifest and the blob and runs the inverse
	// Pipeline with ContentHash verification. It ignores
	// CapNativeChecksum and VerifyOnRead.
	Verify(ctx context.Context, id domain.ArtifactID) error

	// RollbackSession is a group rollback of every artifact carrying
	// the given SessionID. It is idempotent: when interrupted, a
	// repeat call resumes the cleanup.
	RollbackSession(ctx context.Context, sessionID string) error

	// Walk iterates over user manifests. namespace = "*" — every user
	// namespace; an empty string — only the default one.
	// system.* is unreachable through Walk.
	Walk(ctx context.Context, namespace string, cb func(domain.Manifest) error) error

	// WalkSystem iterates over system manifests (the system.*
	// namespace). It requires a CapabilityToken.
	WalkSystem(ctx context.Context, namespace string, cb func(domain.Manifest) error) error

	// Capacity returns aggregated storage metrics.
	Capacity(ctx context.Context) (domain.StorageInfo, error)
}

DataStore is the artifact-facing API. This interface is sufficient for client code, decorators (bundler.Wrapper, chunker.Wrapper), and Curator. It does not expose administrative operations.

type Decoder 

type Decoder interface {
	Transform(r io.Reader) io.Reader
}

Decoder is the per-read transformation plugin (used by Get). Created via TransformerFactory.NewDecoder(stage); receives IV and other stage parameters via PipelineStage.

type DescriptorCache 

type DescriptorCache struct {
	// Blob is the full JSON serialisation of the descriptor —
	// byte-identical to what Persist would write to L0.
	Blob []byte

	// Sequence duplicates Blob's sequence field. Held separately
	// so that "is the cache stale relative to Location" can be
	// answered without parsing Blob.
	Sequence uint64

	// Checksum is SHA-256 over Blob. ChecksumLen bytes; hex-
	// encoded in store_meta.
	Checksum []byte
}

DescriptorCache is the L2 cached projection of the on-disk descriptor. The three fields are written together by saveDescriptorCache; corruption (partial write, manual edit) surfaces at load time as an error rather than a half-populated struct.

type Encoder 

type Encoder interface {
	// Transform takes an incoming io.Reader and returns a wrapped
	// one. The Pipeline runner builds the chain: the output of one
	// stage is the input of the next. O(1) memory — no buffering of
	// the entire stream.
	Transform(r io.Reader) io.Reader

	// Result is called by the Pipeline runner after EOF — once the
	// whole stream has flowed through this Encoder. It returns the
	// transformation metrics.
	Result() TransformResult
}

Encoder is the per-write transformation plugin (used by Put). Created via TransformerFactory.NewEncoder(); lives for one operation. It is not required to be safe for concurrent use.

type KeyResolver 

type KeyResolver interface {
	GetKeys(keyID string) ([][]byte, error)
	DefaultKeyID() string
}

KeyResolver is the plugin that resolves a DEK by its string KeyID. It allows a Store to support several DEKs simultaneously: multi-tenant stores, mixed recovered data, intermediate states during key rotation, crypto-shredding.

On write the engine takes the KeyID from DefaultKeyID() and writes it into the manifest header. On read the KeyID is read from the header, GetKeys returns a list of candidates, and the engine transparently iterates over them until one decrypts successfully or the list is exhausted.

func NewStaticKeyResolver 

func NewStaticKeyResolver(dek []byte) KeyResolver

NewStaticKeyResolver creates a KeyResolver that returns the same DEK for any request. DefaultKeyID returns an empty string. This is the default behaviour: one Store, one DEK.

type LeaseTakeoverPayload 

type LeaseTakeoverPayload struct {
	LeaseKey       string
	PreviousHolder string
	ExpiredAt      time.Time
	TakenBy        string
}

LeaseTakeoverPayload is the payload of two events: core.EventStaleLeaseTakeover (Store-level lease — Open under stale location.lock) and agent.EventAgentStaleLease (an agent took over from a previous holder that stopped renewing). The stale-lease concept is layer-agnostic, so a single struct describes both. Lives in core because core was the first emitter; agent imports core for unrelated reasons already, so no new dependency is introduced.

type MaintenanceAgent 

type MaintenanceAgent interface {
	// Validate checks whether the operation is applicable to the
	// current state of the Store: required maintenance mode,
	// presence of required parameters, availability of
	// dependencies.
	Validate(ctx context.Context) error

	// Run starts the operation. It acquires a maintenance/lease,
	// performs the work, and releases the lease. It returns the
	// result with accumulated statistics.
	Run(ctx context.Context) (*AgentResult, error)
}

MaintenanceAgent is the contract of a one-shot administrative operation. Declared here (rather than in agent/) so that Store can require a MaintenanceAgent to be validated through Validate without depending on higher layers.

type ManifestSavedPayload 

type ManifestSavedPayload struct {
	Manifest  domain.Manifest
	IsTransit bool
}

ManifestSavedPayload is the payload of EventManifestSaved. IsTransit is true when the file was placed into HostStorage.system.transit and has not yet been drained to a Target. After Drain (at the Curator level) EventDrainCompleted is emitted.

type OrphanReport 

type OrphanReport struct {
	StagingRemoved   int
	BlobsRemoved     int
	ManifestsRemoved int
	Errors           []error
	Duration         time.Duration
}

OrphanReport is the result of a bootstrap recoverOrphans pass. Counts are files actually removed from disk; Errors collects non-fatal failures (per-file Driver.Remove rejections, parse glitches, individual Resolve/ManifestExists infrastructure errors) — none of these stop the scan or block the Store from opening.

type OrphanScanCompletedPayload 

type OrphanScanCompletedPayload struct {
	StagingRemoved   int
	BlobsRemoved     int
	ManifestsRemoved int
	NonFatalErrors   int
	Duration         time.Duration
}

OrphanScanCompletedPayload is the payload of EventOrphanScanCompleted. Emitted by the bootstrap recovery after every transition into Unlocked, summarising what the scan found and removed. Counts are physical files removed; non-fatal I/O errors during the scan (per-file Remove failures, individual path-parse glitches) are aggregated into NonFatalErrors. The scan never refuses to open a Store — operators read the count here and dig into engine logs for details.

type PackCompactedPayload 

type PackCompactedPayload struct {
	OldPackRef  string
	NewPackRef  string
	LiveEntries int
	DeadEntries int
	OldSize     int64
	NewSize     int64
	FreedBytes  int64
	Duration    time.Duration
}

PackCompactedPayload is the payload of EventPackCompacted. Emitted on a successful compaction of a partially dead pack volume.

type PassphraseHint 

type PassphraseHint struct {
	StoreID string
	Reason  string
}

PassphraseHint is the call context for a PassphraseProvider.

Reason takes one of:

  • "init" — InitStore is generating a fresh Store and needs the passphrase that will wrap the just-generated DEK. StoreID carries the freshly generated UUID.
  • "unlock" — OpenStore, Store.Unlock, or the first half of Store.RotateKEK needs the current passphrase to unwrap the DEK. Hosts that cache passphrases in a keychain key off this Reason for both unlock paths.
  • "set_passphrase" — Store.SetPassphrase is wrapping a DEK that is currently in plaintext. The provider returns the NEW passphrase.
  • "kek_rotation" — the second half of Store.RotateKEK; the provider returns the NEW passphrase that will wrap the existing DEK.

type PassphraseProvider 

type PassphraseProvider func(ctx context.Context, hint PassphraseHint) ([]byte, error)

PassphraseProvider returns a passphrase used to derive the KEK through the KDF. The buffer is zeroed by the engine after the KEK has been derived.

type Publisher 

type Publisher interface {
	Publish(e event.Event)
}

Publisher is the minimal contract for emitting events; it is passed to Store via WithPublisher. It is satisfied by event.EventBus and by any custom implementation (asynchronous, persistent, filtering).

type ReadHandle 

type ReadHandle interface {
	io.ReadCloser
	io.ReaderAt

	// SupportsRandomAccess reports the static availability of
	// ReadAt/ReadAtCtx. It depends on the source's physics and the
	// composition of the Pipeline.
	SupportsRandomAccess() bool

	// ReadAtCtx is the same as ReadAt but takes an explicit
	// cancellation context. Used with network drivers, slow media,
	// and operations that require an external timeout.
	ReadAtCtx(ctx context.Context, p []byte, off int64) (n int, err error)

	// Manifest returns the parsed manifest of the artifact. Available
	// immediately after Get, before the first Read. It does not block
	// or perform I/O.
	Manifest() domain.Manifest
}

ReadHandle is the read primitive returned by Get. It hides the physical source of the bytes (a single file, a HostStorage record, a range read from a .pack volume, or an inline blob). Support for ReadAt/ReadAtCtx is reported by SupportsRandomAccess; outside of those conditions the calls return ErrRandomAccessNotSupported.

type ScrubFailedPayload 

type ScrubFailedPayload struct {
	ArtifactID domain.ArtifactID
	Err        error
}

ScrubFailedPayload is the payload of EventScrubFailed. Emitted by the Scrub Agent or by Store.Verify on a hash divergence.

type Store 

type Store interface {
	DataStore
	AdminStore
}

Store is the union of DataStore and AdminStore. Returned by InitStore and OpenStore. By passing only DataStore (and not Store) to a decorator or Curator, the host application guarantees at the type level that the administrative methods are unreachable from that code.

func InitStore 

func InitStore(ctx context.Context, drv driver.Driver, opts ...StoreOption) (Store, []byte, error)

InitStore creates a new Store at the Location served by drv.

Behaviour:

  1. Probe the Driver for an existing descriptor. If one is present and WithForceReinit is NOT set, return errs.ErrStoreAlreadyExists.
  2. With WithForceReinit, wipe the structural state — the descriptor and the manifests/ directory. Existing blobs/ are NOT removed unless WithPurgeOnReinit is also set; this lets a user start a fresh Store on top of orphan blobs and let GC reclaim them.
  3. Generate a fresh StoreID. Apply config defaults. Validate immutable parameters.
  4. Validate that a StoreIndex was provided via WithStoreIndex. core does NOT open the index itself — the caller wires the concrete implementation (sqlite.NewStore, in-memory, etc.) and passes it as a dependency. This keeps core free of any import dependency on index/* packages (DAG: core ← index).
  5. Generate a 32-byte DEK from crypto/rand. The DEK is generated unconditionally per §3.1; encryption can be turned on later through SetPassphrase without re-keying.
  6. If WithPassphrase is configured, derive a KEK through Argon2id and wrap the DEK with AES-256-GCM. The wrapped DEK plus its KDFParams land in the descriptor. Otherwise the DEK is stored in the descriptor in plaintext — semantically honest: no passphrase, no protection.
  7. Write store.json (both replicas) and the L2 cache.
  8. Construct the *store object in StateUnlocked and return. For encrypted Stores, also return the Recovery Kit bytes — the host MUST persist them before reporting success to the user.

Recovery Kit:

  • nil for Plain-DEK Stores (no encryption to recover).
  • non-nil text bytes per §10.3 for encrypted Stores.

Refusal cases:

  • ManifestCrypto != Plain without WithPassphrase → errs.ErrPassphraseRequired. An unprotected DEK plus encrypted manifests is the worst-of-both-worlds shape: anyone who reads store.json gets the keys to all the manifests for free.

func OpenStore 

func OpenStore(ctx context.Context, drv driver.Driver, opts ...StoreOption) (Store, error)

OpenStore opens an existing Store at the Location served by drv.

Behaviour:

  1. Read both descriptor replicas (L0 store.json, L1 .store.backup.json) per §10.1.5 and reconcile them. If both are absent → errs.ErrStoreNotFound. If both are unrecoverable (corrupted, or one corrupted + one absent) → errs.ErrStoreCorrupted. If both are valid but content differs at the same Sequence → errs.ErrDescriptorSplitBrain. Otherwise the canonical replica is selected (sequence-wins; equal-content short-circuit) and any heal action is performed against the Driver before proceeding.
  2. Reconcile the L2 cache (store_meta) with the canonical descriptor. The cache is rewritten when absent, when its checksum diverges, or when its load fails — Location is the source of truth, the cache is a fast-start aid.
  3. Validate that a StoreIndex was provided via WithStoreIndex. core never opens an index itself; the caller is responsible for the dependency. Missing → error.
  4. Load the active StoreConfig from system.config/current. Defaults are applied; immutable parameters are validated.
  5. Validate WithConfig (when supplied) against the active config: immutable mismatch → errs.ErrConfigMismatch. A caller without WithConfig accepts the on-disk config as-is — a legitimate scenario for diagnostic tools and projection-only consumers.
  6. State machine and bootstrap: - DEKEncrypted=false: Plain DEK in descriptor, used directly. Run Orphan Scan, transition to StateUnlocked. - DEKEncrypted=true + WithAutoUnlock: invoke the configured PassphraseProvider with Reason="unlock", unwrap DEK, run Orphan Scan, transition to StateUnlocked. - DEKEncrypted=true without WithAutoUnlock: skip Orphan Scan (the index walk needs the DEK in M2.3+ for Envelope manifests; until then it would still succeed for Plain manifests, but we treat the state uniformly), transition to StateLocked. The next Store.Unlock call completes bootstrap.

MetadataOnly and Envelope manifest crypto are still rejected pending M2.3. The split between "encrypted DEK" (this pack) and "encrypted manifests" (next milestone) is deliberate: they are independent axes of the configuration and a Store with a passphrase-protected DEK plus Plain manifests is a useful intermediate.

What does NOT happen yet (planned milestones in parens):

  • location.lock acquisition / lease model (M3.1).
  • StoreIndex schema cross-check against descriptor (M3.4).

type StoreDegradedPayload 

type StoreDegradedPayload struct {
	Reason string
}

StoreDegradedPayload is the payload of EventStoreDegraded. Emitted on a transition into StateDegraded (descriptor-replica divergence).

type StoreIndex 

type StoreIndex interface {

	// IndexManifest registers an artifact in the index. It branches
	// on manifest.Type:
	//   - blob: upsert blob, increment ref_count, insert manifest.
	//   - toc:  + increment ref_count for each chunkRef.
	//   - pack: transitive registration of every packed artifact via
	//     packedEntries (see docs/2. Internals/09 §9.2.1).
	IndexManifest(
		ctx context.Context,
		m domain.Manifest,
		addr domain.PhysicalAddress,
		chunkRefs []string,
		packedEntries []domain.PackedEntry,
	) error

	// DeleteManifest performs a logical deletion: a single
	// transaction, DELETE manifest + decrement ref_count for each
	// blobRef.
	DeleteManifest(ctx context.Context, artifactID domain.ArtifactID, blobRefs []string) error

	// RebindBlob moves a blob from Workspace: Host to
	// Workspace: Location after a successful Drain. ref_count is
	// not changed. Idempotent: a no-op when the record is missing.
	RebindBlob(ctx context.Context, blobRef string, newAddr domain.PhysicalAddress) error

	// Resolve returns the physical address for a BlobRef.
	Resolve(ctx context.Context, blobRef string) (domain.PhysicalAddress, error)

	// ExistsByContent is an exact check by the composite key
	// (ContentHash, OriginalSize). The deduplication key for regular
	// blobs.
	ExistsByContent(ctx context.Context, hash domain.ContentHash, originalSize int64) (blobRef string, exists bool, err error)

	// ExistsByHash is the check by ContentHash with tombstone
	// distinction. Used by chunker.Wrapper for chunk deduplication.
	ExistsByHash(ctx context.Context, hash domain.ContentHash) (domain.BlobExistStatus, error)

	// GetRefCount returns the current reference count of a blob.
	GetRefCount(ctx context.Context, blobRef string) (int, error)

	// LookupPacked returns the data needed for a range read by the
	// ArtifactID of a packed artifact. The second return value is
	// false when the artifact is not packed (it lives in /blobs/ or
	// /manifests/ as usual).
	LookupPacked(ctx context.Context, artifactID domain.ArtifactID) (domain.PackedBlobInfo, bool, error)

	// ManifestExists reports whether a manifest row with the given
	// ArtifactID is present in the index. It is the manifests-side
	// counterpart of Resolve: a point-lookup that does not return
	// the row contents, only its presence. Used by the bootstrap
	// Orphan Scan to find manifest files on disk that have no
	// matching index row (the crash window between Driver.Put on
	// the manifest path and the IndexManifest transaction).
	//
	// A false return with a nil error is the normal "not present"
	// signal. Errors are reserved for index-infrastructure
	// failures.
	ManifestExists(ctx context.Context, id domain.ArtifactID) (bool, error)

	// ListByNamespace iterates over manifests with the given
	// namespace. "*" — all namespaces; "" — only the default
	// (empty). Returns blob and toc; pack is excluded.
	ListByNamespace(ctx context.Context, ns string, cb func(domain.Manifest) error) error

	// ListOrphanBlobs iterates over blobs with ref_count = 0. Used
	// by the GC Agent.
	ListOrphanBlobs(ctx context.Context, cb func(blobRef string) error) error

	// ListUnverified iterates over blobs whose last_verified_at is
	// older than `before`. Used by the Scrub Agent.
	ListUnverified(ctx context.Context, before time.Time, cb func(blobRef string) error) error

	// GetBySession returns every ArtifactID with the given
	// SessionID. Used by RollbackSession.
	GetBySession(ctx context.Context, sessionID string) ([]domain.ArtifactID, error)

	// MarkVerified updates last_verified_at for a blob.
	MarkVerified(ctx context.Context, blobRef string, timestamp time.Time) error

	// DeletePacked removes every packed_blobs record of a given
	// pack volume. Called by the GC Agent before tombstoning the
	// pack blob.
	DeletePacked(ctx context.Context, packBlobRef string) error

	// VacuumInto creates a snapshot of the index at the given path.
	// Used by the Snapshot Agent.
	VacuumInto(ctx context.Context, destPath string) error

	GetMeta(ctx context.Context, key string) (string, error)
	SetMeta(ctx context.Context, key, value string) error

	// Close releases resources held by the index — database
	// connections, file handles, internal goroutines. The host
	// application owns the StoreIndex's lifetime (DI contract: see
	// core.WithStoreIndex doc) and must call Close after the Store
	// has been shut down.
	//
	// Idempotent: a second Close on an already-closed index returns
	// nil. Operations on a closed index return an implementation-
	// defined error; do not call Close while reads are in flight.
	Close() error
}

StoreIndex is the index of a single Store. Every mutating method encapsulates its transaction inside; the calling code never drives transactions explicitly.

Implementations (in-memory, sqlite, postgres) live in subpackages of index/.

Every method that performs I/O takes a context.Context as the first argument. Backends honour cancellation: a deadline expiry or explicit cancel during a SQL call surfaces as the standard context error, classified through the backend's error mapper to the appropriate scrinium sentinel where applicable. The single exception is Close, which is an idempotent shutdown step and follows the standard io.Closer signature.

type StoreOption 

type StoreOption func(*storeOptions)

StoreOption is an option for the Store constructor. It applies to InitStore and OpenStore. The order in which options are passed is irrelevant.

func WithAutoUnlock 

func WithAutoUnlock() StoreOption

WithAutoUnlock instructs OpenStore to call Unlock automatically on an encrypted Store. Without this flag, OpenStore returns the Store in StateLocked.

func WithCapabilityToken 

func WithCapabilityToken(token []byte) StoreOption

WithCapabilityToken provides a token for elevated permissions (such as access to system.* through WalkSystem).

func WithConfig 

func WithConfig(cfg domain.StoreConfig) StoreOption

WithConfig provides the Store configuration. At InitStore it fixes the immutable parameters. At OpenStore it is checked against the configuration loaded from system.config/current — a divergence in immutable fields produces errs.ErrConfigMismatch.

func WithForceReinit 

func WithForceReinit() StoreOption

WithForceReinit allows InitStore to run on top of an existing Store (deleting L0, L1, the StoreIndex, and the manifests/ directory). The operation is irreversible.

func WithHashRegistry 

func WithHashRegistry(r domain.HashRegistry) StoreOption

WithHashRegistry provides the registry of hash algorithms. Required. Used by the Pipeline runner, Recovery Agent, and parsers.

func WithKeyResolver 

func WithKeyResolver(r KeyResolver) StoreOption

WithKeyResolver provides the key-resolver plugin. By default the engine uses StaticKeyResolver populated with the DEK from store.json.

func WithPassphrase 

func WithPassphrase(provider PassphraseProvider) StoreOption

WithPassphrase provides the KEK provider. Required when ManifestCrypto != Plain. With Plain it is ignored.

func WithPublisher 

func WithPublisher(p Publisher) StoreOption

WithPublisher provides a Publisher implementation for emitting events.

func WithPurgeOnReinit 

func WithPurgeOnReinit() StoreOption

WithPurgeOnReinit, in combination with WithForceReinit, also removes physical blobs (rather than leaving them as orphans for later GC).

func WithReadRegistry 

func WithReadRegistry(r TransformerRegistry) StoreOption

WithReadRegistry provides the registry of transformation plugins. Required when StoreConfig.Pipeline is non-empty or MetadataTransformer is set.

func WithStoreIndex 

func WithStoreIndex(idx StoreIndex) StoreOption

WithStoreIndex provides the StoreIndex implementation. Required.

type TransformResult 

type TransformResult struct {
	// OutputSize — number of bytes that left the stage's output.
	OutputSize int64

	// IV — initialisation vector for crypto plugins. Written to
	// manifest.Pipeline[i].IV. nil for non-crypto plugins.
	IV []byte

	// Entropy — Shannon entropy of the output stream (for
	// compressors). Used to decide whether to skip compressing
	// uncompressible input.
	Entropy float64
}

TransformResult is the result of an Encoder, captured by the Pipeline runner after EOF.

type TransformerFactory 

type TransformerFactory interface {
	NewEncoder() Encoder
	NewDecoder(stage domain.PipelineStage) Decoder
}

TransformerFactory is the factory of Encoder/Decoder instances for a single algorithm. State shared between instances (a common zstd dictionary, a common encryption key) belongs to the factory.

type TransformerRegistry 

type TransformerRegistry interface {
	Get(id string) (TransformerFactory, error)
	Register(id string, f TransformerFactory) TransformerRegistry
}

TransformerRegistry is the registry of transformation factories keyed by algorithm identifier (for example, "zstd", "aes-gcm"). The identifier appears in the manifest in pipeline[].algorithm.

func NewTransformerRegistry 

func NewTransformerRegistry() TransformerRegistry

NewTransformerRegistry creates an empty transformer registry. The host application registers factories through Register.

Source Files

View all Source files

Directories

Path Synopsis
internal
descriptor
Package descriptor reads and writes the Store descriptor file (store.json).
 Package descriptor reads and writes the Store descriptor file (store.json).
kdf
Package kdf is the Scrinium key-derivation primitive.
 Package kdf is the Scrinium key-derivation primitive.
keywrap
Package keywrap wraps and unwraps the data-encryption key (DEK) with the key-encryption key (KEK) using AES-256-GCM.
 Package keywrap wraps and unwraps the data-encryption key (DEK) with the key-encryption key (KEK) using AES-256-GCM.
recoverykit
Package recoverykit encodes and decodes the Scrinium Recovery Kit per §10.3.
 Package recoverykit encodes and decodes the Scrinium Recovery Kit per §10.3.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.