domain

Documentation

Overview

Package domain holds the value types of the Scrinium engine — artifacts, manifests, physical addresses, configuration enums. They are pure data: structs and typed strings, no methods that do I/O, no imports beyond the Go standard library.

Why a separate package: helpers under internal/* (blobpath, manifestcodec, future codecs) need these types. If they lived in core/, the helpers would import core, and core imports the helpers — a cycle. Pulling the value types one level below breaks the cycle by making the dependency one-way:

domain  ← core  ← internal/lib
   ↑              ↑
   └──────────────┘  (helpers also import domain)

User code imports both packages: "core" for the verbs (InitStore, OpenStore, options, plugins, events) and "domain" for the values passed into them (StoreConfig, PutOptions, Manifest, the policy enums). The convention follows database/sql ↔ database/sql/driver and net/http ↔ net/url in the standard library.

DAG: domain imports nothing from the project. It is the leaf.

Index

• Constants
• type Artifact
• type ArtifactID
• type BlobExistStatus
• type BlobRef
• type BlobStorage
• type BlobType
• type ContentHash
• type ContentHashAlgorithm
• type DeletionPolicy
• type GCLeasePolicy
• type GetOptions
• type HashRegistry
• type KDFParams
• type LayoutHeader
• type MaintenanceMode
• type Manifest
• type ManifestCrypto
• type ManifestEncoding
• type ManifestStorage
• type ManifestSystemFlags
• type ManifestType
• type PackAlignmentPolicy
• type PackedBlobInfo
• type PackedEntry
• type PathTopology
• type PhysicalAddress
• type PipelineStage
• type PutOptions
• type RoutingHints
• type StorageInfo
• type StoreConfig
• type StoreID
• type StoreState
• type VerifyOnReadPolicy
• type Workspace

Constants

const (
	LayoutInline      = "Inline"
	LayoutTarget      = "Target"
	LayoutExternalRef = "ExternalRef"
)

Layout* are the canonical values for LayoutHeader.BlobStorage. Distinct from BlobStorage* (which is the StoreConfig-time policy): the configuration says "InlineFallback", but the resolved layout for any specific manifest is either LayoutInline (the payload fit the inline limit) or LayoutTarget (it overflowed).

Production code must compare against these constants, not the equivalent string literals.

const (
	// NamespaceSystemTransit holds files staged in HostStorage's
	// transit area before Drain delivers them to a Target.
	NamespaceSystemTransit = "system.transit"

	// NamespaceSystemManifests holds bundler-produced .pack
	// manifests (engine-internal; invisible through Walk).
	NamespaceSystemManifests = "system.manifests"

	// NamespaceSystemState holds engine-managed state artifacts:
	// staging blobs (system.state/staging/), the scrub cursor,
	// index snapshots, agent leases.
	NamespaceSystemState = "system.state"

	// NamespaceSystemConfig holds the active StoreConfig artifact;
	// the pointer file lives at NamespaceSystemConfig + "/current".
	NamespaceSystemConfig = "system.config"
)

Reserved system namespaces. The engine emits these from internal writers (system.config from InitStore; system.state from agents; system.transit and system.manifests from Curator+bundler in M3+); user code reads them through Store.WalkSystem with a capability token.

const MaxInlineBlobLimit = 64 * 1024

MaxInlineBlobLimit is the maximum value StoreConfig.InlineBlobLimit can take per docs/4 §5.6. 64 KiB. Bigger limits would push hot index pages out of SQLite page cache because inline blobs live inside the manifest row. Returns errs.ErrInvalidConfig when exceeded.

const MaxKeyIDLength = 255

MaxKeyIDLength is the upper bound on KeyID byte length in the manifest file header per §7.1. The KeyID-length byte is one octet, so the KeyID itself can be at most 255 bytes; producers that hit the limit must shorten the identifier or prefix-hash it externally. Returns a wrapped error from manifestcodec when exceeded.

const MaxManifestSize = 1024 * 1024

MaxManifestSize is the maximum byte size of a serialised Manifest. 1 MiB. Returns errs.ErrManifestTooLarge when exceeded. A manifest with thousands of chunks or huge Metadata is a design error.

const MaxMetadataSize = 64 * 1024

MaxMetadataSize is the maximum byte size of the Manifest Metadata block. 64 KiB. Returns errs.ErrMetadataTooLarge when exceeded. Metadata is for tags and paths, not for documents.

const MaxNamespaceLen = 255

MaxNamespaceLen is the maximum byte length of Namespace. Returns errs.ErrNamespaceTooLong when exceeded. Standard FS/DB name limit.

const MaxSessionIDLen = 255

MaxSessionIDLen is the maximum byte length of SessionID. Returns errs.ErrSessionIDTooLong when exceeded. Standard FS/DB name limit.

const MinRetentionPeriod = time.Hour

MinRetentionPeriod is the minimum non-zero RetentionPeriod per docs/4 §5.6. 1 hour. A shorter period makes deferred deletion pointless — by the time GC runs, the retention window has already expired. Returns errs.ErrInvalidConfig when violated.

const MinTombstoneGracePeriod = time.Hour

MinTombstoneGracePeriod is the minimum non-zero TombstoneGracePeriod. 1 hour. Returns errs.ErrInvalidTombstoneGracePeriod when violated. A shorter period breaks the Revive flow across hosts.

const NamespaceSystemPrefix = "system."

NamespaceSystemPrefix is the reserved prefix for engine-private namespaces. A user-supplied Namespace starting with this prefix is rejected at the Put boundary with errs.ErrReservedNamespace.

const NamespaceWildcard = "*"

NamespaceWildcard is the special token meaning "every user namespace" in Store.Walk and "deny" in Store.Put.

Types

type Artifact

type Artifact struct {
	Payload  io.Reader
	Metadata json.RawMessage
}

Artifact is the abstraction at the system boundary (input/output). It consists of a byte stream (Payload) and a business context (Metadata). The engine treats Metadata as an opaque blob — parsing tags and paths is strictly the responsibility of the host application.

type ArtifactID

type ArtifactID string

ArtifactID is the public identifier of an Artifact. It is a cryptographic hash of the final serialised manifest file (header included). Format: "<algo>-<hex>" (for example, "sha256-abc..."). Any change to the metadata produces a new Manifest and a new ArtifactID.

type BlobExistStatus

type BlobExistStatus uint8

BlobExistStatus is the result of ExistsByHash.

const (
	BlobNotFound    BlobExistStatus = 0
	BlobExists      BlobExistStatus = 1
	BlobIsTombstone BlobExistStatus = 2
)

type BlobRef

type BlobRef string

BlobRef is the hash of the final transformed blob stream (after compression and encryption). Used as the physical filename when blobs are stored individually. Applies to all blobs, including chunks and TOC blobs.

type BlobStorage

type BlobStorage string

BlobStorage is the blob placement strategy.

const (
	BlobStorageTarget         BlobStorage = "Target"
	BlobStorageInlineFallback BlobStorage = "InlineFallback"
	BlobStorageExternalRef    BlobStorage = "ExternalRef"
)

type BlobType

type BlobType string

BlobType is the type of a blob; it determines the target write directory.

const (
	BlobTypeRegular BlobType = "Regular"
	BlobTypePack    BlobType = "Pack"
	BlobTypeChunk   BlobType = "Chunk"
)

type ContentHash

type ContentHash string

ContentHash is the hash of the original payload before any transformation. The global deduplication key: two files with the same content share a ContentHash regardless of Pipeline configuration.

type ContentHashAlgorithm

type ContentHashAlgorithm string

ContentHashAlgorithm identifies a content-hashing algorithm. An immutable Store parameter: changing it breaks deduplication and verification of historical artifacts.

const (
	HashSHA256 ContentHashAlgorithm = "sha256"
	HashBLAKE3 ContentHashAlgorithm = "blake3"
)

type DeletionPolicy

type DeletionPolicy string

DeletionPolicy is the deletion policy.

const (
	DeletionPolicyNoDelete  DeletionPolicy = "NoDelete"
	DeletionPolicyRetention DeletionPolicy = "Retention"
	DeletionPolicyFree      DeletionPolicy = "Free"
)

type GCLeasePolicy

type GCLeasePolicy string

GCLeasePolicy is the policy for GC Agent coordination.

const (
	GCLeaseAuto           GCLeasePolicy = "Auto"
	GCLeaseSingleHost     GCLeasePolicy = "SingleHost"
	GCLeaseLeaderElection GCLeasePolicy = "LeaderElection"
)

type GetOptions

type GetOptions struct {
	AllowColdRead bool
}

GetOptions is the call context for Store.Get.

type HashRegistry

type HashRegistry interface {
	// Parse splits an "<algo>-<hex>" identifier into the algorithm
	// name and the raw hash bytes.
	Parse(h string) (algo string, raw []byte, err error)

	// NewHasher creates a fresh hash.Hash for the given algorithm.
	NewHasher(algo string) (hash.Hash, error)

	// Format builds an identifier string from an algorithm name and
	// raw bytes.
	Format(algo string, raw []byte) string

	// Register registers a hasher factory under an algorithm name.
	// Returns the registry itself for chained registration.
	Register(algo string, fn func() hash.Hash) HashRegistry
}

HashRegistry is the registry of hash algorithms. Used by the Pipeline runner for TeeReader at every stage, by the Recovery Agent when parsing TOC blobs and Pack TOCs, and by parsers of "<algo>-<hex>" identifiers.

Lives in domain so that helpers (manifestcodec, future codecs, maintenance agents) can depend on the contract without pulling in core. The default implementation lives in core, constructed via core.NewHashRegistry().

type KDFParams

type KDFParams struct {
	Time    uint32
	Memory  uint32
	Threads uint8
}

KDFParams are the parameters for deriving a KEK.

type LayoutHeader

type LayoutHeader struct {
	BlobStorage string
}

LayoutHeader is a service attribute inside a Manifest that "freezes" the physical-projection rules applied at write time.

type MaintenanceMode

type MaintenanceMode uint8

MaintenanceMode is a maintenance regime orthogonal to StoreState.

const (
	MaintenanceModeNone MaintenanceMode = iota
	MaintenanceModeReadOnly
	MaintenanceModeOffline
)

type Manifest

type Manifest struct {
	// ArtifactID is the in-memory identity of the manifest. It is
	// NOT serialised: per docs/2. Internals/07 §7.4, ArtifactID is
	// computed as the hash of the full file bytes (including the
	// header), so it cannot live inside the body. The field is set
	// at two places only:
	//   - manifestcodec.ComputeArtifactID, after writing the body
	//     and hashing the result;
	//   - core.loadManifest, from the id used to fetch the file.
	// On the wire (manifestcodec) the field is invisible; in the
	// index (sqlite) it is the primary key. See codec_test.go for
	// the "ArtifactID does not appear in JSON" invariant.
	ArtifactID ArtifactID

	Type      ManifestType
	Namespace string
	SessionID string
	CreatedAt time.Time

	ContentHash  ContentHash
	OriginalSize int64
	BlobRef      BlobRef
	LayoutHeader LayoutHeader
	Pipeline     []PipelineStage
	InlineBlob   []byte
	ExternalURI  string

	RetentionUntil time.Time
	KeyID          string

	SystemFlags ManifestSystemFlags
	Metadata    json.RawMessage
}

Manifest is the logical passport of an Artifact.

type ManifestCrypto

type ManifestCrypto string

ManifestCrypto controls manifest protection. Immutable.

const (
	ManifestCryptoPlain        ManifestCrypto = "Plain"
	ManifestCryptoMetadataOnly ManifestCrypto = "MetadataOnly"
	ManifestCryptoEnvelope     ManifestCrypto = "Envelope"
)

type ManifestEncoding

type ManifestEncoding string

ManifestEncoding is the on-disk serialisation format of a manifest.

const (
	ManifestEncodingJSON   ManifestEncoding = "JSON"
	ManifestEncodingBinary ManifestEncoding = "Binary"
)

type ManifestStorage

type ManifestStorage string

ManifestStorage controls where the manifest file lives. Immutable.

const (
	ManifestStorageRemote     ManifestStorage = "Remote"
	ManifestStorageLocal      ManifestStorage = "Local"
	ManifestStorageReplicated ManifestStorage = "Replicated"
)

type ManifestSystemFlags

type ManifestSystemFlags struct {
	TOCOffset int64
	TOCSize   int64
}

ManifestSystemFlags is the system block of a Manifest. Present only for type: "pack".

type ManifestType

type ManifestType string

ManifestType is the role of a Manifest.

const (
	ManifestTypeBlob ManifestType = "blob"
	ManifestTypeTOC  ManifestType = "toc"
	ManifestTypePack ManifestType = "pack"
)

type PackAlignmentPolicy

type PackAlignmentPolicy int

PackAlignmentPolicy is the alignment policy for blobs inside a pack.

const (
	PackAlignmentAuto PackAlignmentPolicy = -1
	PackAlignmentNone PackAlignmentPolicy = 0
	PackAlignment512  PackAlignmentPolicy = 512
	PackAlignment4096 PackAlignmentPolicy = 4096
)

type PackedBlobInfo

type PackedBlobInfo struct {
	PackBlobRef    string
	ManifestOffset int64
	ManifestSize   int64
	BlobOffset     int64
	BlobSize       int64
	PipelineParams []byte
}

PackedBlobInfo is the data needed for a range read of a single packed blob from a .pack volume.

type PackedEntry

type PackedEntry struct {
	ArtifactID     ArtifactID
	BlobRef        string
	ManifestOffset int64
	ManifestSize   int64
	BlobOffset     int64
	BlobSize       int64

	ContentHash    ContentHash
	Namespace      string
	SessionID      string
	PipelineParams []byte
}

PackedEntry describes one entry inside a .pack volume.

type PathTopology

type PathTopology string

PathTopology is the topology of paths inside a Location. Immutable.

const (
	PathTopologyFlat    PathTopology = "Flat"
	PathTopologySharded PathTopology = "Sharded"
	PathTopologyNative  PathTopology = "Native"
)

type PhysicalAddress

type PhysicalAddress struct {
	Workspace Workspace
	Path      string
	PackRef   string
	Offset    int64
	Size      int64
}

PhysicalAddress is the physical address of a blob inside a Store.

type PipelineStage

type PipelineStage struct {
	Algorithm string
	Hash      string
	IV        []byte
}

PipelineStage is a single transformation stage in the Pipeline.

type PutOptions

type PutOptions struct {
	SessionID      string
	Namespace      string
	ExternalURI    string
	BlobType       BlobType
	RetentionUntil time.Time
	Routing        RoutingHints
}

PutOptions is the call context for Store.Put.

type RoutingHints

type RoutingHints struct {
	ContentType string
	Source      string
	Attributes  map[string]string
}

RoutingHints are policy hints for curator.RoutingFunc.

type StorageInfo

type StorageInfo struct {
	TotalBytes     int64
	UsedBytes      int64
	AvailableBytes int64
	ArtifactCount  int64
	BlobCount      int64
}

StorageInfo holds aggregated storage metrics. -1 means unavailable.

type StoreConfig

type StoreConfig struct {
	PathTopology     PathTopology
	ManifestStorage  ManifestStorage
	BlobStorage      BlobStorage
	ManifestEncoding ManifestEncoding
	ManifestCrypto   ManifestCrypto
	PackAlignment    PackAlignmentPolicy
	EagerFetchLimit  int64

	Pipeline            []string
	ContentHasher       ContentHashAlgorithm
	MetadataTransformer string
	VerifyOnRead        VerifyOnReadPolicy

	DeletionPolicy       DeletionPolicy
	DeletionPolicyLock   bool
	RetentionPeriod      time.Duration
	TombstoneGracePeriod time.Duration
	InlineBlobLimit      int64
	GCLeasePolicy        GCLeasePolicy

	KDFParams *KDFParams
}

StoreConfig is the full Store configuration.

type StoreID

type StoreID string

StoreID is the global identifier of a Store. A UUID v4, generated once at InitStore; never changes.

type StoreState

type StoreState string

StoreState is the state of a Store.

const (
	// StateBootstrapping — initialisation, Orphan Scan, or descriptor
	// consensus is in progress. The API is blocked.
	StateBootstrapping StoreState = "Bootstrapping"

	// StateLocked — the descriptor has been read; the DEK has not
	// yet been decrypted. Awaits Unlock with a passphrase.
	StateLocked StoreState = "Locked"

	// StateUnlocked — normal working state. All operations are
	// available unless restricted by MaintenanceMode or configuration
	// policy.
	StateUnlocked StoreState = "Unlocked"

	// StateDegraded — a divergence in descriptor consensus has been
	// detected. The API is available; Auto-Heal will reconcile the
	// divergence and transition the Store to Unlocked.
	StateDegraded StoreState = "Degraded"

	// StateCorrupted — a critical initialisation failure (every
	// descriptor replica is corrupted, or the StoreIndex is
	// corrupted). The API is blocked. Recovery is performed through
	// an explicit RebuildIndexAgent.
	StateCorrupted StoreState = "Corrupted"
)

type VerifyOnReadPolicy

type VerifyOnReadPolicy string

VerifyOnReadPolicy controls explicit ContentHash verification on Get.

const (
	VerifyOnReadAuto         VerifyOnReadPolicy = "Auto"
	VerifyOnReadForceEnabled VerifyOnReadPolicy = "ForceEnabled"
	VerifyOnReadDisabled     VerifyOnReadPolicy = "Disabled"
)

type Workspace

type Workspace uint8

Workspace is the physical placement of a file in the StoreIndex schema. The numeric values are part of the schema format.

const (
	WorkspaceLocation Workspace = 0
	WorkspaceHost     Workspace = 1
)