object

Documentation

Overview

Package object defines the core data types for the blobstore. This package has no I/O and no external dependencies — it is pure data.

Index

• func ValidateNamespaceID(id string) error
• type BlobEntry
• type BlobID
  • func (id BlobID) IsZero() bool
  • func (id BlobID) String() string
• type BlobInfo
• type ChunkEntry
• type ChunkID
  • func NewChunkID(blobID BlobID, seq int) ChunkID
  • func (id ChunkID) String() string
• type EncryptionInfo
• type Metadata
• type Namespace
• type NamespaceStats
• type Quota
• type RefEntry
• type SegmentEntry
• type SegmentID
  • func (id SegmentID) String() string
• type SegmentState
• type StoreStats

Functions

func ValidateNamespaceID

func ValidateNamespaceID(id string) error

ValidateNamespaceID returns an error if id is not a valid namespace identifier.

Types

type BlobEntry

type BlobEntry struct {
	BlobID    BlobID    `json:"blob_id"`
	ChunkIDs  []ChunkID `json:"chunk_ids"` // ordered; reassemble in this order
	TotalSize int64     `json:"total_size"`
	RefCount  int64     `json:"ref_count"`
	CreatedAt time.Time `json:"created_at"`
}

BlobEntry is the index record for a content-addressed blob. Shared across all namespaces that reference the same content. RefCount tracks how many RefEntries point here; zero means GC-eligible.

type BlobID

type BlobID string

BlobID is a content-addressed identifier: "sha256:<hex>". Two blobs with identical bytes always produce the same BlobID. This is the foundation of deduplication and idempotent replication.

func (BlobID) IsZero

func (id BlobID) IsZero() bool

func (BlobID) String

func (id BlobID) String() string

type BlobInfo

type BlobInfo struct {
	Key         string   `json:"key"`
	NamespaceID string   `json:"namespace_id"`
	Metadata    Metadata `json:"metadata"`
}

BlobInfo is what callers see — a key plus its resolved metadata.

type ChunkEntry

type ChunkEntry struct {
	ChunkID     ChunkID   `json:"chunk_id"`
	BlobID      BlobID    `json:"blob_id"`
	SegmentID   SegmentID `json:"segment_id"`
	NamespaceID string    `json:"namespace_id"`
	PageOffset  int64     `json:"page_offset"` // byte offset of the page in the segment file
	PageCount   int       `json:"page_count"`  // number of pages this chunk occupies
	Length      int64     `json:"length"`      // payload byte length (excludes page headers/padding)
	Seq         int       `json:"seq"`         // 0-based position within the blob
	RefCount    int64     `json:"ref_count"`   // blobs referencing this content; zero means GC-eligible

	// Nonce and Tag are set only when this chunk's namespace has
	// encryption enabled (see Namespace.Encryption); both are nil for
	// every chunk in an unencrypted namespace. Nonce is the 12-byte GCM
	// nonce and Tag the 16-byte detached GCM authentication tag used by
	// package encryption to encrypt/decrypt this chunk's payload. The
	// payload bytes on disk are ciphertext of exactly Length bytes —
	// nonce and tag are carried here, in the index, rather than inline
	// in the page, specifically so the volume/segment format and
	// compaction's byte-level segment rewrite never need to change or
	// even be aware encryption exists.
	Nonce []byte `json:"nonce,omitempty"`
	Tag   []byte `json:"tag,omitempty"`
}

ChunkEntry is the index record for a chunk's physical location on disk.

type ChunkID

type ChunkID string

ChunkID uniquely identifies a chunk within the volume engine.

Production chunk IDs are content-addressed: "sha256:<hex>" over the chunk's own bytes (see volume.chunkIDFromContent), so identical runs of bytes in different blobs share one ChunkID — the foundation of cross-blob deduplication. NewChunkID below remains available as a legacy/utility constructor for the older "<blobID>#<seq>" format.

func NewChunkID

func NewChunkID(blobID BlobID, seq int) ChunkID

NewChunkID constructs a ChunkID from a BlobID and 0-based sequence number. Format: "<blobID>#<decimal_seq>" using variable-width decimal.

Panics if:

• seq < 0 or seq > maxChunkSeq: would require more than 4 billion chunks.
• len(blobID) != canonicalBlobIDLen: a non-canonical BlobID would overflow internal scratch buffers and silently truncate the sequence field, causing two different chunk sequences to share the same ChunkID (index aliasing / silent data corruption).

func (ChunkID) String

func (id ChunkID) String() string

type EncryptionInfo

type EncryptionInfo struct {
	// Enabled is always true when Namespace.Encryption is non-nil; kept
	// as an explicit field (rather than relying on the pointer alone)
	// so a future schema revision can distinguish "encryption turned
	// off" from "record predates this field" without an ambiguous zero
	// value.
	Enabled bool `json:"enabled"`

	// WrappedDEK is this namespace's DEK, encrypted under a master key
	// (see package encryption's WrapKey/UnwrapKey). The store never
	// persists a DEK in the clear.
	WrappedDEK []byte `json:"wrapped_dek"`

	// KeyVersion identifies which master key WrappedDEK was wrapped
	// under. Opaque to this package — it exists purely so a caller
	// holding multiple master keys (e.g. across a rotation) can look up
	// the right one via encryption.KeyProvider.Key(KeyVersion).
	KeyVersion string `json:"key_version,omitempty"`
}

EncryptionInfo records that a namespace has encryption-at-rest enabled and holds what's needed to recover its data-encryption key (DEK). The DEK itself is never stored — only WrappedDEK, the DEK encrypted (envelope encryption) under a caller-supplied master key.

type Metadata

type Metadata struct {
	ContentType string            `json:"content_type,omitempty"` // MIME type; default "application/octet-stream"
	Size        int64             `json:"size"`                   // total logical blob size in bytes
	BlobID      BlobID            `json:"blob_id"`
	ChunkCount  int               `json:"chunk_count"`
	CreatedAt   time.Time         `json:"created_at"`
	UpdatedAt   time.Time         `json:"updated_at"`
	Custom      map[string]string `json:"custom,omitempty"` // caller-defined; keys starting "_bs_" are reserved
}

Metadata holds all descriptive information about a stored blob. Stored in the index alongside the ref — never inside the volume. This means metadata can evolve without rewriting blob bytes.

type Namespace

type Namespace struct {
	ID          string            `json:"id"`
	DisplayName string            `json:"display_name,omitempty"`
	CreatedAt   time.Time         `json:"created_at"`
	Quota       *Quota            `json:"quota,omitempty"` // nil = unlimited
	Custom      map[string]string `json:"custom,omitempty"`

	// Encryption describes this namespace's encryption-at-rest state.
	// Nil (the default) means the namespace is unencrypted. This is
	// decided once, at namespace creation, and is not meant to change
	// afterward — see package encryption's doc comment for why turning
	// it on later does not retroactively encrypt already-written data.
	Encryption *EncryptionInfo `json:"encryption,omitempty"`
}

Namespace is an isolated logical partition within a Store.

type NamespaceStats

type NamespaceStats struct {
	NamespaceID   string    `json:"namespace_id"`
	BlobCount     int64     `json:"blob_count"`     // live refs
	BytesStored   int64     `json:"bytes_stored"`   // logical bytes (sum of blob sizes)
	BytesPhysical int64     `json:"bytes_physical"` // actual bytes on disk (after dedup)
	ChunkCount    int64     `json:"chunk_count"`    // live chunks
	DeadBytes     int64     `json:"dead_bytes"`     // unreferenced bytes awaiting compaction
	DeadChunks    int64     `json:"dead_chunks"`    // unreferenced chunks awaiting compaction
	SegmentCount  int64     `json:"segment_count"`
	UpdatedAt     time.Time `json:"updated_at"`
}

NamespaceStats holds live usage metrics for a single namespace. Maintained incrementally on every write/delete — never computed by scanning.

type Quota

type Quota struct {
	MaxBytes     int64 `json:"max_bytes,omitempty"`
	MaxBlobCount int64 `json:"max_blob_count,omitempty"`
	MaxBlobSize  int64 `json:"max_blob_size,omitempty"`
}

Quota defines resource limits for a namespace. Zero values mean unlimited for that dimension.

type RefEntry

type RefEntry struct {
	NamespaceID string   `json:"namespace_id"`
	Key         string   `json:"key"`
	BlobID      BlobID   `json:"blob_id"`
	Metadata    Metadata `json:"metadata"`
}

RefEntry is the index record for a (namespaceID, key) → BlobID mapping. It is the only mutable layer — the blob it points at is immutable.

type SegmentEntry

type SegmentEntry struct {
	SegmentID   SegmentID    `json:"segment_id"`
	NamespaceID string       `json:"namespace_id"`
	State       SegmentState `json:"state"`
	PageSize    int          `json:"page_size"`
	PageCount   int64        `json:"page_count"`
	BytesUsed   int64        `json:"bytes_used"`  // live payload bytes
	BytesTotal  int64        `json:"bytes_total"` // total file size
	CreatedAt   time.Time    `json:"created_at"`
	SealedAt    *time.Time   `json:"sealed_at,omitempty"`
}

SegmentEntry is the index record for a segment file.

type SegmentID

type SegmentID uint64

SegmentID identifies a segment file. It is a monotonically increasing uint64 formatted as a zero-padded 16-char hex string.

func (SegmentID) String

func (id SegmentID) String() string

type SegmentState

type SegmentState int

SegmentState describes the lifecycle of a segment file.

const (
	SegmentActive     SegmentState = iota // currently being written to
	SegmentSealed                         // full and read-only
	SegmentCompacting                     // being rewritten; read-only
	SegmentDead                           // all live data migrated; awaiting deletion
)

type StoreStats

type StoreStats struct {
	NamespaceCount     int64            `json:"namespace_count"`
	TotalBlobCount     int64            `json:"total_blob_count"`
	TotalBytesStored   int64            `json:"total_bytes_stored"`
	TotalBytesPhysical int64            `json:"total_bytes_physical"`
	TotalDeadBytes     int64            `json:"total_dead_bytes"`
	SegmentCount       int64            `json:"segment_count"`
	DeduplicationRatio float64          `json:"deduplication_ratio"` // >1 means dedup is saving space
	PerNamespace       []NamespaceStats `json:"per_namespace"`
	ComputedAt         time.Time        `json:"computed_at"`
}

StoreStats aggregates metrics across all namespaces.