Help us shape the future of Go! Take the 2026 Go Developer Survey

pstore

package
v0.0.0-...-8e407e5 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Oct 22, 2026 License: Apache-2.0 Imports: 41 Imported by: 0

Documentation

Overview

Package pstore ("pelican store") uses the cache V2 block store as an origin's primary data store rather than as a cache.

The block store underneath local_cache — encrypted 4080-byte blocks, multi-directory spreading, chunking for large objects, and a BadgerDB catalog that is itself encrypted at rest — is a general-purpose object store that happens to be used by a cache. pstore drives it directly, adds the one thing a cache never needed (a directory index), and exposes the result as an afero.Fs so it drops into the origin's existing serving stack.

Objects are immutable: Pelican has no mid-file edits, so a write always creates a complete new version and atomically swaps it in. There is no eviction; when the store fills, writes fail with ErrNoSpace until something is deleted.

See docs/pstore-design.md for the full design.

Index

Constants

View Source 
const FsckNoGracePeriod = time.Duration(-1)

FsckNoGracePeriod disables the in-flight-write grace period.

Only safe when nothing can be writing to the store -- a test, or an offline pass an operator has taken responsibility for.

Variables

View Source 
var (
	// ErrNotExist is returned when a path has no entry.
	ErrNotExist = fs.ErrNotExist
	// ErrExist is returned when a path already has an entry and the caller
	// required that it not.
	ErrExist = fs.ErrExist
	// ErrNotDir is returned when a path component is not a directory.
	ErrNotDir = syscall.ENOTDIR
	// ErrIsDir is returned when a directory is used where a file is required.
	ErrIsDir = syscall.EISDIR
	// ErrNotEmpty is returned when a non-recursive removal targets a
	// directory that still has entries.
	ErrNotEmpty = syscall.ENOTEMPTY
	// ErrNoSpace is returned when a write would exceed the store's capacity.
	//
	// golang.org/x/net/webdav reports any failed write as 405 Method Not
	// Allowed, so the origin rewrites the status itself: see
	// origin_serve.statusForBackendError, which turns this into 507
	// Insufficient Storage.
	ErrNoSpace = syscall.ENOSPC
	// ErrInvalidPath is returned for paths the index cannot represent.
	ErrInvalidPath = fs.ErrInvalid
	// ErrNotSupported is returned for operations pstore deliberately does not
	// implement, such as writing at an arbitrary offset.
	ErrNotSupported = errors.New("operation not supported by pstore")
	// ErrDraining is returned when a path is inside a subtree that a
	// recursive delete has removed but the janitor has not finished
	// reclaiming.  Retrying shortly succeeds; the janitor speeds up while a
	// backlog exists.  origin_serve.statusForBackendError reports it as 409
	// Conflict, which is what tells a client the retry is worth making.
	ErrDraining = errors.New("path is still being reclaimed")
	// ErrPreconditionFailed is returned when a conditional write's
	// precondition does not hold -- that is, when Close finds the object no
	// longer carries the generation the caller required.
	// origin_serve.statusForBackendError reports it as 412 Precondition
	// Failed.
	ErrPreconditionFailed = errors.New("precondition failed")
	// ErrConflict is returned when two writers raced to install the same path
	// and this one's index transaction lost.
	//
	// The store's index is serializable, so a commit that reads an entry
	// another commit is replacing is aborted rather than serialized behind it.
	// Nothing is lost -- the winner's version is installed and the loser's is
	// discarded -- and retrying succeeds.  The WebDAV lock system serializes
	// same-path PUTs before they reach here, so a client normally sees 423
	// Locked instead; this is what a caller using the store directly gets.
	// origin_serve.statusForBackendError reports it as 409 Conflict rather
	// than letting an opaque database error surface as a 500.
	ErrConflict = badger.ErrConflict
	// ErrClosed is returned when a handle is used after Close.
	ErrClosed = fs.ErrClosed
)

Sentinel errors. These alias the syscall/fs values the afero and WebDAV layers already understand, so that errors.Is(err, fs.ErrNotExist), os.IsNotExist, and the handler's status mapping all work without translation.

Functions

func DeriveBackupKey 

func DeriveBackupKey(issuerKey jwk.Key) ([]byte, error)

DeriveBackupKey returns the level-2 pstore backup private key for one issuer key, which is what `metadata-backup-key` prints for escrow.

The same issuer key always yields the same backup key, so printing it twice gives the same answer -- that is the property that makes escrowing it useful. It opens every snapshot this origin writes; for the key that opens exactly one of them, see DeriveFileKey.

func DeriveFileKey 

func DeriveFileKey(r io.Reader, keys BackupKeys) ([]byte, error)

DeriveFileKey returns the level-3 key that opens exactly the archive in r, and nothing else.

It reads only the header, so r may be a file opened purely for this. keys supplies whatever the caller has: the origin's issuer keys, or an escrowed backup key -- either reaches the same answer, because the archive carries its own backup key sealed to both.

func FormatBackupKey 

func FormatBackupKey(key []byte) string

FormatBackupKey renders a raw backup or per-file key the way a key file holds it.

func ParseBackupKey 

func ParseBackupKey(encoded string) ([]byte, error)

ParseBackupKey decodes a base64 backup or per-file key and checks its length.

Types

type BackupConfig 

type BackupConfig struct {
	// Dir receives the snapshot files.  Empty disables backups.
	Dir string
	// Interval between snapshots.
	Interval time.Duration
	// Keep bounds how many snapshots are retained; zero keeps them all.
	Keep int
	// Keys seal each snapshot.  They are supplied by the caller rather than
	// read from configuration here, so this package stays independent of
	// config and a test can seal to keys of its own.  Backups do not start
	// without them: a snapshot is a logical dump and is never written in the
	// clear.
	Keys BackupKeys
}

BackupConfig describes the periodic metadata backup.

type BackupKeys 

type BackupKeys struct {
	// IssuerKeys are the origin's current issuer keys, by key ID (level 1).
	// The archive's backup key is sealed to the pstore-domain key pair derived
	// from each, and any one of them opens it -- so a snapshot written before a
	// rotation still opens afterwards, as long as one of the keys it was
	// written under survives.
	IssuerKeys map[string]jwk.Key

	// BackupKey is a raw 32-byte Curve25519 private key from level 2: the
	// escrowed key printed by `pelican-server origin pstore metadata-backup-key`, or
	// an independent key for backups that must be restorable somewhere that
	// will never hold this origin's issuer keys.
	//
	// When writing, it *replaces* the issuer-derived recipients rather than
	// joining them.  When reading, it opens the envelope and yields the
	// archive's own backup key, from which the per-file key is derived.
	BackupKey []byte

	// FileKey is one archive's own key (level 3), as printed by
	// `metadata-backup-key <file>`.  It decrypts that archive's body directly:
	// no derivation, no envelope, and no issuer key need be present.  It opens
	// nothing else, which is the point of having it.
	//
	// It is read-only.  Sealing a new archive requires a backup key to embed,
	// and a per-file key is downstream of one.
	FileKey []byte
}

BackupKeys names the keys a snapshot is sealed to, or opened with.

The three fields are the three levels of the hierarchy, and on the reading side any one of them is enough. They are passed in rather than read from configuration so that pstore stays free of the config package: the caller resolves the origin's issuer keys and hands them over, and a test supplies whatever keys it likes.

func (BackupKeys) Empty 

func (k BackupKeys) Empty() bool

Empty reports whether there is nothing here to *seal* to.

FileKey is deliberately not consulted: it opens one existing archive and cannot write a new one, so a caller holding only that has nothing to back up with.

type Config 

type Config struct {
	// BaseDir holds the BadgerDB catalog.  It is also the first storage
	// directory when StorageDirs is empty.
	BaseDir string

	// StorageDirs are the directories object data is spread across.  When
	// empty, BaseDir is used as the sole storage directory.
	StorageDirs []local_cache.StorageDirConfig

	// InlineMaxBytes is the largest object stored directly in the catalog
	// rather than on disk.  Zero selects local_cache.InlineThreshold.
	InlineMaxBytes int

	// NamespaceLabel names the store in the catalog's usage counters.
	//
	// A store is one accounting unit spanning however many exports are mapped
	// onto subtrees of it, because capacity is enforced store-wide (§7).
	// Per-export accounting arrives with path-based quotas, where the lot
	// tree is the natural place for it.  Empty selects the store root.
	NamespaceLabel string
}

Config describes a store to open.

type Dirent 

type Dirent struct {
	Type       EntryType `msgpack:"t"`
	Generation string    `msgpack:"g,omitempty"` // files only; also the ETag
	Size       int64     `msgpack:"s,omitempty"`
	MTimeNanos int64     `msgpack:"m"`
	Mode       uint32    `msgpack:"o"`
}

Dirent is one entry in a directory.

Size, MTimeNanos, and Generation are denormalized from the object's CacheMetadata so that a listing is a single scan: without them, a PROPFIND over a large directory would need one additional point lookup per child. They are rewritten only at commit, in a transaction the write path performs anyway.

func (*Dirent) IsDir 

func (d *Dirent) IsDir() bool

IsDir reports whether the entry is a directory.

type EntryType 

type EntryType uint8

EntryType distinguishes files from directories in a dirent. 

const (
	// EntryFile is a regular object.
	EntryFile EntryType = 0
	// EntryDir is a directory.
	EntryDir EntryType = 1
)

type ErrNoMatchingKey 

type ErrNoMatchingKey struct {
	SealedKeyIDs []string
}

ErrNoMatchingKey reports a snapshot that none of the supplied keys open.

SealedKeyIDs names the keys it *was* sealed to, which is the only thing that tells an operator holding the wrong keyring what to go and find.

func (*ErrNoMatchingKey) Error 

func (e *ErrNoMatchingKey) Error() string

type ExportOptions 

type ExportOptions struct {
	// Root is the subtree to export; empty means the whole store.
	Root string
	// Dest is the directory to write into.  It is created if missing.
	Dest string
	// Verify checks each object against its recorded checksums as it is
	// written, so a recovery does not quietly reproduce corrupt data.
	Verify bool
	// DryRun reports what would be written without writing it.
	DryRun bool
	// Resume skips objects already present in the destination instead of
	// reporting them.
	//
	// An export of a large store is interrupted sooner or later -- a full
	// disk, a lost session, an operator's Ctrl-C -- and the natural response
	// is to run it again.  Without this the second run reports every file the
	// first one wrote as a failure, which is indistinguishable from real
	// corruption at exactly the moment an operator most needs to tell the
	// difference.
	Resume bool
}

ExportOptions controls a recovery export.

type ExportReport 

type ExportReport struct {
	// FilesWritten is how many objects were exported.
	FilesWritten int
	// DirsCreated is how many directories were created.
	DirsCreated int
	// BytesWritten totals the object content exported.
	BytesWritten int64
	// Failed lists objects that could not be exported, with the reason.
	// A recovery continues past a bad object rather than abandoning the rest.
	Failed map[string]string
	// AlreadyPresent counts objects skipped because the destination already
	// held a file of that name.
	//
	// Kept apart from Failed on purpose.  "The previous run already wrote
	// this" and "this object cannot be read" call for opposite responses, and
	// a report that conflates them turns a resumed export into a false
	// corruption report.
	AlreadyPresent int
}

ExportReport summarizes a recovery export.

type FS 

type FS struct {
	// contains filtered or unexported fields
}

FS adapts a Store to afero.Fs.

func NewFS 

func NewFS(s *Store) *FS

NewFS returns an afero.Fs backed by the store.

func (*FS) Chmod 

func (fs *FS) Chmod(name string, _ os.FileMode) error

Chmod is not supported: pstore stores no per-entry permissions.

func (*FS) Chown 

func (fs *FS) Chown(name string, _, _ int) error

Chown is not supported: pstore stores no per-entry ownership.

func (*FS) Chtimes 

func (fs *FS) Chtimes(name string, _, _ time.Time) error

Chtimes is not supported: an object's mtime is set when its version is committed.

func (*FS) Create 

func (fs *FS) Create(name string) (afero.File, error)

Create makes (or replaces) an object and returns it open for writing, creating any missing parent directories.

Auto-creating parents is what lets a client PUT into a prefix that does not exist yet, which is how uploads normally arrive -- no explicit MKCOL. The origin has a shared afero wrapper that does this, but it derives the parent with filepath, and a pstore path is a namespace path that is always slash-separated: on Windows the wrapper would ask for a directory literally named "\a\b\c". Doing it here keeps the separator right on every platform.

func (*FS) Mkdir 

func (fs *FS) Mkdir(name string, _ os.FileMode) error

Mkdir creates a directory. perm is ignored: pstore has no per-entry permissions, since access control is the origin's authorization layer.

func (*FS) MkdirAll 

func (fs *FS) MkdirAll(path string, _ os.FileMode) error

MkdirAll creates a directory and any missing ancestors.

func (*FS) Name 

func (fs *FS) Name() string

Name identifies the filesystem in afero diagnostics.

func (*FS) Open 

func (fs *FS) Open(name string) (afero.File, error)

Open opens an object or directory for reading.

func (*FS) OpenFile 

func (fs *FS) OpenFile(name string, flag int, _ os.FileMode) (afero.File, error)

OpenFile opens a path with the given flags.

Objects are immutable, so the write flags all mean the same thing: build a new version and swap it in at Close. O_APPEND is refused rather than silently truncating, since there is no way to honor it.

func (*FS) OpenFileConditional 

func (fs *FS) OpenFileConditional(name string, flag int, perm os.FileMode, size int64,
	requireAbsent bool, requireETag string) (afero.File, error)

OpenFileConditional is OpenFileSized with an HTTP conditional-write precondition attached, implementing origin_serve's ConditionalWriteFs.

The precondition is enforced by Close, inside the single transaction that resolves the existing entry and installs the new one, which is the only place it can be enforced without a race. Checking it here, at open time, would be exactly the stat-then-write the caller is trying to avoid: two concurrent "If-None-Match: *" PUTs would both find the path absent and both commit.

requireETag carries the client's entity tag verbatim; an object's tag is its generation, so the tag is unwrapped rather than compared as a string.

func (*FS) OpenFileSized 

func (fs *FS) OpenFileSized(name string, flag int, perm os.FileMode, size int64) (afero.File, error)

OpenFileSized is OpenFile with the object's expected length, which lets the write path pick its storage tier immediately instead of buffering to discover it. It implements origin_serve's size-hint interface.

The size is advisory: the object is stored at whatever length actually arrives, so a wrong or absent hint costs nothing but the buffering it would have avoided.

func (*FS) Remove 

func (fs *FS) Remove(name string) error

Remove deletes an object, or an empty directory.

func (*FS) RemoveAll 

func (fs *FS) RemoveAll(path string) error

RemoveAll deletes a path and everything beneath it.

func (*FS) Rename 

func (fs *FS) Rename(oldname, newname string) error

Rename moves an entry.

func (*FS) Stat 

func (fs *FS) Stat(name string) (os.FileInfo, error)

Stat returns file information for a path.

type FsckOptions 

type FsckOptions struct {
	// Repair acts on what the pass finds rather than only reporting it.
	Repair bool
	// Deep additionally reads every object back and verifies it against the
	// checksums recorded at ingest.
	//
	// This is the only check that detects at-rest corruption -- a flipped bit
	// in a block file, or a truncated chunk -- because everything else reads
	// only the catalog.  It costs a full pass over the data, so it is off by
	// default and belongs on a schedule rather than in a routine check.
	Deep bool

	// SkipOrphanScan omits the search for object versions no entry refers to.
	//
	// That search is the only part of a pass whose cost is not streaming: it
	// must hold the set of referenced versions while it scans the catalog.
	// Everything else -- unreadable entries, interrupted writes, counter
	// drift -- is a streaming check.  A frequent background pass therefore
	// skips it, or covers one slice at a time; an operator running fsck by
	// hand wants the whole answer and gets it.
	SkipOrphanScan bool

	// OrphanShardOf, when greater than one, splits the orphan comparison into
	// that many slices of the instance-hash space and examines only
	// OrphanShard.  Rotating the slice across successive passes gives full
	// coverage over time at a fraction of the memory.
	//
	// It is rounded up to a power of sixteen, because the slices are hex
	// prefixes of the instance hash.  Rotate over FsckReport.OrphanScanShards
	// rather than over this value: they differ whenever it is not already a
	// power of sixteen, and rotating over the smaller one would leave most of
	// the store permanently unexamined.
	OrphanShardOf int
	// OrphanShard selects the slice, modulo the effective slice count.
	OrphanShard int

	// MinAge is how long an unreferenced version must have been complete
	// before it counts as an orphan rather than a write still in flight.
	//
	// Zero selects fsckDefaultMinAge.  The zero value is the safe one on
	// purpose: a caller that forgets this field gets the grace period, not
	// the hazard it exists to prevent.  Pass FsckNoGracePeriod to switch it
	// off, which only a test with no concurrent writers should do.
	MinAge time.Duration
}

FsckOptions selects what a consistency pass checks.

type FsckReport 

type FsckReport struct {
	// EntriesScanned is how many index entries were examined.
	EntriesScanned int
	// DirectoriesScanned is how many of those were directories.
	DirectoriesScanned int

	// DanglingEntries are files whose object version has no metadata.  The
	// entry is unreadable, so it is reported and, when repairing, removed.
	DanglingEntries []string
	// MissingData are objects whose metadata exists but whose blocks are
	// incomplete, meaning a write was interrupted before it committed.
	MissingData []string
	// OrphanedInstances are object versions no entry refers to, which have
	// been complete for longer than the grace period.  They hold space and
	// are queued for reclamation when repairing.
	OrphanedInstances []string
	// PendingInstances are object versions no entry refers to *yet*: a write
	// that has not completed, or one that completed within the grace period.
	//
	// These are reported for visibility and never repaired.  A live upload
	// looks exactly like a crash-window orphan from the catalog's point of
	// view, and reclaiming one would delete an object out from under the
	// client still writing it.  Reclamation of writes that really were
	// abandoned belongs to the append-intent sweep, which can tell the
	// difference.
	PendingInstances []string
	// CorruptObjects are objects whose stored data no longer matches the
	// checksums recorded when they were written.  Only populated by a deep
	// pass.
	CorruptObjects []string
	// ObjectsVerified counts the objects a deep pass read back.
	ObjectsVerified int

	// UsageDrift is the difference, per storage directory, between the
	// catalog's recorded usage and what the objects actually occupy.
	UsageDrift map[local_cache.StorageID]int64

	// OrphanScanPartial reports that the orphan comparison covered only a
	// slice of the instance-hash space, so its findings are not exhaustive.
	OrphanScanPartial bool
	// OrphanScanShards is how many slices the space was divided into, which
	// is the number of passes a caller must rotate through for full coverage.
	//
	// It can exceed what the caller asked for: the slices are hex prefixes of
	// the instance hash, so a request is rounded up to a power of sixteen.  A
	// caller that rotates over its own number instead of this one would leave
	// most of the store permanently unexamined.
	OrphanScanShards int

	// Repaired reports whether a repair pass ran.  It says nothing about
	// whether every finding was resolved -- see Resolved and Unresolved.
	Repaired bool
	// UsageResynced reports that the drift above was written back to the
	// catalog, not merely recomputed in memory.
	UsageResynced bool
}

FsckReport summarizes a consistency pass.

func (*FsckReport) Healthy 

func (r *FsckReport) Healthy() bool

Healthy reports whether the pass found nothing wrong.

Pending instances are deliberately excluded: a write in flight is what a busy origin looks like, not a fault.

func (*FsckReport) Resolved 

func (r *FsckReport) Resolved() []string

Resolved lists what a repair pass actually acted on.

This exists because the two lists are not the same and reporting them as though they were is how an operator comes to believe a store was fixed. Repair can unlink an unreadable entry, queue an unreferenced version, and rewrite a drifted counter. It cannot manufacture data that is gone, and it cannot un-corrupt a block.

func (*FsckReport) Unresolved 

func (r *FsckReport) Unresolved() []string

Unresolved lists findings that survive a repair pass, so a caller can exit non-zero rather than reporting success.

type GCStats 

type GCStats struct {
	// InstancesFreed is the number of object versions whose blocks were
	// released.
	InstancesFreed int
	// InstancesPinned is the number skipped because a reader still holds
	// them; they are retried on a later pass.
	InstancesPinned int
	// SubtreeEntriesRemoved is the number of directory entries removed from
	// detached subtrees.
	SubtreeEntriesRemoved int
	// SubtreesCompleted is the number of detached subtrees fully drained.
	SubtreesCompleted int
	// AbandonedAppendsReclaimed is the number of streaming writes whose
	// writer never finished and whose space has now been released.
	AbandonedAppendsReclaimed int
	// BytesFreed is how much space the released object versions occupied,
	// summed from each version's metadata before it was deleted.  It excludes
	// abandoned streaming writes, whose footprint the block store refunds
	// directly without reporting it.
	BytesFreed int64

	// HitBatchLimit reports that the queue had more work than one sweep
	// handles, so another should follow sooner than the resting period.
	HitBatchLimit bool
}

GCStats reports what one sweep accomplished.

type IntegrityConfig 

type IntegrityConfig struct {
	// IndexInterval is how often the catalog-only check runs.  Zero disables
	// it.
	IndexInterval time.Duration
	// DataInterval is how often every object is read back and verified.
	// Zero disables it.
	DataInterval time.Duration
	// DataScanBytesPerSec caps the verification read rate so the scan does
	// not compete with serving.  Zero selects the block store's default.
	DataScanBytesPerSec int64
}

IntegrityConfig schedules the background checks.

type ListEntry 

type ListEntry struct {
	Name   string
	Dirent *Dirent
}

ListEntry pairs a child name with its dirent during a listing.

type ReadHandle 

type ReadHandle struct {
	*local_cache.ObjectReader
	// contains filtered or unexported fields
}

ReadHandle streams an object version.

The underlying ObjectReader pins the version for its own lifetime, so the janitor cannot reclaim it mid-read; nothing extra is needed here.

func (*ReadHandle) Close 

func (r *ReadHandle) Close() error

Close releases the reader, and with it the pin it holds.

func (*ReadHandle) Dirent 

func (r *ReadHandle) Dirent() *Dirent

Dirent returns the entry the handle was opened from.

type Store 

type Store struct {
	// contains filtered or unexported fields
}

Store is the object store backing a pstore origin.

It owns the catalog and the block store, and is safe for concurrent use. A store has exclusive ownership of its directories for its lifetime, exactly as the cache does.

func Open 

func Open(ctx context.Context, egrp *errgroup.Group, cfg Config) (*Store, error)

Open initializes a store, creating it if the directory is empty.

Requires issuer keys to be available (config.GetIssuerPublicJWKS or the test helper), because the catalog and every object's data key are encrypted under a master key derived from them.

func OpenMaintenance 

func OpenMaintenance(ctx context.Context, baseDir string, allowWrites bool) (*Store, error)

OpenMaintenance opens a stopped store for inspection or repair.

When allowWrites is false every mutating entry point refuses, so an inspection command cannot modify the store even by mistake.

Requires issuer keys, since the catalog is encrypted under a key derived from them. The caller must Close the result.

func (*Store) AccessTime 

func (s *Store) AccessTime(name string) (time.Time, error)

AccessTime returns when an object was last read, as maintained by the LRU index (§8).

This is deliberately not surfaced through FileInfo: it lives on the object's metadata rather than its directory entry, so reporting it from Stat would add a metadata read to every lookup to answer a question almost no caller asks. Operators get it from `pelican-server origin pstore stat`.

A zero time means the object has not been read since it was written, or was read within the debounce window before any access was recorded.

func (*Store) BackupEncrypted 

func (s *Store) BackupEncrypted(w io.Writer, keys BackupKeys) (uint64, error)

BackupEncrypted writes a snapshot sealed to keys.

This is the only way to write a snapshot. Encryption is not optional and there is no key to forget to configure: the archive's backup key is derived from the origin's issuer keys unless one is supplied outright, and a caller that can produce neither gets an error rather than a cleartext dump of the namespace.

**What ends up in the file** is the compressed catalog encrypted under a key belonging to this snapshot alone, preceded by a header that carries the salt that key was derived from *and the archive's backup key sealed once per recipient* -- one recipient per issuer key in keys.IssuerKeys, or the single supplied keys.BackupKey when there is one. Embedding the sealed backup key is what makes the file self-opening: any of those issuer keys reaches it, and so does the escrowed backup key on a machine that has never held them. An operator who escrowed the backup key can restore this file after the issuer keys are gone. See backup_crypto.go for the derivation and the envelope.

Truncation and tampering are detected by the AEAD construction itself, so there is no second framing layer inside the sealed one.

func (*Store) Close 

func (s *Store) Close() error

Close releases the store's resources.

Safe to call more than once: the origin closes a store both explicitly and from the shutdown goroutine watching the server context, and the block store's TTL caches do not tolerate being stopped twice.

func (*Store) Create 

func (s *Store) Create(name string) (*WriteHandle, error)

Create begins a new version of the object at name. The parent directory must exist. Nothing changes on disk until Close.

func (*Store) CreateSized 

func (s *Store) CreateSized(name string, size int64) (*WriteHandle, error)

CreateSized is Create for a caller that already knows the object's length.

The size only selects the storage tier up front; it is a hint, not a promise, and the object is stored at whatever length actually arrives. A declared size at or above the spill threshold goes straight to the streaming writer, so a large upload never buffers a megabyte first.

A negative size means unknown and behaves exactly like Create.

Declaring the size also sizes the streaming chunk, which is what lets a bounded store accept an object a little over the spill threshold: an undeclared write has to assume defaultStreamChunkSize.

func (*Store) DB 

func (s *Store) DB() *local_cache.CacheDB

DB exposes the catalog, for introspection and tests.

func (*Store) Digests 

func (s *Store) Digests(name string) ([]local_cache.Checksum, error)

Digests returns the checksums recorded for an object when it was written.

An object written before checksums were recorded yields an empty slice rather than an error, so callers can fall back rather than fail.

func (*Store) Export 

func (s *Store) Export(ctx context.Context, opts ExportOptions) (*ExportReport, error)

Export writes a subtree out as ordinary files.

It continues past objects it cannot read, collecting them in the report: on a damaged store, recovering most of the data and being told exactly what was lost is far more useful than stopping at the first bad block.

func (*Store) Fsck 

func (s *Store) Fsck(ctx context.Context, repair bool) (*FsckReport, error)

Fsck checks the store's internal consistency.

When repair is false nothing is modified, which is the mode an operator should reach for first. When true, dangling entries are removed, orphaned versions are queued for reclamation, and usage counters are resynced from the catalog.

func (*Store) FsckWith 

func (s *Store) FsckWith(ctx context.Context, opts FsckOptions) (*FsckReport, error)

FsckWith checks the store's internal consistency with explicit options.

func (*Store) HasCapacityFor 

func (s *Store) HasCapacityFor(contentLen int64) error

HasCapacityFor reports whether a write of contentLen content bytes could currently be accepted.

It models exactly what the write path will reserve -- the per-block overhead, the chunk rounding a streamed object pays, and the per-directory placement unit -- via plannedFootprint. Answering a different question here than the write path asks would be worse than not checking at all: the origin would accept a PUT it is about to fail with a 507 halfway through the body, or refuse one that would have fit.

It remains advisory in one respect: a concurrent write may consume the headroom before this caller reserves it, so the write path enforces the limit independently. Its purpose is to let the origin refuse a PUT before the client streams a body that cannot land.

func (*Store) List 

func (s *Store) List(name string, after string, limit int) ([]ListEntry, bool, error)

List returns the direct children of a directory in name order.

after resumes a previous listing at the first name strictly greater than it; limit bounds the result (0 means unbounded). The bool reports whether more entries remain.

func (*Store) Mkdir 

func (s *Store) Mkdir(name string) error

Mkdir creates a single directory. The parent must already exist.

func (*Store) MkdirAll 

func (s *Store) MkdirAll(name string) error

MkdirAll creates a directory and any missing ancestors. It succeeds when the directory already exists.

func (*Store) OpenRead 

func (s *Store) OpenRead(name string) (*ReadHandle, error)

OpenRead opens the current version of an object for reading.

Resolving the path and opening the version are two steps, and an overwrite can land between them: the swap commits, the superseded version is queued, the janitor finds it unpinned, and the open then fails on a generation that was current a microsecond earlier. That surfaces as a 500 on an object that existed the whole time.

Two things close that. The pin is taken inside the resolving transaction, so a janitor that can see the version is superseded can also see that somebody holds it — queuedInstance.claim refuses pinned versions. And the open is retried against a freshly resolved generation, which covers the residual window between the read snapshot and the pin actually landing. Retrying is safe because it is idempotent and bounded, and it terminates: either the path resolves to a version that opens, or it stops changing.

func (*Store) ReadAll 

func (s *Store) ReadAll(name string) ([]byte, error)

ReadAll is a convenience wrapper that returns an object's whole content.

func (*Store) ReadOnly 

func (s *Store) ReadOnly() bool

ReadOnly reports whether the store refuses mutations.

func (*Store) ReclaimAbandonedAppends 

func (s *Store) ReclaimAbandonedAppends(minAge time.Duration) (int, error)

ReclaimAbandonedAppends releases capacity held by streaming writes whose writer never finished, and returns how many objects it freed.

This closes the one gap the pg: queue cannot cover. A write is queued for reclamation at the start of materialization (WriteHandle.beginMaterialize), which protects everything from there to the commit. Before that point the object exists only as a partly written stream, and the block store records it under its own append-in-flight marker; a process that dies mid-upload leaves chunk files allocated and charged with nothing in the pg: queue pointing at them. The cache reclaims those from its eviction manager, which a pstore deliberately never constructs -- so without this the store that makes the heaviest use of streaming writes was the one never cleaning up after them.

The two mechanisms compose rather than double-count. Deleting an object clears its append marker in the same transaction, so a version the pg: queue already reclaimed is found here with no metadata and only its stale marker is dropped; conversely an object reclaimed here is gone before any pg: entry could refer to it.

minAge is the grace period; see appendReclaimGrace. Passing zero reclaims every record with no live writer, which is for tooling and tests rather than the janitor.

func (*Store) Remove 

func (s *Store) Remove(name string) error

Remove deletes a file, or an empty directory.

The entry and its reclamation record are written together, so the object's bytes can never become unreachable without also being queued for the janitor.

func (*Store) RemoveAll 

func (s *Store) RemoveAll(name string) error

RemoveAll deletes a path and everything beneath it.

A large subtree cannot be removed in one transaction, so the entry is unlinked from its parent — making the subtree unreachable at once — and its root handed to the janitor to drain. Deletion need not be atomic: a partially drained subtree is invisible either way.

func (*Store) Rename 

func (s *Store) Rename(oldName, newName string) error

Rename moves an entry, replacing any existing file at the destination.

Renaming a file rewrites one key. Renaming a directory rewrites every descendant's key, which is why it is O(descendants) — but object identity does not depend on the path, so no data file is touched and no bytes move.

func (*Store) Restore 

func (s *Store) Restore(r io.Reader, keys BackupKeys) error

Restore loads a metadata backup into this store.

There is one container and it is sealed, so the only question the file itself answers is whether it is a pstore snapshot at all: a file without the magic is refused as "not a backup", and one whose format version this build does not read is refused by number rather than misparsed. The two are separate messages on purpose -- they send an operator to different places.

**Any one of three keys opens an archive**, and keys may carry whichever the operator still has:

  • keys.IssuerKeys -- the origin's issuer keys, the ordinary case on the origin itself. Any single key the archive was sealed to is enough, so a rotation does not strand the archives that overlapped it.
  • keys.BackupKey -- the escrowed backup key from DeriveBackupKey. The archive carries its own backup key sealed to this one, so this restores on a machine that has never held the origin's issuer keys. **Escrowing it is what makes the loss of the issuer keys survivable.**
  • keys.FileKey -- that one archive's own key from DeriveFileKey, which opens it and nothing else.

The store must be empty: restoring over live records would interleave two namespaces rather than replace one.

**A failed restore is not a no-op.** BadgerDB's loader writes as it reads, so a snapshot that turns out to be truncated or damaged part-way through leaves records in the directory and this handle unusable. There is no way to unwind that from here; the directory has to be discarded and the restore started again into a fresh one. This is why the container authenticates its header before a byte of the body is opened and marks its terminating chunk: most damage is caught before a single record is written, and what is caught late says plainly what state it left behind.

**The store must be reopened afterwards.** A restore replaces the whole database, including everything a store reads once at open and then caches: the hash salt that every instance hash is derived from, and the mapping from storage IDs to directories. Continuing to use the handle would compute hashes that match nothing and look for block files under empty paths. The salt is refreshed here so that a caller inspecting the namespace immediately after a restore sees the right thing, but reading object data requires a fresh open.

func (*Store) RunGC 

func (s *Store) RunGC(ctx context.Context) (GCStats, error)

RunGC performs one reclamation pass and reports what it did.

Safe to call directly; the janitor simply calls it on a timer.

func (*Store) StartBackups 

func (s *Store) StartBackups(ctx context.Context, egrp *errgroup.Group, cfg BackupConfig)

StartBackups snapshots the catalog to dir every interval, for as long as the context lives.

Each pass writes a new timestamped file and prunes the oldest beyond keep, so a corrupted catalog can be rolled back to rather than only forward from. A failed pass is logged and retried at the next interval: a backup problem must not take the origin down.

One snapshot is taken immediately. Waiting for the first interval to elapse means an unwritable backup directory -- the wrong owner, a full filesystem, a typo in the configured path -- goes unreported for a whole interval, and a store that dies inside its first hour has no metadata backup at all, which is the case backups exist for.

After that the interval is a gap, not a cadence: the next snapshot starts an interval after the previous one *finished*. A snapshot of a large catalog is not instant, and the six-hour default is close enough to the time one can take that a fixed cadence would have the origin snapshotting back to back. See runPeriodically in integrity.go.

func (*Store) StartIntegrityChecks 

func (s *Store) StartIntegrityChecks(ctx context.Context, egrp *errgroup.Group, cfg IntegrityConfig)

StartIntegrityChecks runs the scheduled checks until the context is cancelled.

func (*Store) StartJanitor 

func (s *Store) StartJanitor(ctx context.Context, egrp *errgroup.Group)

StartJanitor runs the reclamation sweep until the context is cancelled.

The interval adapts to the backlog. Each sweep handles a bounded batch, so a fixed period puts a ceiling on how fast space can come back: deleting a tree of a million objects at one batch per resting period would take days, during which the store stays full and refuses writes. A sweep that fills its batch halves the wait, down to janitorMinInterval; a sweep that does not steps back toward the resting period. Reclamation therefore accelerates exactly when there is something to reclaim and idles otherwise.

func (*Store) Stat 

func (s *Store) Stat(name string) (*Dirent, error)

Stat returns the entry at the given path.

func (*Store) Storage 

func (s *Store) Storage() *local_cache.StorageManager

Storage exposes the block store, for introspection and tests.

func (*Store) Usage 

func (s *Store) Usage() (used, max int64)

Usage reports the store's committed-plus-reserved bytes and its ceiling. A ceiling of zero means the store is unbounded.

func (*Store) VerifyObject 

func (s *Store) VerifyObject(name string) (bool, error)

VerifyObject reads one object back and checks it against the checksums recorded when it was written.

Returns false when the data no longer matches, which means at-rest corruption: every other check reads only the catalog and cannot see it.

type WriteHandle 

type WriteHandle struct {
	// contains filtered or unexported fields
}

WriteHandle accumulates a new version of an object and installs it at Close.

Not safe for concurrent use. Nothing the handle writes is visible to readers until Close returns successfully.

func (*WriteHandle) Abort 

func (w *WriteHandle) Abort() error

Abort discards the pending version without touching the namespace.

func (*WriteHandle) Close 

func (w *WriteHandle) Close() error

Close finalizes the object's data and installs it at its path.

The install is one transaction: it re-checks preconditions, writes the new dirent, and queues the superseded version for reclamation. Committing the swap and the queue entry together is what makes overwrite crash-safe. A concurrent writer to the same path can abort that transaction, in which case it is retried against fresh state; see install.

func (*WriteHandle) Generation 

func (w *WriteHandle) Generation() string

Generation returns the version token this write will install, which is also the object's ETag.

func (*WriteHandle) Path 

func (w *WriteHandle) Path() string

Path returns the object path this write will install to.

func (*WriteHandle) RequireAbsent 

func (w *WriteHandle) RequireAbsent()

RequireAbsent makes Close fail with ErrExist if the object already exists, implementing "If-None-Match: *".

func (*WriteHandle) RequireGeneration 

func (w *WriteHandle) RequireGeneration(generation string)

RequireGeneration makes Close fail with ErrPreconditionFailed unless the object still carries the given generation, implementing "If-Match".

func (*WriteHandle) Size 

func (w *WriteHandle) Size() int64

Size returns how many bytes have been accepted so far. The object is not visible at its path until Close, so this is the only way to observe the pending version's length.

func (*WriteHandle) Write 

func (w *WriteHandle) Write(p []byte) (int, error)

Write appends to the new version.

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.