backup

package
v0.21.3 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Aug 19, 2026 License: Apache-2.0 Imports: 33 Imported by: 0

Documentation

Overview

Package backup is a snapshot engine for applications whose state consists of metadata plus a tree of content-addressed files. Metadata may be captured either as incremental SQLite page maps or as an application-defined portable stream that the application rebuilds into its current runtime database on restore.

Create pins the database in a frozen read transaction, captures changed SQLite pages and any new content files into content-addressed packs (kit/pack), and writes a manifest chained to the previous snapshot. Restore and Verify reconstruct and validate snapshots from the repository. Plain content capture, verification, and loose restore use bounded-memory streams; format-v1 encrypted entries retain their whole-entry authentication contract and are not exposed as verified-prefix streams.

Existing Create, Verify, and Restore callers receive the plain-content streaming implementation without changing repository format or application adapters. Repo.OpenBlob is available to sequential consumers; bytes from it become authoritative only after terminal EOF or a successful Verify, and an early Close reports incomplete verification. Capture preparation trades object-sized heap for private repository scratch, so callers must provision scratch capacity for the configured capture concurrency.

The engine is application-neutral: everything specific to a given application (its database filename, content directory, referenced-file enumeration, portable metadata format, and the opaque stats payload recorded per snapshot) is supplied through application interfaces. The engine records stats bytes at create and byte-compares them at restore without interpreting them.

On-disk formats are versioned by the FormatVersion and MinReaderVersion constants and the per-snapshot manifest fields; readers refuse snapshots that require a newer reader.

Index

Constants

View Source 
const (
	// FormatVersion is the backup repository format version this code writes.
	FormatVersion = 1
	// MinReaderVersion is the oldest reader able to read repos this code
	// writes.
	MinReaderVersion = 1
	// SupportedReaderVersion is the newest format this code can read. It is
	// deliberately distinct from FormatVersion ("what we write"): a future
	// release may read formats newer than the one it writes, or vice versa.
	// Repo.Open and LoadManifest refuse anything whose min_reader_version
	// exceeds this.
	SupportedReaderVersion = 4
)

Variables

View Source 
var ErrRepoLocked = errors.New("backup: repository is locked")

ErrRepoLocked reports that another operation holds a conflicting repo lock.

Functions

func BuildBlobContent 

func BuildBlobContent(r io.ReaderAt, pageSize uint32, plan BlobPlan) ([]byte, error)

BuildBlobContent reads the plan's pages in order into one blob.

func CaptureExtras 

func CaptureExtras(ctx context.Context, opts ExtrasOptions, appender *PackAppender) (pack.BlobID, bool, error)

CaptureExtras stores extras file blobs and the tree object. ctx is checked before each file read, so a canceled backup stops within one file instead of walking and reading every remaining extras source.

func ComputeSnapshotID 

func ComputeSnapshotID(createdAt time.Time, m *Manifest) (string, error)

ComputeSnapshotID derives the time-ordered, content-derived snapshot ID (FORMAT.md): UTC timestamp plus the first 32 hex chars (128 bits) of the SHA-256 of the manifest JSON with snapshot_id blanked. The digest must be long enough that crafting a different manifest with the same ID is infeasible: LoadManifest's recompute check is what stops a forged manifest from being served under a known snapshot ID, and a short, brute-forceable suffix would defeat it.

func EncodeAttachmentList 

func EncodeAttachmentList(refs []ContentRef) ([]byte, error)

EncodeAttachmentList serializes refs in their given (first-seen) order.

func EncodeHashDelta 

func EncodeHashDelta(d *PageHashDelta) []byte

EncodeHashDelta serializes a hash-map delta.

func EncodeHashKeyframe 

func EncodeHashKeyframe(m *PageHashMap) []byte

EncodeHashKeyframe serializes a full hash map.

func EncodeIndex 

func EncodeIndex(entries []IndexEntry) ([]byte, error)

EncodeIndex serializes entries sorted ascending by blob ID with a SHA-256 integrity trailer.

func EncodePageMap 

func EncodePageMap(m *PageMap, delta bool) []byte

EncodePageMap serializes a page map as a keyframe or delta object.

func PageHash 

func PageHash(page []byte) [pageHashSize]byte

PageHash returns the truncated SHA-256 dedup hash of one page.

func SaveHashMapCache 

func SaveHashMapCache(cacheDir, repoID, snapshotID string, m *PageHashMap) error

SaveHashMapCache atomically replaces the local hash-map cache. repoID must be a canonical generated repository ID: it becomes a filename under cacheDir, so any other value could write outside the cache. An empty cacheDir means the cache is disabled — the convention loading already uses — so saving is an explicit successful no-op rather than an accident of empty-path handling (MkdirAll("") happens to fail).

Types

type App 

type App interface {
	FrozenView(s *FrozenSession) FrozenView
	DBFileName() string     // e.g. "app.db"
	ContentDirName() string // e.g. "content"
	// PackFileExtension returns the file extension for pack files, including
	// the leading dot (e.g. ".kpack"). Like the other layout names it must
	// remain fixed for the life of a repository: packs are located by
	// <packID><ext>, so changing it strands previously written packs.
	PackFileExtension() string
	// RestoredContentPaths re-derives hash → relative paths from a restored
	// DB so restore can materialize and verify every referenced file. Returned
	// paths must be relative and local to the content directory (no absolute
	// paths, no ".." escapes); the engine also rejects any non-local path at
	// restore time.
	RestoredContentPaths(ctx context.Context, db *sql.DB) (map[string][]string, error)
	// RestoredStats recomputes stats from a restored DB for the fidelity proof.
	RestoredStats(ctx context.Context, db *sql.DB) (json.RawMessage, error)
	// CheckManifest returns app-level manifest consistency problems (verify).
	CheckManifest(m *Manifest) []string
	ExcludedPaths() []string
	Version() string // recorded as the manifest's app version
}

App supplies every application-specific behavior the engine needs. The engine treats stats payloads as opaque bytes: it records them at create and byte-compares them at restore.

type AttachmentCapture 

type AttachmentCapture struct {
	NewList     []ContentRef
	NewListBlob pack.BlobID
	HasNewList  bool
	Blobs       int64
	BlobBytes   int64
}

AttachmentCapture reports one snapshot's attachment capture results.

func CaptureAttachments 

func CaptureAttachments(
	ctx context.Context,
	attachmentsDir string, refs []ContentRef, parentSeen map[string]bool, appender *PackAppender,
	opts CaptureOptions,
) (*AttachmentCapture, error)

CaptureAttachments stores every referenced attachment content blob, re-hashing each file as it goes (FORMAT.md, Attachment Lists: backup verifies the live store). Refs not present in parentSeen become the snapshot's new list segment.

Reading, hashing, and trial compression fan out to opts.Jobs workers; results are recorded in ref order by a single collector feeding the appender, so pack contents, list order, accounting, and progress reporting match a serial capture exactly. Blobs already stored in the repository are detected before compression and skip it entirely, keeping the no-change incremental case cheap.

attachmentsDir is ignored entirely when opts.Source is non-nil; content is read through the source instead.

type AuxiliaryArtifact added in v0.17.0 

type AuxiliaryArtifact struct {
	Name   string
	Format string
	Open   func(context.Context) (io.ReadCloser, int64, error)
}

AuxiliaryArtifact is one application-defined, immutable snapshot artifact. Open must return the same bytes and exact size for the lifetime of its originating snapshot.

type AuxiliaryRestore added in v0.17.0 

type AuxiliaryRestore interface {
	Commit(context.Context) error
	Rollback(context.Context) error
}

AuxiliaryRestore is one staged auxiliary-state replacement. Commit runs only after Kit has published and durably synced the restored target. Rollback must discard all staged work and clean up any partial effects from a failed Commit. Kit calls Commit at most once and, if Commit does not succeed, Rollback exactly once with an independently bounded context.

type AuxiliarySource added in v0.17.0 

type AuxiliarySource interface {
	AuxiliaryArtifacts(context.Context) ([]AuxiliaryArtifact, error)
}

AuxiliarySource optionally extends a pinned MetadataSnapshot or FrozenView with application-neutral auxiliary artifacts captured under the same preservation boundary.

type AuxiliaryTarget added in v0.17.0 

type AuxiliaryTarget interface {
	StageAuxiliary(context.Context, []RestoredAuxiliary) (AuxiliaryRestore, error)
}

AuxiliaryTarget stages verified application-defined artifacts without making them externally visible. A StageAuxiliary error must leave no work for Kit to clean up. Kit never interprets artifact payloads.

type BlobPlan 

type BlobPlan struct {
	Ranges []PageRange
}

BlobPlan is one storage blob: either a single large run or a group of scattered small ranges concatenated in page order.

func PlanBlobs 

func PlanBlobs(dirty []PageRange) []BlobPlan

PlanBlobs groups dirty ranges into storage blobs.

func (*BlobPlan) Pages 

func (p *BlobPlan) Pages() uint64

Pages returns the plan's total page count.

type BlobStream added in v0.8.0 

type BlobStream struct {
	// contains filtered or unexported fields
}

BlobStream reads one repository blob without buffering the complete raw content. Successful terminal EOF or Verify proves the blob's stored CRC, decoded length, and content identity. Close before verification returns pack.ErrVerificationIncomplete.

func (*BlobStream) Close added in v0.8.0 

func (s *BlobStream) Close() error

Close releases the blob stream and its pack descriptor.

func (*BlobStream) Read added in v0.8.0 

func (s *BlobStream) Read(p []byte) (int, error)

Read implements io.Reader.

func (*BlobStream) Size added in v0.8.0 

func (s *BlobStream) Size() int64

Size returns the authoritative decoded length from the pack footer.

func (*BlobStream) Verified added in v0.8.0 

func (s *BlobStream) Verified() bool

Verified reports whether terminal verification succeeded.

func (*BlobStream) Verify added in v0.8.0 

func (s *BlobStream) Verify() error

Verify consumes the stream and verifies its terminal integrity.

type CaptureOptions 

type CaptureOptions struct {
	// Jobs is the number of concurrent read+hash+compress workers. Zero or
	// negative selects one per CPU. Use 1 for strictly serial file reads —
	// the right choice when the live archive sits on a spinning disk or NAS
	// share that degrades under concurrent reads.
	Jobs int
	// Progress, if non-nil, is called after each file is captured with the
	// number of files done so far, the total file count, and the cumulative
	// bytes read; it does not otherwise affect capture behavior.
	Progress func(done, total int, bytesRead int64)
	// Source, when non-nil, supplies attachment bytes instead of the engine
	// reading them from the attachments directory; the directory is then
	// ignored entirely. Reads are still hash-verified and size-capped.
	Source ContentSource
}

CaptureOptions tunes CaptureAttachments.

type ContentInfo 

type ContentInfo struct {
	Refs []ContentRef // one per unique hash, first-seen order
	Rows int64        // DB rows referencing content (manifest attachments.rows)
	// NonCanonicalPaths reports any ref recorded at a path other than the
	// canonical "<hash[:2]>/<hash>" layout; such snapshots require a
	// path-aware restore and a higher manifest reader version.
	NonCanonicalPaths bool
}

ContentInfo is what the engine needs to know about the application's content-addressed files, computed inside the frozen snapshot.

type ContentRef 

type ContentRef struct {
	Hash        string
	Size        int64
	StoragePath string
}

ContentRef identifies one attachment (or thumbnail) content blob by its SHA-256 and size. Size -1 means unknown until read from disk.

StoragePath is the blob's location relative to the attachments directory as recorded in the archive database; importers may namespace it (for example synctech-sms writes "synctech-sms/<aa>/<hash>"). Empty means the canonical loose layout "<aa>/<hash>". It is capture-time routing only and is not serialized into attachment list segments, which carry hash and size.

func DecodeAttachmentList 

func DecodeAttachmentList(data []byte) ([]ContentRef, error)

DecodeAttachmentList parses and integrity-checks a list segment.

func LoadListRefs 

func LoadListRefs(r *Repo, known map[pack.BlobID]IndexEntry, listBlobIDs []string, crypter *pack.Crypter, ext string) ([]ContentRef, map[string]bool, error)

LoadListRefs fetches and decodes a manifest's attachment list blobs. ext is the pack file extension (App.PackFileExtension).

type ContentSource added in v0.4.0 

type ContentSource interface {
	Open(ctx context.Context, ref ContentRef) (io.ReadCloser, error)
}

ContentSource supplies attachment content bytes during capture, replacing the engine's own reads of the attachments directory. Implementations resolve a ref however the application stores content (loose files, pack files, object stores); the engine still verifies every blob's SHA-256 against ref.Hash and enforces the per-blob size cap, so a source cannot weaken capture integrity. Open is called from concurrent capture workers and must be safe for concurrent use; it should honor ctx and return promptly once ctx is done, or a cancelled capture blocks until every in-flight Open returns. Capture uses ref.Size to pace concurrent work and scratch use; a declared size that understates the payload weakens that admission policy (never integrity), so sources should report actual sizes.

type CreateOptions 

type CreateOptions struct {
	DBPath     string
	ContentDir string
	// SQLiteOpener selects the SQLite implementation used for page-map
	// capture. Nil preserves Kit's mattn/go-sqlite3 default.
	SQLiteOpener SQLiteOpener
	// MetadataSource selects portable application metadata instead of SQLite
	// page capture. DBPath is ignored in this mode.
	MetadataSource MetadataSource
	// ContentSource, when non-nil, supplies attachment content bytes during
	// capture instead of the engine reading ContentDir; ContentDir is then
	// ignored for content reads (it may be empty). Extras and the page scan
	// are unaffected. See ContentSource.
	ContentSource ContentSource
	// DataDir anchors the Extras spec's directory walks and glob matches.
	DataDir string
	// Extras selects the operational files that ride along with the
	// snapshot. The engine imposes no default set; the application supplies
	// an explicit spec (ExtrasSpec).
	Extras                ExtrasSpec
	AllowPlaintextSecrets bool
	// IncludeConfig/IncludeTokens are recorded verbatim into the manifest's
	// options (wire-frozen fields, FORMAT.md); they select nothing by
	// themselves — the Extras spec does.
	IncludeConfig bool
	IncludeTokens bool
	Tag           string
	ZstdLevel     int
	CacheDir      string
	Freezer       FreezeCoordinator
	ForceUnlock   bool
	// Jobs is the number of concurrent attachment read+compress workers.
	// Zero or negative selects one per CPU. Use 1 for strictly serial file
	// reads when the live archive sits on a spinning disk or NAS share. The
	// page scan is unaffected: its disk reads are sequential at any setting.
	Jobs int
	// Progress, if non-nil, receives structured progress events as Create
	// runs. nil means fully silent. Create emits events freely and cheaply;
	// throttling for display is a rendering concern of the callback, not
	// Create's.
	Progress func(ProgressEvent)
}

CreateOptions parameterizes one snapshot capture.

type DefaultSQLiteOpener added in v0.10.0 

type DefaultSQLiteOpener struct{}

DefaultSQLiteOpener uses mattn/go-sqlite3 and preserves Kit's historical connection behavior.

func (DefaultSQLiteOpener) OpenSQLite added in v0.10.0 

func (DefaultSQLiteOpener) OpenSQLite(path string, opts SQLiteOpenOptions) (*sql.DB, error)

OpenSQLite implements SQLiteOpener.

type ExtrasDirSpec 

type ExtrasDirSpec struct {
	Name string
	// Sensitive marks the directory as carrying secrets (see ExtrasSpec).
	Sensitive bool
}

ExtrasDirSpec walks one DataDir-relative directory recursively; every regular file found is recorded under its DataDir-relative path. A missing directory is skipped, not an error: operational directories often appear only once the application has something to put in them.

type ExtrasEntry 

type ExtrasEntry struct {
	Path string `json:"path"`
	Mode uint32 `json:"mode"`
	Size int64  `json:"size"`
	Blob string `json:"blob"`
}

ExtrasEntry is one captured file in the extras tree.

type ExtrasFileSpec 

type ExtrasFileSpec struct {
	Path      string
	RecordAs  string
	Sensitive bool
}

ExtrasFileSpec captures one file, which may live outside DataDir, recorded in the tree under RecordAs (a slash-separated relative path). Unlike dir walks, a missing file is an error: naming a specific file is an explicit request that must not silently produce a snapshot without it.

type ExtrasGlobSpec 

type ExtrasGlobSpec struct {
	Pattern   string
	Sensitive bool
}

ExtrasGlobSpec matches file basenames at DataDir's top level. The pattern must be a pure basename (no separators); matching happens against directory entries, so a DataDir path containing glob metacharacters cannot corrupt it.

type ExtrasOptions 

type ExtrasOptions struct {
	// DataDir anchors the spec's Dirs walks and Globs matches. Empty means
	// no directory- or glob-based extras are captured.
	DataDir               string
	Spec                  ExtrasSpec
	AllowPlaintextSecrets bool
	Encrypted             bool
	// ContentDirName/DBFileName name the restore-layout paths an extras
	// record path may never claim — the database, its SQLite sidecars, and
	// the attachments tree — so capture refuses them with the same rules
	// restore enforces (validateExtrasEntryPath) instead of publishing a
	// snapshot restore and verify then reject. Create fills them from the
	// App; a caller leaving them empty skips only the reserved-name rule
	// (locality and Windows-safe component rules always apply).
	ContentDirName string
	DBFileName     string
}

ExtrasOptions parameterizes one extras capture (docs/usage/backup.md, Extras).

type ExtrasSpec 

type ExtrasSpec struct {
	Dirs  []ExtrasDirSpec
	Globs []ExtrasGlobSpec
	Files []ExtrasFileSpec
}

ExtrasSpec declares which operational files ride along with a snapshot. The engine imposes no default set: the application decides what its snapshots carry. Sources marked Sensitive are refused on an unencrypted repository unless the caller sets AllowPlaintextSecrets, so secrets never land in plaintext packs by accident.

type ExtrasTree 

type ExtrasTree struct {
	Entries []ExtrasEntry `json:"entries"`
}

ExtrasTree is the small JSON tree object referenced by the manifest.

type FreezeCoordinator 

type FreezeCoordinator interface {
	Begin(ctx context.Context) error
	End(ctx context.Context) error
}

FreezeCoordinator brackets the freeze window: Begin drains and holds the daemon's operation gate, End releases it (FORMAT.md, Freeze Protocol). The pinned read transaction — which keeps the main DB file frozen afterwards — is owned by FrozenSession, not the coordinator.

type FrozenSession 

type FrozenSession struct {
	PageSize  uint32
	PageCount uint64
	// contains filtered or unexported fields
}

FrozenSession holds the pinned read transaction that freezes the main DB file in content and size while writers proceed into the WAL.

func OpenFrozenSession 

func OpenFrozenSession(ctx context.Context, dbPath string, fc FreezeCoordinator) (*FrozenSession, error)

OpenFrozenSession executes the freeze protocol: gate -> checkpoint TRUNCATE -> pinned read transaction -> capture geometry -> gate release.

The gate is released here, before attachment capture runs, so a gated operation that deletes attachment files (remove-account) can race a long-running capture and delete a file the pinned transaction still references. That backup fails loudly (read or hash error) and is retryable; holding the gate through the whole capture window would block every daemon write for minutes instead. Accepted limitation — see FORMAT.md, Current Limitations.

func OpenFrozenSessionWithOpener added in v0.10.0 

func OpenFrozenSessionWithOpener(
	ctx context.Context, dbPath string, fc FreezeCoordinator, opener SQLiteOpener,
) (*FrozenSession, error)

OpenFrozenSessionWithOpener executes the freeze protocol with the supplied SQLite implementation. Nil opener preserves the default.

func (*FrozenSession) Close 

func (s *FrozenSession) Close() error

Close releases the pinned transaction and connection. Idempotent.

func (*FrozenSession) Tx 

func (s *FrozenSession) Tx() *sql.Tx

Tx exposes the pinned read transaction so an App's FrozenView can run its schema queries inside the frozen snapshot.

type FrozenView 

type FrozenView interface {
	ContentInfo(ctx context.Context) (*ContentInfo, error)
	Stats(ctx context.Context) (json.RawMessage, error)
}

FrozenView answers the application-schema questions Create asks against the pinned read transaction of a FrozenSession. It may additionally implement AuxiliarySource to contribute artifacts from that same view.

type IndexEntry 

type IndexEntry struct {
	Blob      pack.BlobID
	PackID    string
	Offset    uint64
	StoredLen uint64
	Flags     pack.BlobFlags
}

IndexEntry maps one blob to its location inside a sealed pack.

func DecodeIndex 

func DecodeIndex(data []byte) ([]IndexEntry, error)

DecodeIndex parses and integrity-checks an index object.

type LockInfo 

type LockInfo struct {
	Hostname   string `json:"hostname"`
	PID        int    `json:"pid"`
	Operation  string `json:"operation"`
	AcquiredAt string `json:"acquired_at"`
}

LockInfo is the JSON body of a repo lock file. Freshness is carried by the file's mtime (heartbeat), not by fields, so observers need no clock sync.

type Manifest 

type Manifest struct {
	FormatVersion    int `json:"format_version"`
	MinReaderVersion int `json:"min_reader_version"`
	// AppVersion records the application version that wrote the snapshot. The
	// wire key is frozen at format v1: renaming it would break every existing
	// repo's snapshot-ID recompute.
	AppVersion  string              `json:"msgvault_version"`
	SnapshotID  string              `json:"snapshot_id"`
	ParentID    string              `json:"parent_id"`
	CreatedAt   string              `json:"created_at"`
	Options     ManifestOptions     `json:"options"`
	DB          ManifestDB          `json:"db"`
	Metadata    *ManifestMetadata   `json:"metadata,omitempty"`
	Auxiliary   []ManifestAuxiliary `json:"auxiliary,omitempty"`
	Attachments ManifestAttachments `json:"attachments"`
	Extras      ManifestExtras      `json:"extras"`
	Excluded    []string            `json:"excluded"`
	// Stats is the application-defined stats payload. The engine treats it as
	// opaque bytes: recorded at create, byte-compared at restore.
	Stats           json.RawMessage `json:"stats"`
	NewPacks        []string        `json:"new_packs"`
	NewIndex        string          `json:"new_index"`
	DurationSeconds float64         `json:"duration_seconds"`
	BytesAdded      int64           `json:"bytes_added"`
}

func Create 

func Create(ctx context.Context, r *Repo, app App, opts CreateOptions) (*Manifest, error)

Create captures one snapshot: freeze -> scan -> pack -> index -> manifest (written last). See FORMAT.md.

type ManifestAttachments 

type ManifestAttachments struct {
	Layout    []string `json:"layout"`
	Rows      int64    `json:"rows"`
	Blobs     int64    `json:"blobs"`
	BlobBytes int64    `json:"blob_bytes"`
	Recipes   []string `json:"recipes"`
	Lists     []string `json:"lists"`
}

type ManifestAuxiliary added in v0.17.0 

type ManifestAuxiliary struct {
	Name   string `json:"name"`
	Format string `json:"format"`
	Blob   string `json:"blob"`
	Bytes  int64  `json:"bytes"`
	SHA256 string `json:"sha256"`
}

ManifestAuxiliary identifies one content-addressed auxiliary artifact.

type ManifestDB 

type ManifestDB struct {
	Engine        string `json:"engine"`
	PageSize      uint32 `json:"page_size"`
	PageCount     uint64 `json:"page_count"`
	PageMap       string `json:"page_map"`
	PageHashMap   string `json:"page_hash_map"`
	MapChainDepth int    `json:"map_chain_depth"`
}

type ManifestExtras 

type ManifestExtras struct {
	Tree string `json:"tree"`
}

type ManifestMetadata added in v0.9.0 

type ManifestMetadata struct {
	Format string `json:"format"`
	Blob   string `json:"blob"`
	Bytes  int64  `json:"bytes"`
}

ManifestMetadata identifies one application-owned portable metadata blob. nil denotes the historical SQLite page-map representation in DB.

type ManifestOptions 

type ManifestOptions struct {
	IncludeConfig bool   `json:"include_config"`
	IncludeTokens bool   `json:"include_tokens"`
	ZstdLevel     int    `json:"zstd_level"`
	Tag           string `json:"tag"`
}

type MetadataRestorer added in v0.9.0 

type MetadataRestorer interface {
	RestoreMetadata(ctx context.Context, format string, metadata io.Reader, targetPath string) error
}

MetadataRestorer builds the application's current runtime database at targetPath from one verified portable metadata stream. targetPath is inside Kit-owned private scratch, not the eventual restore target. The restorer must consume the stream through EOF, close and checkpoint the database, and leave no sidecars. Kit copies the closed result into its confined target root.

type MetadataSnapshot added in v0.9.0 

type MetadataSnapshot interface {
	OpenMetadata(context.Context) (io.ReadCloser, int64, error)
	ContentInfo(context.Context) (*ContentInfo, error)
	Stats(context.Context) (json.RawMessage, error)
	Close() error
}

MetadataSnapshot supplies portable metadata bytes, content membership, and fidelity stats from one stable application view. It may additionally implement AuxiliarySource to contribute artifacts from that same view.

type MetadataSource added in v0.9.0 

type MetadataSource interface {
	Format() string
	OpenSnapshot(context.Context) (MetadataSnapshot, error)
}

MetadataSource opens one application-owned logical metadata snapshot while Create holds its FreezeCoordinator. OpenSnapshot must establish a stable view before returning; the coordinator is released immediately afterward.

type NoopFreezeCoordinator 

type NoopFreezeCoordinator struct{}

NoopFreezeCoordinator is for tests and capture paths with no daemon.

func (NoopFreezeCoordinator) Begin 

func (NoopFreezeCoordinator) Begin(context.Context) error

Begin implements FreezeCoordinator.

func (NoopFreezeCoordinator) End 

func (NoopFreezeCoordinator) End(context.Context) error

End implements FreezeCoordinator.

type PackAppender 

type PackAppender struct {
	// contains filtered or unexported fields
}

PackAppender routes blobs into sealed packs, deduplicating against the repository's known blob set plus everything added during this run.

After any returned error, the appender is poisoned and must not be used again; callers must call Abort() and discard it. All subsequent Add and Finish calls will return the original error without writing.

func NewPackAppender 

func NewPackAppender(r *Repo, known map[pack.BlobID]IndexEntry, zstdLevel int, crypter *pack.Crypter, ext string) *PackAppender

NewPackAppender creates an appender. known is mutated: added blobs join it so later Adds in the same run dedup against them. ext is the pack file extension (App.PackFileExtension) new packs are sealed with.

func (*PackAppender) Abort 

func (a *PackAppender) Abort()

Abort discards the open pack; already-sealed packs remain (they are unreferenced without a manifest and get ignored, see FORMAT.md, Crash Consistency).

func (*PackAppender) Add 

func (a *PackAppender) Add(raw []byte) (pack.BlobID, bool, error)

Add stores raw as a blob unless it is already known. It returns the blob ID and whether a new pack entry was written.

func (*PackAppender) AddEncoded 

func (a *PackAppender) AddEncoded(id pack.BlobID, frame []byte, rawLen uint64, compressed bool) (bool, error)

AddEncoded stores one blob whose frame the caller already encoded with pack.EncodeFrame, unless it is already known. The caller inherits pack.Writer.AppendEncoded's contract: id and (frame, compressed) must derive from the same raw bytes. It reports whether a new pack entry was written.

func (*PackAppender) AddPrepared added in v0.8.0 

func (a *PackAppender) AddPrepared(ctx context.Context, prepared *pack.PreparedBlob) (bool, error)

AddPrepared consumes prepared and appends its bounded-scratch frame unless the blob is already known. Prepared is consumed or closed on every return.

func (*PackAppender) Finish 

func (a *PackAppender) Finish() ([]string, []IndexEntry, error)

Finish seals the open pack (aborting it if empty) and returns the packs sealed this run and their index entries.

type PackedContentTarget added in v0.7.0 

type PackedContentTarget interface {
	// Limits returns the target store's configured compatibility and
	// allocation ceilings.
	Limits() packstore.Limits
	// AcquireRestoreLease acquires a mutation lease from the same Coordinator
	// used by every maintainer of the target content store. The target transfers
	// sole ownership of a successful lease to Restore. Applications must acquire
	// their own operation gates before this method is called.
	AcquireRestoreLease(context.Context) (*packstore.Lease, error)
	// OpenRestoreCatalog returns the packed-authority adapter for db. db is
	// Restore's unpublished staged database, not the currently visible target.
	// It and the returned catalog's ReplaceRestoredPacks method run while the
	// restore lease is held and must not reenter its Coordinator.
	OpenRestoreCatalog(context.Context, *sql.DB) (packstore.RestoreCatalog, error)
}

PackedContentTarget supplies the application-owned packed-storage policy for an optional mixed packed-and-loose restore. It opens catalog authority only against Restore's unpublished staged SQLite database; Kit never opens an application's live catalog through this interface. Implementations must keep catalog replacement structurally valid and neutral to RestoredStats so Restore can prove the final staged database before publishing it.

type PageHashDelta 

type PageHashDelta struct {
	PageSize  uint32
	PageCount uint64
	Pages     []uint64
	Hashes    []byte
}

PageHashDelta patches a parent PageHashMap: PageCount is the new total, Pages lists changed page numbers ascending, Hashes their new hashes.

func BuildHashDelta 

func BuildHashDelta(res *ScanResult) *PageHashDelta

BuildHashDelta converts a scan's dirty set into a hash-map delta.

func DecodeHashDelta 

func DecodeHashDelta(data []byte) (*PageHashDelta, error)

DecodeHashDelta parses and integrity-checks a delta object.

type PageHashMap 

type PageHashMap struct {
	PageSize  uint32
	PageCount uint64
	Hashes    []byte
}

PageHashMap holds the truncated SHA-256 of every DB page. It is the input to incremental change detection (FORMAT.md, Page-Map Objects).

func ApplyHashDelta 

func ApplyHashDelta(base *PageHashMap, d *PageHashDelta) (*PageHashMap, error)

ApplyHashDelta produces the child hash map from a parent and a delta.

func DecodeHashKeyframe 

func DecodeHashKeyframe(data []byte) (*PageHashMap, error)

DecodeHashKeyframe parses and integrity-checks a keyframe object.

func LoadHashMapCache 

func LoadHashMapCache(cacheDir, repoID string) (string, *PageHashMap, error)

LoadHashMapCache reads the disposable local hash-map cache. Any read or parse failure returns empty results, never an error: the cache is rebuilt from the repository when unusable. A repoID that is not a canonical generated ID is an error, not a miss — it is joined into cacheDir as a filename, so anything else could address a file outside the cache.

func MaterializeHashMap 

func MaterializeHashMap(
	fetch func(pack.BlobID) ([]byte, error),
	chain []pack.BlobID,
) (*PageHashMap, error)

MaterializeHashMap walks a newest-to-oldest blob chain until it finds a keyframe, then replays the deltas oldest-first.

type PageMap 

type PageMap struct {
	PageSize  uint32
	PageCount uint64
	Blobs     []pack.BlobID
	Runs      []PageRun
}

PageMap is the run-length-encoded page -> (blob, offset) mapping (FORMAT.md, Page-Map Objects). A keyframe covers every page exactly once; a delta covers only the ranges it replaces.

func ApplyPageMapDelta 

func ApplyPageMapDelta(base, delta *PageMap) (*PageMap, error)

ApplyPageMapDelta merges a delta into a base map: delta ranges win, base runs are split around them, and the result is resized to the delta's page count.

func DecodePageMap 

func DecodePageMap(data []byte) (*PageMap, bool, error)

DecodePageMap parses either page-map object form and validates structure.

func MaterializePageMap 

func MaterializePageMap(fetch func(pack.BlobID) ([]byte, error), chain []pack.BlobID) (*PageMap, error)

MaterializePageMap walks a newest-to-oldest blob chain to a keyframe and replays the deltas oldest-first.

func (*PageMap) CheckCoverage 

func (m *PageMap) CheckCoverage() error

CheckCoverage verifies a complete map covers every page exactly once.

func (*PageMap) Lookup 

func (m *PageMap) Lookup(page uint64) (pack.BlobID, uint64, error)

Lookup returns the blob and byte offset holding a page's content.

type PageRange 

type PageRange struct {
	Start uint64
	Count uint64
}

PageRange is a contiguous run of pages.

type PageRun 

type PageRun struct {
	StartPage  uint64
	PageCount  uint32
	BlobIndex  uint32
	BlobOffset uint64
}

PageRun maps a contiguous page range to consecutive bytes within a blob.

func RunsForPlan 

func RunsForPlan(plan BlobPlan, blobIndex uint32, pageSize uint32) []PageRun

RunsForPlan emits the page-map runs describing where the plan's pages live inside its stored blob.

type Problem 

type Problem struct {
	SnapshotID string
	Detail     string
}

Problem names one verification failure precisely (FORMAT.md, Verification Model).

type ProgressEvent 

type ProgressEvent struct {
	Stage      ProgressStage
	Done       int64
	Total      int64
	BytesDone  int64
	BytesTotal int64
	Final      bool
}

ProgressEvent reports one step of progress within a Stage. Done and Total are item counts (pages, files, blobs, snapshots) — Total is 0 when the item count isn't known in advance. BytesDone and BytesTotal are the corresponding byte counts where meaningful; BytesTotal is 0 when the byte total isn't known ahead of time (a renderer can still show BytesDone and a derived rate). Final marks the last event Create or Verify will emit for this Stage, i.e., the stage has completed.

type ProgressStage 

type ProgressStage string

ProgressStage names one phase of a Create or Verify run that reports progress. 

const (
	// ProgressStageFreeze covers opening the frozen read session.
	ProgressStageFreeze ProgressStage = "freeze"
	// ProgressStageScan covers the full page-hash scan.
	ProgressStageScan ProgressStage = "scan"
	// ProgressStageMetadata covers portable metadata capture or restore.
	ProgressStageMetadata ProgressStage = "metadata"
	// ProgressStagePack covers compressing and writing changed-page blobs
	// into pack files. On a first backup this is most of the wall clock
	// (every page is new); on a no-change backup it is skipped entirely.
	ProgressStagePack ProgressStage = "pack"
	// ProgressStageAttachments covers attachment content capture.
	ProgressStageAttachments ProgressStage = "attachments"
	// ProgressStageSeal covers sealing packs and writing the index.
	ProgressStageSeal ProgressStage = "seal"
	// ProgressStageVerify covers Verify's per-snapshot and per-blob checks.
	ProgressStageVerify ProgressStage = "verify"
	// ProgressStageRestoreDB covers materializing the database file from
	// page-map runs during Restore.
	ProgressStageRestoreDB ProgressStage = "db"
	// ProgressStageExtras covers laying out captured extras files during
	// Restore.
	ProgressStageExtras ProgressStage = "extras"
	// ProgressStageIntegrityCheck covers Restore's optional SQLite
	// integrity_check. The check reports only start and completion because
	// SQLite does not expose intermediate progress.
	ProgressStageIntegrityCheck ProgressStage = "integrity_check"
	// ProgressStageRestoreStats covers recomputing and comparing the restored
	// database's application-defined manifest statistics.
	ProgressStageRestoreStats ProgressStage = "restore_stats"
	// ProgressStageProof is retained for source compatibility. Restore emits
	// the more specific IntegrityCheck and RestoreStats stages instead.
	// Deprecated: use ProgressStageIntegrityCheck and ProgressStageRestoreStats.
	ProgressStageProof ProgressStage = "proof"
)

type Repo 

type Repo struct {
	// contains filtered or unexported fields
}

Repo is an opened backup repository rooted at a directory.

func Init 

func Init(root string) (*Repo, error)

Init creates a new empty repository at root. It refuses to reuse a directory that already contains a repository config.

func Open 

func Open(root string) (*Repo, error)

Open loads an existing repository and enforces version compatibility.

func (*Repo) AcquireExclusiveLock 

func (r *Repo) AcquireExclusiveLock(operation string, force bool) (*RepoLock, error)

AcquireExclusiveLock takes locks/exclusive.json for a mutating operation. It removes stale locks, refuses fresh ones unless force is set, and after planting the exclusive file waits out fresh shared locks (releasing and failing if they persist past sharedWaitTimeout).

func (*Repo) AcquireSharedLock 

func (r *Repo) AcquireSharedLock(operation string, force bool) (*RepoLock, error)

AcquireSharedLock takes locks/shared-<ulid>.json for a read-walking operation (verify, restore). It refuses under a fresh exclusive lock.

The pre-plant check alone is racy: AcquireExclusiveLock could plant exclusive.json and finish its (single) freshSharedLocks scan in the window between our check and our own plant, and both sides would then believe they hold a compatible lock. Closing that requires the standard create-then-verify handshake: after planting our shared file we re-check for a fresh exclusive lock and back off if one is now present. This is safe for the mirrored ordering too — if our shared file lands first, AcquireExclusiveLock's freshSharedLocks scan (which always runs after its own plant) will see it and wait.

func (*Repo) CleanStaging 

func (r *Repo) CleanStaging() error

CleanStaging removes in-flight write debris. Callers must hold the exclusive repo lock (concurrent writers stage under the same directory).

func (*Repo) Config 

func (r *Repo) Config() RepoConfig

Config returns the repository descriptor.

func (*Repo) HashMapChain 

func (r *Repo) HashMapChain(head *Manifest) ([]pack.BlobID, error)

HashMapChain collects the newest-to-oldest page-hash-map blob chain from head down to (and including) its keyframe manifest.

func (*Repo) LatestSnapshot 

func (r *Repo) LatestSnapshot() (*Manifest, error)

LatestSnapshot returns the newest manifest, or nil for an empty repo.

func (*Repo) ListSnapshots 

func (r *Repo) ListSnapshots() ([]*Manifest, error)

ListSnapshots returns every manifest sorted ascending by snapshot ID (IDs are time-prefixed, so this is chronological). Create enforces strictly increasing CreatedAt timestamps per repo (see nextCreatedAt in create.go), so even snapshots created within the same wall-clock second still sort chronologically by ID. Lock-free by design.

func (*Repo) LoadBlobIndex 

func (r *Repo) LoadBlobIndex() (map[pack.BlobID]IndexEntry, error)

LoadBlobIndex reads every index object and returns the union blob map, trusting every *.mvidx file in the indexes directory without checking whether a manifest actually references it.

An index can be orphaned if Create fails after WriteIndex succeeds but before the manifest is written (e.g. SetPageSize or WriteManifest fails afterward). That orphan is still safe to dedupe against: WriteIndex is only ever called after appender.Finish() has sealed its packs durably (create.go), so every entry in a published index — orphaned or not — points at a real, durable, sealed blob. A later run that re-derives the same content will see it already indexed here and skip re-storing it, reusing the orphaned blob instead of duplicating it. There is no unsafe window: the failure that can orphan an index happens strictly after the data it describes is already durable.

func (*Repo) LoadManifest 

func (r *Repo) LoadManifest(id string) (*Manifest, error)

LoadManifest reads one manifest by snapshot ID.

func (*Repo) OpenBlob added in v0.8.0 

func (r *Repo) OpenBlob(
	ctx context.Context, known map[pack.BlobID]IndexEntry, id pack.BlobID,
	crypter *pack.Crypter, ext string,
) (*BlobStream, error)

OpenBlob opens a verified-on-EOF stream by resolving id through known and matching that index record to the pack's authoritative footer entry. Plain format-v1 entries are streamable; encrypted format-v1 entries return pack.ErrStreamUnsupported and remain available through ReadBlob.

func (*Repo) PageMapChain 

func (r *Repo) PageMapChain(head *Manifest) ([]pack.BlobID, error)

PageMapChain collects the newest-to-oldest page-map blob chain.

func (*Repo) Path 

func (r *Repo) Path(parts ...string) string

Path joins parts under the repository root.

func (*Repo) ReadBlob 

func (r *Repo) ReadBlob(known map[pack.BlobID]IndexEntry, id pack.BlobID, crypter *pack.Crypter, ext string) ([]byte, error)

ReadBlob fetches one blob by resolving its pack through the index map. The pack footer is authoritative for the entry's RawLen and CRC. ext is the pack file extension (App.PackFileExtension).

func (*Repo) Root 

func (r *Repo) Root() string

Root returns the repository root directory.

func (*Repo) SetPageSize 

func (r *Repo) SetPageSize(pageSize int) error

SetPageSize records the DB page size after the first backup.

func (*Repo) WriteIndex 

func (r *Repo) WriteIndex(entries []IndexEntry) (string, error)

WriteIndex publishes a new immutable index object and returns its ULID.

func (*Repo) WriteManifest 

func (r *Repo) WriteManifest(m *Manifest) (string, error)

WriteManifest fills the snapshot ID and publishes the manifest. It must be the final write of a backup: a manifest's existence asserts closure.

type RepoConfig 

type RepoConfig struct {
	RepoID           string `toml:"repo_id"`
	FormatVersion    int    `toml:"format_version"`
	MinReaderVersion int    `toml:"min_reader_version"`
	Encryption       string `toml:"encryption"`
	CreatedAt        string `toml:"created_at"`
	PageSize         int    `toml:"page_size"`
}

RepoConfig is the plaintext repository descriptor (FORMAT.md). It stays unencrypted even in encrypted repos because it bootstraps everything else.

type RepoLock 

type RepoLock struct {
	// contains filtered or unexported fields
}

RepoLock is a held repository lock with a heartbeat goroutine. info holds the exact LockInfo this process wrote, so Release can verify it still owns the file at path before removing it (the file may have been reaped as stale and replanted by another holder in the meantime).

func (*RepoLock) Release 

func (l *RepoLock) Release() error

Release stops the heartbeat and removes the lock file, but only if the file still holds the LockInfo this RepoLock planted. If this holder was slow enough to be reaped as stale, another holder may have replanted the same path with its own live lock; removing it would delete that lock out from under its owner. Removal is made atomic against that race by claiming the file with a rename first: only one caller can win the rename, so the file this Release inspects and deletes is one it solely owns. When the claimed file is not ours it is restored to its path (without clobbering any newer lock), and — because our lock is already gone — Release reports no error.

A cheap pre-read confines the claim protocol to the only case that needs it. If the file at l.path already, provably, holds a different holder's LockInfo, it belongs to a successor and Release returns without claiming: the old behavior of never touching a file whose body is not ours. This avoids the claim's momentary rename-away vacancy — into which a third acquirer could plant a lock that the return step then drops — in the common replanted case. Only an ours-looking or unreadable lock proceeds to the claim handshake.

type RestoreOptions 

type RestoreOptions struct {
	SnapshotID string // empty: latest
	// TargetDir receives the restored archive: the database file, content dir,
	// and any captured extras. It must not exist, or be an empty directory,
	// unless Overwrite is set. Overwrite merges into the existing tree:
	// restored files replace same-named ones, files the snapshot does not
	// carry are left in place, and the existing database and its SQLite
	// sidecars survive until the replacement database is fully materialized,
	// every attachment and extras blob has been read and verified, and the
	// replacement has passed the restore proof — only then are the sidecars
	// set aside (a stale -wal, -shm, or -journal would otherwise be replayed
	// over the restored file on its first normal open), the database renamed
	// into place, and the set-aside sidecars removed. A failed rename puts
	// the sidecars back.
	TargetDir string
	Overwrite bool
	// Jobs is the number of concurrent pack-read workers. Zero or negative
	// selects one per CPU. Use 1 to read packs strictly one at a time when
	// the repository lives on a spinning disk or NAS share.
	Jobs        int
	ForceUnlock bool
	// SkipIntegrityCheck omits SQLite's full PRAGMA integrity_check after
	// materialization. Database pages and content blobs remain SHA-256 verified,
	// and restored application statistics are still compared with the manifest.
	SkipIntegrityCheck bool
	// Progress, if non-nil, receives structured progress events as Restore
	// runs. nil means fully silent.
	Progress func(ProgressEvent)
	// MetadataRestorer is required for snapshots whose application state is a
	// portable metadata artifact instead of SQLite page maps.
	MetadataRestorer MetadataRestorer
	// PackedContent optionally restores compatible repository packs into the
	// target content store and replaces packed authority in the unpublished
	// staged DB before its integrity/stats proof and final publication. Pack and
	// entry compatibility use the target's configured limits; declined hashes
	// are restored and verified loose. nil preserves a fully-loose restore.
	// When non-nil, "packs" is reserved as the first component below the
	// application's content directory.
	PackedContent PackedContentTarget
	// TargetCoordinator optionally acquires application-owned coordination
	// against the exact pre-opened target root Restore will mutate. This closes
	// the pathname gap that would exist if a caller locked TargetDir before Kit
	// opened it. The lease is held through publication and durability sync.
	TargetCoordinator RestoreTargetCoordinator
	// SQLiteOpener selects the SQLite implementation used to validate and
	// update the staged database. Nil preserves Kit's mattn/go-sqlite3 default.
	SQLiteOpener SQLiteOpener
	// AuxiliaryTarget stages verified application-defined snapshot artifacts
	// after all restore proofs and commits them only after the target is
	// published and durably synced.
	AuxiliaryTarget AuxiliaryTarget
	// BeforePublication runs after every content object has been restored and
	// verified but before the staged database's integrity/stats proof and
	// canonical publication. Applications may update a private scratch copy to
	// bind restored physical state; DBPath is outside TargetDir and does not
	// resolve through it. The callback must close/checkpoint its database and
	// leave no SQLite sidecars; returning an error leaves it unpublished.
	BeforePublication func(context.Context, RestorePublicationTarget) error
}

RestoreOptions parameterizes one restore run (FORMAT.md, Restore).

type RestorePublicationTarget added in v0.17.0 

type RestorePublicationTarget struct {
	TargetDir string
	DBPath    string
}

RestorePublicationTarget identifies the private scratch state passed to BeforePublication. DBPath is outside TargetDir and valid only for the duration of the callback.

type RestoreResult 

type RestoreResult struct {
	SnapshotID      string
	DBPath          string
	DBBytes         int64
	AttachmentBlobs int64
	AttachmentBytes int64
	// PackedAttachmentBlobs and LooseAttachmentBlobs partition
	// AttachmentBlobs by restored representation.
	PackedAttachmentBlobs int64
	LooseAttachmentBlobs  int64
	// AttachmentPacks is the number of repository packs imported and granted
	// authority in the restored application catalog.
	AttachmentPacks int
	// PackFallbacks records why packs or individual selected hashes were
	// restored loose. An empty Hash means the reason applies to the whole pack.
	PackFallbacks []packstore.ImportFallback
	ExtrasFiles   int
	// AuxiliaryArtifacts is the number of verified artifacts committed through
	// AuxiliaryTarget.
	AuxiliaryArtifacts int
	// DatabaseIntegrityChecked reports whether Restore ran SQLite's full
	// PRAGMA integrity_check against the staged database.
	DatabaseIntegrityChecked bool
	Duration                 time.Duration
}

RestoreResult reports what Restore materialized and proved.

func Restore 

func Restore(ctx context.Context, r *Repo, app App, opts RestoreOptions) (res *RestoreResult, err error)

Restore materializes one snapshot into TargetDir and then proves the result (FORMAT.md, Restore): every database page is hash-verified against the snapshot's page-hash map as it is written, every blob read re-derives its SHA-256 identity, and the restored database reproduces the manifest's recorded stats exactly before Restore reports success, and passes PRAGMA integrity_check unless SkipIntegrityCheck is set. When PackedContent is set, compatible packs are durably published before one staged catalog replacement; every fallback is durably restored loose before that authority change.

It takes a SHARED repository lock: concurrent restores and verifies are safe, a running create is not.

type RestoreTargetCoordinator added in v0.9.2 

type RestoreTargetCoordinator interface {
	AcquireRestoreTarget(context.Context, *os.Root) (RestoreTargetLease, error)
}

RestoreTargetCoordinator binds application-owned coordination to the exact directory descriptor Restore uses for every target mutation. Restore calls AcquireRestoreTarget after securely opening the target root and before stale staging cleanup or content publication. The root is borrowed: implementations must not close it or retain it after the lease is released.

type RestoreTargetLease added in v0.9.2 

type RestoreTargetLease interface {
	Release() error
}

RestoreTargetLease holds application-owned target coordination for one complete restore. Release runs before Kit closes the borrowed target root.

type RestoredAuxiliary added in v0.17.0 

type RestoredAuxiliary struct {
	Name   string
	Format string
	SHA256 string
	Data   []byte
}

RestoredAuxiliary carries independently verified artifact bytes to the application before target publication.

type SQLiteAccess added in v0.10.0 

type SQLiteAccess uint8

SQLiteAccess describes how a backup operation may use a database file. 

const (
	// SQLiteReadWriteExisting opens an existing database without creating it.
	SQLiteReadWriteExisting SQLiteAccess = iota + 1
	// SQLiteReadOnlyImmutable opens a completed, writer-free database without
	// creating SQLite sidecars.
	SQLiteReadOnlyImmutable
)

type SQLiteOpenFunc added in v0.10.0 

type SQLiteOpenFunc func(path string, opts SQLiteOpenOptions) (*sql.DB, error)

SQLiteOpenFunc adapts a function to SQLiteOpener.

func (SQLiteOpenFunc) OpenSQLite added in v0.10.0 

func (f SQLiteOpenFunc) OpenSQLite(path string, opts SQLiteOpenOptions) (*sql.DB, error)

OpenSQLite implements SQLiteOpener.

type SQLiteOpenOptions added in v0.10.0 

type SQLiteOpenOptions struct {
	Access      SQLiteAccess
	BusyTimeout time.Duration
}

SQLiteOpenOptions describes the filesystem and locking semantics required by one backup database open.

type SQLiteOpener added in v0.10.0 

type SQLiteOpener interface {
	OpenSQLite(path string, opts SQLiteOpenOptions) (*sql.DB, error)
}

SQLiteOpener lets an application keep backup and restore on the same SQLite implementation as its live metadata store.

type ScanResult 

type ScanResult struct {
	PageSize  uint32
	PageCount uint64
	Hashes    []byte
	Dirty     []PageRange
}

ScanResult carries the full page-hash pass and the dirty set vs the parent.

func ScanPages 

func ScanPages(
	ctx context.Context,
	r io.ReaderAt, pageSize uint32, pageCount uint64, parent *PageHashMap, progress func(done, total uint64),
) (*ScanResult, error)

ScanPages hashes every page and diffs against the parent hash map. The full scan is the honest cost of on-demand backup (FORMAT.md, Page-Map Objects); it doubles as live-DB bitrot detection. progress, if non-nil, is called once per chunk with the page count scanned so far and the total page count; it does not otherwise affect scan behavior.

Internally the scan pipelines: one goroutine reads chunks strictly sequentially (the disk access pattern is identical to a serial scan, so this is safe on spinning disks), a worker per CPU hashes pages, and a collector reassembles per-chunk results in order. The output is byte-identical to a serial scan.

type VerifyOptions 

type VerifyOptions struct {
	SnapshotID  string // empty: latest
	All         bool
	Quick       bool
	ForceUnlock bool
	// Jobs is the number of concurrent content-blob read workers used in
	// full mode. Zero or negative selects one worker per CPU. Use 1 to read
	// packs strictly one at a time — the right choice when the repository
	// lives on a spinning disk or NAS share that degrades under concurrent
	// reads. Quick mode reads no content and ignores Jobs.
	Jobs int
	// Progress, if non-nil, receives structured progress events as Verify
	// runs. nil means fully silent. Verify emits events freely and cheaply;
	// throttling for display is a rendering concern of the callback, not
	// Verify's.
	Progress func(ProgressEvent)
}

VerifyOptions parameterizes one integrity check run (FORMAT.md, Verification Model).

type VerifyResult 

type VerifyResult struct {
	Snapshots    []string
	BlobsChecked int64
	BytesRead    int64
	Problems     []Problem
}

VerifyResult reports what Verify checked and found.

func Verify 

func Verify(ctx context.Context, r *Repo, app App, opts VerifyOptions) (*VerifyResult, error)

Verify checks a backup repository's integrity (FORMAT.md, Verification Model). It takes a SHARED repo lock (released on return): concurrent verifies and restores are safe, but a running create/prune (exclusive) is not.

Snapshot selection: All verifies every manifest, SnapshotID verifies one (an error if it does not exist), and the default verifies only the latest.

In Quick mode, every referenced blob is resolved through the index and its pack footer, but content blobs are not read. The default full mode also reads and hash-verifies every content blob, checks the materialized page map's coverage, and cross-checks attachment-list totals against the manifest. Per-object failures are collected as Problems naming the snapshot, blob, and pack; Verify keeps going so every affected snapshot is named, rather than stopping at the first Problem.

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.