store

package
v1.0.4 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Feb 26, 2026 License: MIT Imports: 26 Imported by: 0

Documentation

Index

Constants

View Source 
const KeySlotPrefix = "keys/"

KeySlotPrefix is the object key prefix for encryption key slot objects. These objects are stored unencrypted (they contain already-wrapped keys) so they can be read without the encryption key — avoiding a chicken-and-egg problem during key loading.

Variables

View Source 
var ErrQuotaExceeded = errors.New("storage quota exceeded during backup")

Functions

func AddRecoverySlot 

func AddRecoverySlot(s ObjectStore, masterKey []byte) (mnemonic string, err error)

AddRecoverySlot generates a recovery key, wraps the given master key with it, stores the recovery slot, and returns the BIP39 24-word mnemonic.

func ExtractMasterKey 

func ExtractMasterKey(slots []KeySlot, platformKey []byte, password string) ([]byte, error)

ExtractMasterKey unwraps and returns the raw master key from whichever credential matches. Unlike the OpenWith* functions that return a derived encryption key, this returns the master key itself — needed when adding new key slots to an existing repo.

func HasKeySlots 

func HasKeySlots(s ObjectStore) bool

HasKeySlots reports whether the store contains any encryption key slots.

func InitEncryptionKey 

func InitEncryptionKey(s ObjectStore, platformKey []byte, password string) ([]byte, error)

InitEncryptionKey initializes encryption for a new repository. It generates a master key and creates key slots for whatever credentials are provided. At least one of platformKey or password must be non-empty. Returns the derived encryption key.

func OpenWithPassword 

func OpenWithPassword(slots []KeySlot, password string) ([]byte, error)

OpenWithPassword finds a password slot, derives the wrapping key using Argon2id, unwraps the master key, and returns the derived encryption key.

func OpenWithPlatformKey 

func OpenWithPlatformKey(slots []KeySlot, platformKey []byte) ([]byte, error)

OpenWithPlatformKey finds a platform slot, unwraps the master key using the given platform key, and returns the derived encryption key.

func OpenWithRecoveryKey 

func OpenWithRecoveryKey(slots []KeySlot, recoveryKey []byte) ([]byte, error)

OpenWithRecoveryKey finds a recovery slot, unwraps the master key using the given raw recovery key, and returns the derived encryption key.

func SlotTypes 

func SlotTypes(slots []KeySlot) string

SlotTypes returns the slot types present among the given slots.

func SyncKeySlots 

func SyncKeySlots(s ObjectStore, slots []KeySlot)

SyncKeySlots writes the given key slots to the store (best-effort). This is used to keep B2 in sync with the authoritative DB slots so that B2-only mode can find the correct key.

Types

type B2Store 

type B2Store struct {
	Client *b2.Client
	Bucket *b2.Bucket
	Prefix string
}

func NewB2Store 

func NewB2Store(keyID, appKey, bucketName string) (*B2Store, error)

func NewB2StoreWithPrefix 

func NewB2StoreWithPrefix(keyID, appKey, bucketName, prefix string) (*B2Store, error)

func (*B2Store) Delete 

func (s *B2Store) Delete(key string) error

func (*B2Store) DeletePrefix 

func (s *B2Store) DeletePrefix(prefix string) error

DeletePrefix deletes all objects under the given prefix.

func (*B2Store) Exists 

func (s *B2Store) Exists(key string) (bool, error)

func (*B2Store) Get 

func (s *B2Store) Get(key string) ([]byte, error)

func (*B2Store) List 

func (s *B2Store) List(prefix string) ([]string, error)

func (*B2Store) NewWriter 

func (s *B2Store) NewWriter(key string) io.WriteCloser

NewWriter returns a streaming writer to the given key in B2. The caller must Close the writer to finalize the upload.

func (*B2Store) Put 

func (s *B2Store) Put(key string, data []byte) error

func (*B2Store) SignedURL 

func (s *B2Store) SignedURL(key string, validFor time.Duration) (string, error)

SignedURL returns a time-limited download URL for the given key.

func (*B2Store) Size 

func (s *B2Store) Size(key string) (int64, error)

func (*B2Store) TotalSize 

func (s *B2Store) TotalSize() (int64, error)

type ChangeType 

type ChangeType string

ChangeType describes the kind of change reported by an IncrementalSource. 

const (
	ChangeUpsert ChangeType = "upsert"
	ChangeDelete ChangeType = "delete"
)

type EncryptedStore 

type EncryptedStore struct {
	ObjectStore
	// contains filtered or unexported fields
}

EncryptedStore wraps an ObjectStore and transparently encrypts data on Put and decrypts on Get using AES-256-GCM. Unencrypted (legacy) data is returned as-is on Get, enabling gradual migration.

Objects under the "keys/" prefix are passed through unencrypted because they hold the wrapped master key needed to derive the encryption key.

func NewEncryptedStore 

func NewEncryptedStore(inner ObjectStore, key []byte) *EncryptedStore

NewEncryptedStore creates an EncryptedStore that encrypts all Put operations and decrypts Get operations. The key must be 32 bytes (AES-256).

func (*EncryptedStore) Get 

func (s *EncryptedStore) Get(key string) ([]byte, error)

func (*EncryptedStore) Put 

func (s *EncryptedStore) Put(key string, data []byte) error

type FileChange 

type FileChange struct {
	Type ChangeType
	Meta core.FileMeta
}

FileChange pairs a change type with file metadata. For deletions only Meta.FileID is required.

type GDriveChangeSource 

type GDriveChangeSource struct {
	GDriveSource
}

GDriveChangeSource is an IncrementalSource backed by the Google Drive Changes API. It embeds GDriveSource to reuse authentication, full Walk, GetFileStream, and metadata conversion.

func NewGDriveChangeSource 

func NewGDriveChangeSource(credsPath, tokenPath string) (*GDriveChangeSource, error)

func (*GDriveChangeSource) GetStartPageToken 

func (s *GDriveChangeSource) GetStartPageToken() (string, error)

GetStartPageToken returns the token representing the current head of the Google Drive change stream.

func (*GDriveChangeSource) Info 

func (s *GDriveChangeSource) Info() core.SourceInfo

func (*GDriveChangeSource) WalkChanges 

func (s *GDriveChangeSource) WalkChanges(ctx context.Context, token string, callback func(FileChange) error) (string, error)

WalkChanges iterates over all changes since the given page token. Folder changes are emitted before file changes so that the engine can resolve parent references incrementally.

type GDriveSource 

type GDriveSource struct {
	Service      *drive.Service
	DriveID      string // shared drive ID; empty means "My Drive"
	RootFolderID string // if empty, defaults to "root" (entire drive)
	Account      string // Google account email; populated automatically if empty
}

GDriveSource implements Source for Google Drive. By default it backs up the entire "My Drive" root. Set DriveID to back up a shared drive instead, and/or set RootFolderID to restrict to a specific folder within the selected drive.

func NewGDriveSource 

func NewGDriveSource(credsPath, tokenPath string) (*GDriveSource, error)

NewGDriveSource creates a new GDriveSource. If credsPath is non-empty it is used as a Google credentials JSON file (user OAuth or service-account). When credsPath is empty the built-in OAuth client credentials are used instead. tokenPath is where the OAuth token will be cached.

func (*GDriveSource) GetFileStream 

func (s *GDriveSource) GetFileStream(fileID string) (io.ReadCloser, error)

func (*GDriveSource) Info 

func (s *GDriveSource) Info() core.SourceInfo

func (*GDriveSource) Size 

func (s *GDriveSource) Size(ctx context.Context) (*SourceSize, error)

Size returns the total size of the drive. For My Drive it uses the fast about.get endpoint. For shared drives it lists all files and sums sizes.

func (*GDriveSource) Walk 

func (s *GDriveSource) Walk(ctx context.Context, callback func(core.FileMeta) error) error

type HybridStore 

type HybridStore struct {
	// contains filtered or unexported fields
}

HybridStore implements ObjectStore by routing chunk data to B2 and all metadata objects (node, filemeta, content, snapshot, index) to PostgreSQL with write-through to B2 for disaster recovery.

func NewHybridStore 

func NewHybridStore(db TxFunc, b2 *B2Store) *HybridStore

func (*HybridStore) B2 

func (s *HybridStore) B2() *B2Store

B2 returns the underlying B2 store for operations that need direct B2 access (e.g. zip upload for restore, signed URLs).

func (*HybridStore) DB 

func (s *HybridStore) DB() TxFunc

DB returns the TxFunc for direct database access (e.g. reading encryption_key_slots that live outside app.objects).

func (*HybridStore) Delete 

func (s *HybridStore) Delete(key string) error

func (*HybridStore) Exists 

func (s *HybridStore) Exists(key string) (bool, error)

func (*HybridStore) Get 

func (s *HybridStore) Get(key string) ([]byte, error)

func (*HybridStore) List 

func (s *HybridStore) List(prefix string) ([]string, error)

func (*HybridStore) Put 

func (s *HybridStore) Put(key string, data []byte) error

func (*HybridStore) Size 

func (s *HybridStore) Size(key string) (int64, error)

func (*HybridStore) TotalSize 

func (s *HybridStore) TotalSize() (int64, error)

type IncrementalSource 

type IncrementalSource interface {
	Source
	// GetStartPageToken returns the token representing the current head of
	// the change stream. Call this before a full Walk to capture the baseline.
	GetStartPageToken() (string, error)
	// WalkChanges emits only the entries that changed since token.
	// It returns the new token to persist for the next run.
	WalkChanges(ctx context.Context, token string, callback func(FileChange) error) (newToken string, err error)
}

IncrementalSource extends Source with delta-based walking using a change token stored in the snapshot. On the first run (empty token) the engine falls back to the full Walk; on subsequent runs only changed entries are emitted.

type KDFParams 

type KDFParams struct {
	Algorithm string `json:"algorithm"`
	Salt      string `json:"salt"` // base64-encoded
	Time      uint32 `json:"time"`
	Memory    uint32 `json:"memory"`
	Threads   uint8  `json:"threads"`
}

KDFParams holds the parameters for password-based key derivation.

type KeySlot 

type KeySlot struct {
	SlotType   string     `json:"slot_type"`
	WrappedKey string     `json:"wrapped_key"`
	Label      string     `json:"label"`
	KDFParams  *KDFParams `json:"kdf_params,omitempty"`
}

KeySlot is the JSON representation of an encryption key slot stored in B2.

func LoadKeySlots 

func LoadKeySlots(s ObjectStore) ([]KeySlot, error)

LoadKeySlots reads all key slot objects from the store.

func LoadKeySlotsFromDB 

func LoadKeySlotsFromDB(db TxFunc) ([]KeySlot, error)

LoadKeySlotsFromDB reads key slots directly from the encryption_key_slots PostgreSQL table via a TxFunc. This is needed for HybridStore because the web writes key slots to this table (not to app.objects).

type LocalSource 

type LocalSource struct {
	RootPath string
}

LocalSource implements Source for local filesystem

func NewLocalSource 

func NewLocalSource(rootPath string) *LocalSource

func (*LocalSource) GetFileStream 

func (s *LocalSource) GetFileStream(fileID string) (io.ReadCloser, error)

func (*LocalSource) Info 

func (s *LocalSource) Info() core.SourceInfo

func (*LocalSource) Size 

func (s *LocalSource) Size(ctx context.Context) (*SourceSize, error)

func (*LocalSource) Walk 

func (s *LocalSource) Walk(ctx context.Context, callback func(core.FileMeta) error) error

type LocalStore 

type LocalStore struct {
	BasePath string
}

LocalStore implements ObjectStore for the local filesystem.

func NewLocalStore 

func NewLocalStore(basePath string) (*LocalStore, error)

func (*LocalStore) Delete 

func (s *LocalStore) Delete(key string) error

func (*LocalStore) Exists 

func (s *LocalStore) Exists(key string) (bool, error)

func (*LocalStore) Get 

func (s *LocalStore) Get(key string) ([]byte, error)

func (*LocalStore) List 

func (s *LocalStore) List(prefix string) ([]string, error)

List returns all keys matching the given prefix. When a prefix is provided the walk is scoped to just that subdirectory for efficiency.

func (*LocalStore) Put 

func (s *LocalStore) Put(key string, data []byte) error

func (*LocalStore) Size 

func (s *LocalStore) Size(key string) (int64, error)

func (*LocalStore) TotalSize 

func (s *LocalStore) TotalSize() (int64, error)

type MeteredStore 

type MeteredStore struct {
	ObjectStore
	// contains filtered or unexported fields
}

MeteredStore wraps an ObjectStore and tracks the total number of bytes written via Put and deleted via Delete. It is safe for concurrent use.

func NewMeteredStore 

func NewMeteredStore(s ObjectStore) *MeteredStore

func (*MeteredStore) BytesWritten 

func (m *MeteredStore) BytesWritten() int64

func (*MeteredStore) Delete 

func (m *MeteredStore) Delete(key string) error

func (*MeteredStore) DeleteReturnSize 

func (m *MeteredStore) DeleteReturnSize(key string) (int64, error)

func (*MeteredStore) Put 

func (m *MeteredStore) Put(key string, data []byte) error

func (*MeteredStore) Reset 

func (m *MeteredStore) Reset()

type ObjectStore 

type ObjectStore interface {
	Put(key string, data []byte) error
	Get(key string) ([]byte, error)
	Exists(key string) (bool, error)
	Delete(key string) error
	List(prefix string) ([]string, error)
	Size(key string) (int64, error)
	TotalSize() (int64, error)
}

ObjectStore is the interface for content-addressable object storage. Keys are slash-separated paths like "chunk/<hash>" or "snapshot/<hash>".

type OneDriveChangeSource 

type OneDriveChangeSource struct {
	OneDriveSource
}

OneDriveChangeSource is an IncrementalSource backed by the Microsoft Graph delta API. It embeds OneDriveSource to reuse authentication, full Walk, GetFileStream, and metadata conversion.

func NewOneDriveChangeSource 

func NewOneDriveChangeSource(clientID, tokenPath string) (*OneDriveChangeSource, error)

func (*OneDriveChangeSource) GetStartPageToken 

func (s *OneDriveChangeSource) GetStartPageToken() (string, error)

GetStartPageToken returns the current head of the OneDrive delta stream by requesting a "latest" delta token. The returned string is a full deltaLink URL.

func (*OneDriveChangeSource) Info 

func (s *OneDriveChangeSource) Info() core.SourceInfo

func (*OneDriveChangeSource) WalkChanges 

func (s *OneDriveChangeSource) WalkChanges(ctx context.Context, token string, callback func(FileChange) error) (string, error)

WalkChanges iterates over all changes since the given delta token. Folder changes are emitted before file changes so that the engine can resolve parent references incrementally. Returns the new delta token for the next run.

type OneDriveSource 

type OneDriveSource struct {
	Client *http.Client
}

func NewOneDriveSource 

func NewOneDriveSource(clientID, tokenPath string) (*OneDriveSource, error)

func (*OneDriveSource) GetFileStream 

func (s *OneDriveSource) GetFileStream(fileID string) (io.ReadCloser, error)

func (*OneDriveSource) Info 

func (s *OneDriveSource) Info() core.SourceInfo

func (*OneDriveSource) Size 

func (s *OneDriveSource) Size(ctx context.Context) (*SourceSize, error)

Size returns the total storage usage for the OneDrive account by calling the /me/drive endpoint which includes quota information.

func (*OneDriveSource) Walk 

func (s *OneDriveSource) Walk(ctx context.Context, callback func(core.FileMeta) error) error

type QuotaStore 

type QuotaStore struct {
	ObjectStore
	// contains filtered or unexported fields
}

QuotaStore wraps an ObjectStore and cancels the backup context when cumulative bytes written exceed the remaining budget.

func NewQuotaStore 

func NewQuotaStore(inner ObjectStore, budget int64, cancel context.CancelCauseFunc) *QuotaStore

func (*QuotaStore) Put 

func (q *QuotaStore) Put(key string, data []byte) error

func (*QuotaStore) Written 

func (q *QuotaStore) Written() int64

Written returns the total bytes successfully written through this store.

type Source 

type Source interface {
	Walk(ctx context.Context, callback func(core.FileMeta) error) error
	GetFileStream(fileID string) (io.ReadCloser, error)
	Info() core.SourceInfo
	Size(ctx context.Context) (*SourceSize, error)
}

Source is the interface for a backup data source (local filesystem, Google Drive, OneDrive, etc.). Implementations MUST ensure that parent folders are visited before their children during Walk.

type SourceSize 

type SourceSize struct {
	Bytes int64 `json:"bytes"`
	Files int64 `json:"files"`
}

SourceSize holds the total size of a source.

type TxFunc 

type TxFunc func(fn func(ctx context.Context, tx pgx.Tx) error) error

TxFunc executes fn inside a scoped database transaction. The caller is responsible for begin/commit/rollback and any session-level configuration (e.g. RLS tenant_id).

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.