store

Documentation

Overview

Package store provides a concurrency-safe lightweight storage solution with a simple interface. Embedders should call `Lock` and `defer Release` (or WithLock(func()error)) to wrap operations, or series of operations, to ensure secure use. Furthermore, a Store implementation must do atomic writes, providing guarantees that interrupted partial writes never get committed. The Store interface itself is meant to be generic, and alternative stores (memory based, or content-addressable) may be implemented that satisfies it. This package also provides the default, file based implementation that we are using.

Index

• Variables
• func IsFilesystemSafe(identifier string) error
• type Locker
• type Manager
• type Store
  • func New(rootPath string, dirPerm os.FileMode, filePerm os.FileMode) (Store, error)

Variables

var (
	// ErrInvalidArgument may be returned by Get, Set, List, or Delete by specific SafeStore implementations
	// (eg: filesystem), when they want to impose implementation dependent restrictions on the identifiers
	// (filesystems typically do).
	ErrInvalidArgument = errdefs.ErrInvalidArgument
	// ErrNotFound may be returned by Get or Delete when the requested key is not present in the store
	ErrNotFound = errdefs.ErrNotFound
	// ErrSystemFailure may be returned by implementations when an internal failure occurs.
	// For example, for a filesystem implementation, failure to create a file will be wrapped by this error.
	ErrSystemFailure = errors.New("system failure")
	// ErrLockFailure may be returned by ReadLock, WriteLock, or Unlock, when the underlying locking mechanism fails.
	// In the case of the filesystem implementation, inability to lock the directory will return it.
	ErrLockFailure = errors.New("lock failure")
	// ErrFaultyImplementation may be returned by Get or Set when the target key exists and is a dir,
	// or by List when the target key is a file
	// This is indicative the code using the store is not consistent with what it treats as group, and what it treats as key
	// and is definitely a bug in that code
	// Missing lock will also trigger this when detected.
	ErrFaultyImplementation = errors.New("code needs to be fixed")
)

Functions

func IsFilesystemSafe

func IsFilesystemSafe(identifier string) error

Types

type Locker

type Locker interface {
	Lock() error
	Release() error
	WithLock(fun func() error) (err error)
}

Locker describes a locking mechanism that can be used to encapsulate operations in a safe way

type Manager

type Manager interface {
	// List will return a slice of all subgroups (eg: subdirectories), or keys (eg: files), under a specific group (eg: dir)
	// Note that `key...` may be omitted, in which case, all objects' names at the root of the store are returned.
	// Example, in the volumestore, List() will return all existing volumes names
	List(key ...string) ([]string, error)
	// Exists checks that a given key exists
	// Example: Exists("meta.json")
	Exists(key ...string) (bool, error)
	// Get returns the content of a key
	Get(key ...string) ([]byte, error)
	// Set saves bytes to a key
	Set(data []byte, key ...string) error
	// Delete removes a key or a group
	Delete(key ...string) error
	// Location returns the absolute path to a certain resource
	// Note that this technically "leaks" (filesystem) implementation details up.
	// It is necessary though when we are going to pass these filepath to containerd for eg.
	Location(key ...string) (string, error)

	// GroupSize will return the combined size of all objects stored under the group (eg: dir)
	GroupSize(key ...string) (int64, error)
	// GroupEnsure ensures that a given group (eg: directory) exists
	GroupEnsure(key ...string) error
}

Manager describes operations that can be performed on the store

type Store

type Store interface {
	Locker
	Manager
}

Store represents a store that is able to grant an exclusive lock (ensuring concurrency safety, both between go routines and across multiple binaries invocations), and is performing atomic operations. Note that Store allows manipulating nested objects: - Set([]byte("mykeyvalue"), "group", "subgroup", "my key1") - Set([]byte("mykeyvalue"), "group", "subgroup", "my key2") - Get("group", "subgroup", "my key1") - List("group", "subgroup") Note that both Delete and Exists can be applied indifferently to specific keys, or groups.

func New

func New(rootPath string, dirPerm os.FileMode, filePerm os.FileMode) (Store, error)

New returns a filesystem based Store implementation that satisfies both Manager and Locker Note that atomicity is "guaranteed" by `os.Rename`, which arguably is not *always* atomic. In particular, operating-system crashes may break that promise, and windows behavior is probably questionable. That being said, this is still a much better solution than writing directly to the destination file.