api

Documentation

Overview

Package api provides a persistent node database interface for MKVS trees.

Index

• Constants
• Variables
• func ReviveHashedDBWriteLogs(ctx context.Context, logGetter func() (node.Root, HashedDBWriteLog, error), ...) (writelog.Iterator, error)
• func Visit(ctx context.Context, ndb NodeDB, root node.Root, visitor NodeVisitor) error
• type BaseBatch
  • func (b *BaseBatch) Commit(root node.Root) error
  • func (b *BaseBatch) OnCommit(hook func())
• type Batch
• type Config
• type HashedDBLogEntry
• type HashedDBWriteLog
  • func MakeHashedDBWriteLog(writeLog writelog.WriteLog, annotations writelog.Annotations) HashedDBWriteLog
• type NodeDB
  • func NewNopNodeDB() (NodeDB, error)
• type NodeVisitor
• type Subtree

Constants

const ModuleName = "storage/mkvs/db"

ModuleName is the module name.

Variables

var (
	// ErrNodeNotFound indicates that a node with the specified hash couldn't be found
	// in the database.
	ErrNodeNotFound = errors.New(ModuleName, 1, "mkvs: node not found in node db")
	// ErrWriteLogNotFound indicates that a write log for the specified storage hashes
	// couldn't be found.
	ErrWriteLogNotFound = errors.New(ModuleName, 2, "mkvs: write log not found in node db")
	// ErrNotFinalized indicates that the operation requires a version to be finalized
	// but the version is not yet finalized.
	ErrNotFinalized = errors.New(ModuleName, 3, "mkvs: version is not yet finalized")
	// ErrAlreadyFinalized indicates that the given version has already been finalized.
	ErrAlreadyFinalized = errors.New(ModuleName, 4, "mkvs: version has already been finalized")
	// ErrVersionNotFound indicates that the given version cannot be found.
	ErrVersionNotFound = errors.New(ModuleName, 5, "mkvs: version not found")
	// ErrPreviousVersionMismatch indicates that the version given for the old root does
	// not match the previous version.
	ErrPreviousVersionMismatch = errors.New(ModuleName, 6, "mkvs: previous version mismatch")
	// ErrVersionWentBackwards indicates that the new version is earlier than an already
	// inserted version.
	ErrVersionWentBackwards = errors.New(ModuleName, 7, "mkvs: version went backwards")
	// ErrRootNotFound indicates that the given root cannot be found.
	ErrRootNotFound = errors.New(ModuleName, 8, "mkvs: root not found")
	// ErrRootMustFollowOld indicates that the passed new root does not follow old root.
	ErrRootMustFollowOld = errors.New(ModuleName, 9, "mkvs: root must follow old root")
	// ErrBadNamespace indicates that the passed namespace does not match what is
	// actually contained within the database.
	ErrBadNamespace = errors.New(ModuleName, 10, "mkvs: bad namespace")
	// ErrNotEarliest indicates that the given version is not the earliest version.
	ErrNotEarliest = errors.New(ModuleName, 11, "mkvs: version is not the earliest version")
	// ErrReadOnly indicates that a write operation failed due to a read-only database.
	ErrReadOnly = errors.New(ModuleName, 12, "mkvs: read-only database")
	// ErrMultipartInProgress indicates that a multipart restore operation is already
	// in progress.
	ErrMultipartInProgress = errors.New(ModuleName, 13, "mkvs: multipart already in progress")
	// ErrInvalidMultipartVersion indicates that a Finalize, NewBatch or Commit was called with a version
	// that doesn't match the current multipart restore as set with StartMultipartRestore.
	ErrInvalidMultipartVersion = errors.New(ModuleName, 14, "mkvs: operation called with different version than current multipart version")
	// ErrUpgradeInProgress indicates that a database upgrade was started by the upgrader tool and the
	// database is therefore unusable. Run the upgrade tool to finish upgrading.
	ErrUpgradeInProgress = errors.New(ModuleName, 15, "mkvs: database upgrade in progress")
)

Functions

func ReviveHashedDBWriteLogs

func ReviveHashedDBWriteLogs(
	ctx context.Context,
	logGetter func() (node.Root, HashedDBWriteLog, error),
	valueGetter func(node.Root, hash.Hash) (*node.LeafNode, error),
	closer func(),
) (writelog.Iterator, error)

ReviveHashedDBWriteLogs is a helper for hashed database backends that converts a HashedDBWriteLog into a WriteLog.

The provided logGetter will be called first to fetch the next write log to convert. If it returns a nil write log, iteration terminates.

Then the provided valueGetter will be called for each log entry to fetch each of the values in the write log.

After iteration has finished, closer will be called.

func Visit

func Visit(ctx context.Context, ndb NodeDB, root node.Root, visitor NodeVisitor) error

Visit traverses the tree in DFS order using the passed visitor. The traversal is a pre-order DFS where the node is visited first, then its leaf (if any) and then its children (first left then right).

Different to the Visit method in the MKVS tree, this uses the NodeDB API directly to traverse the tree to avoid the overhead of keeping the cache.

Types

type BaseBatch

type BaseBatch struct {
	// contains filtered or unexported fields
}

BaseBatch encapsulates basic functionality of a batch so it doesn't need to be reimplemented by each concrete batch implementation.

func (*BaseBatch) Commit

func (b *BaseBatch) Commit(root node.Root) error

func (*BaseBatch) OnCommit

func (b *BaseBatch) OnCommit(hook func())

type Batch

type Batch interface {
	// MaybeStartSubtree returns a new subtree instance that can be used for
	// persisting nodes under a given root.
	//
	// Depth is the depth of the node that subtreeRoot points to.
	MaybeStartSubtree(subtree Subtree, depth node.Depth, subtreeRoot *node.Pointer) Subtree

	// OnCommit registers a hook to run after a successful commit.
	OnCommit(hook func())

	// PutWriteLog stores the specified write log into the batch.
	PutWriteLog(writeLog writelog.WriteLog, logAnnotations writelog.Annotations) error

	// RemoveNodes marks nodes for eventual garbage collection.
	RemoveNodes(nodes []node.Node) error

	// Commit commits the batch.
	Commit(root node.Root) error

	// Reset resets the batch for another use.
	Reset()
}

Batch is a NodeDB-specific batch implementation.

type Config

type Config struct {
	// DB is the path to the database.
	DB string

	// NoFsync will disable fsync() where possible.
	NoFsync bool

	// MemoryOnly will make the storage memory-only (if the backend supports it).
	MemoryOnly bool

	// ReadOnly will make the storage read-only.
	ReadOnly bool

	// Namespace is the namespace contained within the database.
	Namespace common.Namespace

	// MaxCacheSize is the maximum in-memory cache size for the database.
	MaxCacheSize int64

	// DiscardWriteLogs will cause all write logs to be discarded.
	DiscardWriteLogs bool
}

Config is the node database backend configuration.

type HashedDBLogEntry

type HashedDBLogEntry struct {
	Key          []byte
	InsertedHash *hash.Hash
}

HashedDBLogEntry is a single write log entry for HashedDBWriteLog.

type HashedDBWriteLog

type HashedDBWriteLog []HashedDBLogEntry

HashedDBWriteLog is a write log helper for database backends that can reference nodes by hash.

func MakeHashedDBWriteLog

func MakeHashedDBWriteLog(writeLog writelog.WriteLog, annotations writelog.Annotations) HashedDBWriteLog

MakeHashedDBWriteLog converts the given write log and annotations into a serializable slice with hash node references.

type NodeDB

type NodeDB interface {
	// GetNode looks up a node in the database.
	GetNode(root node.Root, ptr *node.Pointer) (node.Node, error)

	// GetWriteLog retrieves a write log between two storage instances from the database.
	GetWriteLog(ctx context.Context, startRoot, endRoot node.Root) (writelog.Iterator, error)

	// GetLatestVersion returns the most recent version in the node database.
	GetLatestVersion(ctx context.Context) (uint64, error)

	// GetEarliestVersion returns the earliest version in the node database.
	GetEarliestVersion(ctx context.Context) (uint64, error)

	// GetRootsForVersion returns a list of roots stored under the given version.
	GetRootsForVersion(ctx context.Context, version uint64) ([]node.Root, error)

	// StartMultipartInsert prepares the database for a batch insert job from multiple chunks.
	// Batches from this call onwards will keep track of inserted nodes so that they can be
	// deleted if the job fails for any reason.
	StartMultipartInsert(version uint64) error

	// AbortMultipartInsert cleans up the node insertion log that was kept since the last
	// StartMultipartInsert operation. The log will be cleared and the associated nodes can
	// be either removed (if the insertion failed) or left intact (if it was successful).
	//
	// It is not an error to call this method more than once.
	AbortMultipartInsert() error

	// NewBatch starts a new batch.
	//
	// The chunk argument specifies whether the given batch is being used to import a chunk of an
	// existing root. Chunks may contain unresolved pointers (e.g., pointers that point to hashes
	// which are not present in the database). Committing a chunk batch will prevent the version
	// from being finalized.
	NewBatch(oldRoot node.Root, version uint64, chunk bool) (Batch, error)

	// HasRoot checks whether the given root exists.
	HasRoot(root node.Root) bool

	// Finalize finalizes the version comprising the passed list of finalized roots.
	// All non-finalized roots can be discarded.
	Finalize(ctx context.Context, roots []node.Root) error

	// Prune removes all roots recorded under the given version.
	//
	// Only the earliest version can be pruned, passing any other version will result in an error.
	Prune(ctx context.Context, version uint64) error

	// Size returns the size of the database in bytes.
	Size() (int64, error)

	// Sync syncs the database to disk. This is useful if the NoFsync option is used to explicitly
	// perform a sync.
	Sync() error

	// Close closes the database.
	Close()
}

NodeDB is the persistence layer used for persisting the in-memory tree.

func NewNopNodeDB

func NewNopNodeDB() (NodeDB, error)

NewNopNodeDB creates a new no-op node database.

type NodeVisitor

type NodeVisitor func(context.Context, node.Node) bool

NodeVisitor is a function that visits a given node and returns true to continue traversal of child nodes or false to stop.

type Subtree

type Subtree interface {
	// PutNode persists a node in the NodeDB.
	//
	// Depth is the node depth not bit depth.
	PutNode(depth node.Depth, ptr *node.Pointer) error

	// VisitCleanNode is called for any clean node encountered during commit
	// for which no further processing will be done (as it is marked clean).
	//
	// The specific NodeDB implementation may wish to do further processing.
	//
	// Depth is the node depth not bit depth.
	VisitCleanNode(depth node.Depth, ptr *node.Pointer) error

	// Commit marks the subtree as complete.
	Commit() error
}

Subtree is a NodeDB-specific subtree implementation.