store

package module
v2.0.0-...-4666e1d Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Sep 21, 2024 License: Apache-2.0 Imports: 6 Imported by: 3

README

Store

The store package contains the implementation of store/v2, which is the SDK's abstraction around managing historical and committed state. See ADR-065 and Store v2 Design for a high-level overview of the design and rationale.

Usage

The store package contains a root.Store type which is intended to act as an abstraction layer around it's two primary constituent components - state storage (SS) and state commitment (SC). It acts as the main entry point into storage for an application to use in server/v2. Through root.Store, an application can query and iterate over both current and historical data, commit new state, perform state sync, and fetch commitment proofs.

A root.Store is intended to be initialized with already constructed SS and SC backends (see relevant package documentation for instantiation details). Note, from the perspective of root.Store, there is no notion of multi or single tree/store, rather these are implementation details of SS and SC. For SS, we utilize store keys to namespace raw key/value pairs. For SC, we utilize an abstraction, commitment.CommitStore, to map store keys to a commitment trees.

Upgrades

The LoadVersionAndUpgrade API of the root.store allows for adding or removing store keys. This is useful for upgrading the chain with new modules or removing old ones. The Rename feature is not supported in store/v2.

sequenceDiagram
    participant S as Store
    participant SS as StateStorage
    participant SC as StateCommitment
    alt SC is a UpgradeableStore
        S->>SC: LoadVersionAndUpgrade
        SC->>SC: Mount new store keys
        SC->>SC: Prune removed store keys
    end
    SC->>S: LoadVersion Result
    alt SS is a UpgradableDatabase
        S->>SS: PruneStoreKeys
    end

Prune store keys does not remove the data from the SC and SS instantly. It only marks the store keys as pruned. The actual data removal is done by the pruning process of the underlying SS and SC.

Migration

Pruning

The root.Store is NOT responsible for pruning. Rather, pruning is the responsibility of the underlying SS and SC layers. This means pruning can be implementation specific, such as being synchronous or asynchronous.

Test Coverage

The test coverage of the following logical components should be over 60%:

Documentation

Index

Constants

This section is empty.

Variables

View Source 
var (
	// MaxKeyLength is the maximum allowed length for a key in bytes.
	// It is set to 128K - 1 (131,071 bytes).
	MaxKeyLength = (1 << 17) - 1

	// MaxValueLength is the maximum allowed length for a value in bytes.
	// It is set to 2G - 1 (2,147,483,647 bytes).
	MaxValueLength = (1 << 31) - 1
)

Functions

func AssertValidKey 

func AssertValidKey(key []byte)

AssertValidKey checks if the key is valid, i.e. key is not nil, not empty and within length limit.

func AssertValidValue 

func AssertValidValue(value []byte)

AssertValidValue checks if the value is valid, i.e. value is not nil and within length limit.

Types

type Batch 

type Batch interface {
	// Set inserts the given value into the key-value data store.
	//
	// Note: <key, value> are safe to modify and read after calling Set.
	Set(storeKey, key, value []byte) error

	// Delete removes the key from the backing key-value data store.
	//
	// Note: <key> is safe to modify and read after calling Delete.
	Delete(storeKey, key []byte) error

	// Size retrieves the amount of data queued up for writing, this includes
	// the keys, values, and deleted keys.
	Size() int

	// Write flushes any accumulated data to disk.
	Write() error

	// Reset resets the batch.
	Reset() error
}

Batch is a write-only database that commits changes to the underlying database when Write is called. A batch cannot be used concurrently.

type Committer 

type Committer interface {
	// WriteChangeset writes the changeset to the commitment state.
	WriteChangeset(cs *corestore.Changeset) error

	// WorkingCommitInfo returns the CommitInfo for the working tree.
	WorkingCommitInfo(version uint64) *proof.CommitInfo

	// GetLatestVersion returns the latest version.
	GetLatestVersion() (uint64, error)

	// LoadVersion loads the tree at the given version.
	LoadVersion(targetVersion uint64) error

	// Commit commits the working tree to the database.
	Commit(version uint64) (*proof.CommitInfo, error)

	// GetProof returns the proof of existence or non-existence for the given key.
	GetProof(storeKey []byte, version uint64, key []byte) ([]proof.CommitmentOp, error)

	// Get returns the value for the given key at the given version.
	//
	// NOTE: This method only exists to support migration from IAVL v0/v1 to v2.
	// Once migration is complete, this method should be removed and/or not used.
	Get(storeKey []byte, version uint64, key []byte) ([]byte, error)

	// SetInitialVersion sets the initial version of the committer.
	SetInitialVersion(version uint64) error

	// GetCommitInfo returns the CommitInfo for the given version.
	GetCommitInfo(version uint64) (*proof.CommitInfo, error)

	// Close releases associated resources. It should NOT be idempotent. It must
	// only be called once and any call after may panic.
	io.Closer
}

Committer defines an API for committing state.

type DBOptions 

type DBOptions interface {
	Get(string) interface{}
}

DBOptions defines the interface of a database options.

type PausablePruner 

type PausablePruner interface {
	Pruner

	// PausePruning pauses or resumes the pruning process to avoid the parallel writes
	// while committing the state.
	PausePruning(pause bool)
}

PausablePruner extends the Pruner interface to include the API for pausing the pruning process.

type Pruner 

type Pruner interface {
	// Prune prunes the store to the provided version.
	Prune(version uint64) error
}

Pruner defines the interface for pruning old versions of the store or database.

type PruningOption 

type PruningOption struct {
	// KeepRecent sets the number of recent versions to keep.
	KeepRecent uint64 `mapstructure:"keep-recent" toml:"keep-recent" comment:"Number of recent heights to keep on disk."`

	// Interval sets the number of how often to prune.
	// If set to 0, no pruning will be done.
	Interval uint64 `mapstructure:"interval" toml:"interval" comment:"Height interval at which pruned heights are removed from disk."`
}

PruningOption defines the pruning configuration. app.toml config options

func NewPruningOption 

func NewPruningOption(pruningStrategy PruningStrategy) *PruningOption

NewPruningOption returns a new PruningOption instance based on the given pruning strategy.

func NewPruningOptionWithCustom 

func NewPruningOptionWithCustom(keepRecent, interval uint64) *PruningOption

NewPruningOptionWithCustom returns a new PruningOption based on the given parameters.

func (*PruningOption) ShouldPrune 

func (opts *PruningOption) ShouldPrune(version uint64) (bool, uint64)

ShouldPrune returns true if the given version should be pruned. If true, it also returns the version to prune up to. NOTE: The current version is not pruned.

type PruningStrategy 

type PruningStrategy int
const (
	// PruningDefault defines a pruning strategy where the last 362880 heights are
	// kept where to-be pruned heights are pruned at every 10th height.
	// The last 362880 heights are kept(approximately 3.5 weeks worth of state) assuming the typical
	// block time is 6s. If these values do not match the applications' requirements, use the "custom" option.
	PruningDefault PruningStrategy = iota
	// PruningEverything defines a pruning strategy where all committed heights are
	// deleted, storing only the current height and last 2 states. To-be pruned heights are
	// pruned at every 10th height.
	PruningEverything
	// PruningNothing defines a pruning strategy where all heights are kept on disk.
	// This is the only stretegy where KeepEvery=1 is allowed with state-sync snapshots disabled.
	PruningNothing
)

type QueryResult 

type QueryResult struct {
	Key      []byte
	Value    []byte
	Version  uint64
	ProofOps []proof.CommitmentOp
}

QueryResult defines the response type to performing a query on a RootStore.

type RootStore 

type RootStore interface {
	// StateLatest returns a read-only version of the RootStore at the latest
	// height, alongside the associated version.
	StateLatest() (uint64, corestore.ReaderMap, error)

	// StateAt is analogous to StateLatest() except it returns a read-only version
	// of the RootStore at the provided version. If such a version cannot be found,
	// an error must be returned.
	StateAt(version uint64) (corestore.ReaderMap, error)

	// GetStateStorage returns the SS backend.
	GetStateStorage() VersionedDatabase

	// GetStateCommitment returns the SC backend.
	GetStateCommitment() Committer

	// Query performs a query on the RootStore for a given store key, version (height),
	// and key tuple. Queries should be routed to the underlying SS engine.
	Query(storeKey []byte, version uint64, key []byte, prove bool) (QueryResult, error)

	// LoadVersion loads the RootStore to the given version.
	LoadVersion(version uint64) error

	// LoadLatestVersion behaves identically to LoadVersion except it loads the
	// latest version implicitly.
	LoadLatestVersion() error

	// GetLatestVersion returns the latest version, i.e. height, committed.
	GetLatestVersion() (uint64, error)

	// SetInitialVersion sets the initial version on the RootStore.
	SetInitialVersion(v uint64) error

	// SetCommitHeader sets the commit header for the next commit. This call and
	// implementation is optional. However, it must be supported in cases where
	// queries based on block time need to be supported.
	SetCommitHeader(h *coreheader.Info)

	// WorkingHash returns the current WIP commitment hash by applying the Changeset
	// to the SC backend. It is only used to get the hash of the intermediate state
	// before committing, the typical use case is for the genesis block.
	// NOTE: It also writes the changeset to the SS backend.
	WorkingHash(cs *corestore.Changeset) ([]byte, error)

	// Commit should be responsible for taking the provided changeset and flushing
	// it to disk. Note, it will overwrite the changeset if WorkingHash() was called.
	// Commit() should ensure the changeset is committed to all SC and SS backends
	// and flushed to disk. It must return a hash of the merkle-ized committed state.
	Commit(cs *corestore.Changeset) ([]byte, error)

	// LastCommitID returns a CommitID pertaining to the last commitment.
	LastCommitID() (proof.CommitID, error)

	// SetMetrics sets the telemetry handler on the RootStore.
	SetMetrics(m metrics.Metrics)

	Prune(version uint64) error

	io.Closer
}

RootStore defines an abstraction layer containing a State Storage (SS) engine and one or more State Commitment (SC) engines.

type TraceContext 

type TraceContext map[string]any

TraceContext contains KVStore context data. It will be written with every trace operation.

func (TraceContext) Clone 

func (tc TraceContext) Clone() TraceContext

Clone creates a shallow clone of a TraceContext.

func (TraceContext) Merge 

func (tc TraceContext) Merge(newTc TraceContext) TraceContext

Merge merges the receiver TraceContext with the provided TraceContext argument.

type UpgradableDatabase 

type UpgradableDatabase interface {
	// PruneStoreKeys prunes all data associated with the given storeKeys whenever
	// the given version is pruned.
	PruneStoreKeys(storeKeys []string, version uint64) error
}

UpgradableDatabase defines an API for a versioned database that allows pruning deleted storeKeys

type UpgradeableStore 

type UpgradeableStore interface {
	// LoadVersionAndUpgrade behaves identically to LoadVersion except it also
	// accepts a StoreUpgrades object that defines a series of transformations to
	// apply to store keys (if any).
	//
	// Note, handling StoreUpgrades is optional depending on the underlying store
	// implementation.
	LoadVersionAndUpgrade(version uint64, upgrades *corestore.StoreUpgrades) error
}

UpgradeableStore defines the interface for upgrading store keys.

type VersionedDatabase 

type VersionedDatabase interface {
	Has(storeKey []byte, version uint64, key []byte) (bool, error)
	Get(storeKey []byte, version uint64, key []byte) ([]byte, error)
	GetLatestVersion() (uint64, error)
	SetLatestVersion(version uint64) error

	Iterator(storeKey []byte, version uint64, start, end []byte) (corestore.Iterator, error)
	ReverseIterator(storeKey []byte, version uint64, start, end []byte) (corestore.Iterator, error)

	ApplyChangeset(version uint64, cs *corestore.Changeset) error

	// Close releases associated resources. It should NOT be idempotent. It must
	// only be called once and any call after may panic.
	io.Closer
}

VersionedDatabase defines an API for a versioned database that allows reads, writes, iteration and commitment over a series of versions.

Source Files

View all Source files

Directories

Path Synopsis
iavl
mem
conv
Package conv provides internal functions for conversions and data manipulation
 Package conv provides internal functions for conversions and data manipulation
encoding
Package mock is a generated GoMock package.
 Package mock is a generated GoMock package.
types
pebbledb
rocksdb
sqlite
util

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.