README

Tendermint DB

version license API Reference codecov Lint Test Discord chat

Common database interface for various database backends. Primarily meant for applications built on Tendermint, such as the Cosmos SDK, but can be used independently of these as well.

Minimum Go Version

Go 1.13+

Supported Database Backends

  • GoLevelDB [stable]: A pure Go implementation of LevelDB (see below). Currently the default on-disk database used in the Cosmos SDK.

  • MemDB [stable]: An in-memory database using Google's B-tree package. Has very high performance both for reads, writes, and range scans, but is not durable and will lose all data on process exit. Does not support transactions. Suitable for e.g. caches, working sets, and tests. Used for IAVL working sets when the pruning strategy allows it.

  • LevelDB [experimental]: A Go wrapper around LevelDB. Uses LSM-trees for on-disk storage, which have good performance for write-heavy workloads, particularly on spinning disks, but requires periodic compaction to maintain decent read performance and reclaim disk space. Does not support transactions.

  • BoltDB [experimental]: A fork of BoltDB. Uses B+trees for on-disk storage, which have good performance for read-heavy workloads and range scans. Supports serializable ACID transactions.

  • RocksDB [experimental]: A Go wrapper around RocksDB. Similarly to LevelDB (above) it uses LSM-trees for on-disk storage, but is optimized for fast storage media such as SSDs and memory. Supports atomic transactions, but not full ACID transactions.

  • BadgerDB [experimental]: A key-value database written as a pure-Go alternative to e.g. LevelDB and RocksDB, with LSM-tree storage. Makes use of multiple goroutines for performance, and includes advanced features such as serializable ACID transactions, write batches, compression, and more.

Meta-databases

  • PrefixDB [stable]: A database which wraps another database and uses a static prefix for all keys. This allows multiple logical databases to be stored in a common underlying databases by using different namespaces. Used by the Cosmos SDK to give different modules their own namespaced database in a single application database.

  • RemoteDB [experimental]: A database that connects to distributed Tendermint db instances via gRPC. This can help with detaching difficult deployments such as LevelDB, and can also ease dependency management for Tendermint developers.

Tests

To test common databases, run make test. If all databases are available on the local machine, use make test-all to test them all.

Expand ▾ Collapse ▴

Documentation

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

func FileExists

func FileExists(filePath string) bool

func IsKeyInDomain

func IsKeyInDomain(key, start, end []byte) bool

    See DB interface documentation for more information.

    Types

    type BackendType

    type BackendType string
    const (
    	// GoLevelDBBackend represents goleveldb (github.com/syndtr/goleveldb - most
    	// popular implementation)
    	//   - pure go
    	//   - stable
    	GoLevelDBBackend BackendType = "goleveldb"
    	// CLevelDBBackend represents cleveldb (uses levigo wrapper)
    	//   - fast
    	//   - requires gcc
    	//   - use cleveldb build tag (go build -tags cleveldb)
    	CLevelDBBackend BackendType = "cleveldb"
    	// MemDBBackend represents in-memory key value store, which is mostly used
    	// for testing.
    	MemDBBackend BackendType = "memdb"
    	// BoltDBBackend represents bolt (uses etcd's fork of bolt -
    	// github.com/etcd-io/bbolt)
    	//   - EXPERIMENTAL
    	//   - may be faster is some use-cases (random reads - indexer)
    	//   - use boltdb build tag (go build -tags boltdb)
    	BoltDBBackend BackendType = "boltdb"
    	// RocksDBBackend represents rocksdb (uses github.com/tecbot/gorocksdb)
    	//   - EXPERIMENTAL
    	//   - requires gcc
    	//   - use rocksdb build tag (go build -tags rocksdb)
    	RocksDBBackend BackendType = "rocksdb"
    
    	BadgerDBBackend BackendType = "badgerdb"
    )

      These are valid backend types.

      type Batch

      type Batch interface {
      	// Set sets a key/value pair.
      	// CONTRACT: key, value readonly []byte
      	Set(key, value []byte) error
      
      	// Delete deletes a key/value pair.
      	// CONTRACT: key readonly []byte
      	Delete(key []byte) error
      
      	// Write writes the batch, possibly without flushing to disk. Only Close() can be called after,
      	// other methods will error.
      	Write() error
      
      	// WriteSync writes the batch and flushes it to disk. Only Close() can be called after, other
      	// methods will error.
      	WriteSync() error
      
      	// Close closes the batch. It is idempotent, but calls to other methods afterwards will error.
      	Close() error
      }

        Batch represents a group of writes. They may or may not be written atomically depending on the backend. Callers must call Close on the batch when done.

        As with DB, given keys and values should be considered read-only, and must not be modified after passing them to the batch.

        type DB

        type DB interface {
        	// Get fetches the value of the given key, or nil if it does not exist.
        	// CONTRACT: key, value readonly []byte
        	Get([]byte) ([]byte, error)
        
        	// Has checks if a key exists.
        	// CONTRACT: key, value readonly []byte
        	Has(key []byte) (bool, error)
        
        	// Set sets the value for the given key, replacing it if it already exists.
        	// CONTRACT: key, value readonly []byte
        	Set([]byte, []byte) error
        
        	// SetSync sets the value for the given key, and flushes it to storage before returning.
        	SetSync([]byte, []byte) error
        
        	// Delete deletes the key, or does nothing if the key does not exist.
        	// CONTRACT: key readonly []byte
        	Delete([]byte) error
        
        	// DeleteSync deletes the key, and flushes the delete to storage before returning.
        	DeleteSync([]byte) error
        
        	// Iterator returns an iterator over a domain of keys, in ascending order. The caller must call
        	// Close when done. End is exclusive, and start must be less than end. A nil start iterates
        	// from the first key, and a nil end iterates to the last key (inclusive). Empty keys are not
        	// valid.
        	// CONTRACT: No writes may happen within a domain while an iterator exists over it.
        	// CONTRACT: start, end readonly []byte
        	Iterator(start, end []byte) (Iterator, error)
        
        	// ReverseIterator returns an iterator over a domain of keys, in descending order. The caller
        	// must call Close when done. End is exclusive, and start must be less than end. A nil end
        	// iterates from the last key (inclusive), and a nil start iterates to the first key (inclusive).
        	// Empty keys are not valid.
        	// CONTRACT: No writes may happen within a domain while an iterator exists over it.
        	// CONTRACT: start, end readonly []byte
        	ReverseIterator(start, end []byte) (Iterator, error)
        
        	// Close closes the database connection.
        	Close() error
        
        	// NewBatch creates a batch for atomic updates. The caller must call Batch.Close.
        	NewBatch() Batch
        
        	// Print is used for debugging.
        	Print() error
        
        	// Stats returns a map of property values for all keys and the size of the cache.
        	Stats() map[string]string
        }

          DB is the main interface for all database backends. DBs are concurrency-safe. Callers must call Close on the database when done.

          Keys cannot be nil or empty, while values cannot be nil. Keys and values should be considered read-only, both when returned and when given, and must be copied before they are modified.

          func NewDB

          func NewDB(name string, backend BackendType, dir string) (DB, error)

            NewDB creates a new database of type backend with the given name.

            type GoLevelDB

            type GoLevelDB struct {
            	// contains filtered or unexported fields
            }

            func NewGoLevelDB

            func NewGoLevelDB(name string, dir string) (*GoLevelDB, error)

            func NewGoLevelDBWithOpts

            func NewGoLevelDBWithOpts(name string, dir string, o *opt.Options) (*GoLevelDB, error)

            func (*GoLevelDB) Close

            func (db *GoLevelDB) Close() error

              Close implements DB.

              func (*GoLevelDB) DB

              func (db *GoLevelDB) DB() *leveldb.DB

              func (*GoLevelDB) Delete

              func (db *GoLevelDB) Delete(key []byte) error

                Delete implements DB.

                func (*GoLevelDB) DeleteSync

                func (db *GoLevelDB) DeleteSync(key []byte) error

                  DeleteSync implements DB.

                  func (*GoLevelDB) Get

                  func (db *GoLevelDB) Get(key []byte) ([]byte, error)

                    Get implements DB.

                    func (*GoLevelDB) Has

                    func (db *GoLevelDB) Has(key []byte) (bool, error)

                      Has implements DB.

                      func (*GoLevelDB) Iterator

                      func (db *GoLevelDB) Iterator(start, end []byte) (Iterator, error)

                        Iterator implements DB.

                        func (*GoLevelDB) NewBatch

                        func (db *GoLevelDB) NewBatch() Batch

                          NewBatch implements DB.

                          func (*GoLevelDB) Print

                          func (db *GoLevelDB) Print() error

                            Print implements DB.

                            func (*GoLevelDB) ReverseIterator

                            func (db *GoLevelDB) ReverseIterator(start, end []byte) (Iterator, error)

                              ReverseIterator implements DB.

                              func (*GoLevelDB) Set

                              func (db *GoLevelDB) Set(key []byte, value []byte) error

                                Set implements DB.

                                func (*GoLevelDB) SetSync

                                func (db *GoLevelDB) SetSync(key []byte, value []byte) error

                                  SetSync implements DB.

                                  func (*GoLevelDB) Stats

                                  func (db *GoLevelDB) Stats() map[string]string

                                    Stats implements DB.

                                    type Iterator

                                    type Iterator interface {
                                    	// Domain returns the start (inclusive) and end (exclusive) limits of the iterator.
                                    	// CONTRACT: start, end readonly []byte
                                    	Domain() (start []byte, end []byte)
                                    
                                    	// Valid returns whether the current iterator is valid. Once invalid, the Iterator remains
                                    	// invalid forever.
                                    	Valid() bool
                                    
                                    	// Next moves the iterator to the next key in the database, as defined by order of iteration.
                                    	// If Valid returns false, this method will panic.
                                    	Next()
                                    
                                    	// Key returns the key at the current position. Panics if the iterator is invalid.
                                    	// CONTRACT: key readonly []byte
                                    	Key() (key []byte)
                                    
                                    	// Value returns the value at the current position. Panics if the iterator is invalid.
                                    	// CONTRACT: value readonly []byte
                                    	Value() (value []byte)
                                    
                                    	// Error returns the last error encountered by the iterator, if any.
                                    	Error() error
                                    
                                    	// Close closes the iterator, relasing any allocated resources.
                                    	Close() error
                                    }

                                      Iterator represents an iterator over a domain of keys. Callers must call Close when done. No writes can happen to a domain while there exists an iterator over it, some backends may take out database locks to ensure this will not happen.

                                      Callers must make sure the iterator is valid before calling any methods on it, otherwise these methods will panic. This is in part caused by most backend databases using this convention.

                                      As with DB, keys and values should be considered read-only, and must be copied before they are modified.

                                      Typical usage:

                                      var itr Iterator = ... defer itr.Close()

                                      for ; itr.Valid(); itr.Next() {

                                      k, v := itr.Key(); itr.Value()
                                      ...
                                      

                                      } if err := itr.Error(); err != nil {

                                      ...
                                      

                                      }

                                      func IteratePrefix

                                      func IteratePrefix(db DB, prefix []byte) (Iterator, error)

                                        IteratePrefix is a convenience function for iterating over a key domain restricted by prefix.

                                        type MemDB

                                        type MemDB struct {
                                        	// contains filtered or unexported fields
                                        }

                                          MemDB is an in-memory database backend using a B-tree for storage.

                                          For performance reasons, all given and returned keys and values are pointers to the in-memory database, so modifying them will cause the stored values to be modified as well. All DB methods already specify that keys and values should be considered read-only, but this is especially important with MemDB.

                                          func NewMemDB

                                          func NewMemDB() *MemDB

                                            NewMemDB creates a new in-memory database.

                                            func (*MemDB) Close

                                            func (db *MemDB) Close() error

                                              Close implements DB.

                                              func (*MemDB) Delete

                                              func (db *MemDB) Delete(key []byte) error

                                                Delete implements DB.

                                                func (*MemDB) DeleteSync

                                                func (db *MemDB) DeleteSync(key []byte) error

                                                  DeleteSync implements DB.

                                                  func (*MemDB) Get

                                                  func (db *MemDB) Get(key []byte) ([]byte, error)

                                                    Get implements DB.

                                                    func (*MemDB) Has

                                                    func (db *MemDB) Has(key []byte) (bool, error)

                                                      Has implements DB.

                                                      func (*MemDB) Iterator

                                                      func (db *MemDB) Iterator(start, end []byte) (Iterator, error)

                                                        Iterator implements DB. Takes out a read-lock on the database until the iterator is closed.

                                                        func (*MemDB) NewBatch

                                                        func (db *MemDB) NewBatch() Batch

                                                          NewBatch implements DB.

                                                          func (*MemDB) Print

                                                          func (db *MemDB) Print() error

                                                            Print implements DB.

                                                            func (*MemDB) ReverseIterator

                                                            func (db *MemDB) ReverseIterator(start, end []byte) (Iterator, error)

                                                              ReverseIterator implements DB. Takes out a read-lock on the database until the iterator is closed.

                                                              func (*MemDB) Set

                                                              func (db *MemDB) Set(key []byte, value []byte) error

                                                                Set implements DB.

                                                                func (*MemDB) SetSync

                                                                func (db *MemDB) SetSync(key []byte, value []byte) error

                                                                  SetSync implements DB.

                                                                  func (*MemDB) Stats

                                                                  func (db *MemDB) Stats() map[string]string

                                                                    Stats implements DB.

                                                                    type PrefixDB

                                                                    type PrefixDB struct {
                                                                    	// contains filtered or unexported fields
                                                                    }

                                                                      PrefixDB wraps a namespace of another database as a logical database.

                                                                      func NewPrefixDB

                                                                      func NewPrefixDB(db DB, prefix []byte) *PrefixDB

                                                                        NewPrefixDB lets you namespace multiple DBs within a single DB.

                                                                        func (*PrefixDB) Close

                                                                        func (pdb *PrefixDB) Close() error

                                                                          Close implements DB.

                                                                          func (*PrefixDB) Delete

                                                                          func (pdb *PrefixDB) Delete(key []byte) error

                                                                            Delete implements DB.

                                                                            func (*PrefixDB) DeleteSync

                                                                            func (pdb *PrefixDB) DeleteSync(key []byte) error

                                                                              DeleteSync implements DB.

                                                                              func (*PrefixDB) Get

                                                                              func (pdb *PrefixDB) Get(key []byte) ([]byte, error)

                                                                                Get implements DB.

                                                                                func (*PrefixDB) Has

                                                                                func (pdb *PrefixDB) Has(key []byte) (bool, error)

                                                                                  Has implements DB.

                                                                                  func (*PrefixDB) Iterator

                                                                                  func (pdb *PrefixDB) Iterator(start, end []byte) (Iterator, error)

                                                                                    Iterator implements DB.

                                                                                    func (*PrefixDB) NewBatch

                                                                                    func (pdb *PrefixDB) NewBatch() Batch

                                                                                      NewBatch implements DB.

                                                                                      func (*PrefixDB) Print

                                                                                      func (pdb *PrefixDB) Print() error

                                                                                        Print implements DB.

                                                                                        func (*PrefixDB) ReverseIterator

                                                                                        func (pdb *PrefixDB) ReverseIterator(start, end []byte) (Iterator, error)

                                                                                          ReverseIterator implements DB.

                                                                                          func (*PrefixDB) Set

                                                                                          func (pdb *PrefixDB) Set(key []byte, value []byte) error

                                                                                            Set implements DB.

                                                                                            func (*PrefixDB) SetSync

                                                                                            func (pdb *PrefixDB) SetSync(key []byte, value []byte) error

                                                                                              SetSync implements DB.

                                                                                              func (*PrefixDB) Stats

                                                                                              func (pdb *PrefixDB) Stats() map[string]string

                                                                                                Stats implements DB.

                                                                                                Directories

                                                                                                Path Synopsis
                                                                                                remotedb is a package for connecting to distributed Tendermint db.DB instances.
                                                                                                remotedb is a package for connecting to distributed Tendermint db.DB instances.
                                                                                                grpcdb
                                                                                                grpcdb is the distribution of Tendermint's db.DB instances using the gRPC transport to decouple local db.DB usages from applications, to using them over a network in a highly performant manner.
                                                                                                grpcdb is the distribution of Tendermint's db.DB instances using the gRPC transport to decouple local db.DB usages from applications, to using them over a network in a highly performant manner.