tan

README

Tan

Tan is a high performance database for storing Raft log and metadata.

Motivation

Early versions of Dragonboat employed RocksDB style LSM based Key-Value databases to store Raft log and metadata. Such Key-Value stores are easy to use but such convenience comes at huge costs -

• redundant MemTables
• redundant keys
• redundant serializations
• storage amplification
• write amplification
• read amplification
• expensive concurrent access control

Tan aims to overcome all these issues by providing a specifically designed database for storing Raft log and metadata.

License

Tan contains Pebble code and code derived from Pebble. Pebble itself is built from the golang version of Level-DB. Pebble, Level-DB and the golang version of Level-DB are all BSD licensed.

Documentation

Overview

Tan is a log file based LogDB implementation for dragonboat.

Each dragonboat instance owns a tan LogDB instance, which manages all tan db instances hold by a container instance called collection. Each raft node is backed one of those tan db instance.

To allow N raft nodes to share M tan db instance, there are two obvious ways to do it, known as the regular mode, one way is to let each raft node to own a dedicated tan db. Another way is to share the same tan db among multiple raft nodes so there won't be an excessive amount of tan db instances, we call this the multiplexed log mode. Such 1:1 and n:m mapping relationships are managed by a regularKeeper or a multiplexedKeeper intance both of which are of the dbKeeper interface as defined in db_keeper.go. This allows the upper layer to get the relevant tan db by just providing the shardID and replicaID values of the raft node with the mapping details hidden from the outside.

Each tan db instance owns a log file which will be used for storing all log data. For data written into the same tan db from different raft nodes, they will be indexed into different tan nodeIndex instances stored as a part of db.mu.nodeStates. This means each raft node will have its own nodeIndex.

Index

• Constants
• Variables
• func IsInvalidRecord(err error) bool
• type LogDB
  • func CreateLogMultiplexedTan(cfg config.NodeHostConfig, cb config.LogDBCallback, dirs []string, ...) (*LogDB, error)
  • func CreateTan(cfg config.NodeHostConfig, cb config.LogDBCallback, dirs []string, ...) (*LogDB, error)
  • func (l *LogDB) BinaryFormat() uint32
  • func (l *LogDB) Close() (err error)
  • func (l *LogDB) CompactEntriesTo(shardID uint64, replicaID uint64, index uint64) (<-chan struct{}, error)
  • func (l *LogDB) DeleteSnapshot(shardID uint64, replicaID uint64, index uint64) error
  • func (l *LogDB) GetBootstrapInfo(shardID uint64, replicaID uint64) (pb.Bootstrap, error)
  • func (l *LogDB) GetSnapshot(shardID uint64, replicaID uint64) (pb.Snapshot, error)
  • func (l *LogDB) ImportSnapshot(snapshot pb.Snapshot, replicaID uint64) error
  • func (l *LogDB) IterateEntries(ents []pb.Entry, size uint64, shardID uint64, replicaID uint64, low uint64, ...) ([]pb.Entry, uint64, error)
  • func (l *LogDB) ListNodeInfo() ([]raftio.NodeInfo, error)
  • func (l *LogDB) ListSnapshots(shardID uint64, replicaID uint64, index uint64) ([]pb.Snapshot, error)
  • func (l *LogDB) Name() string
  • func (l *LogDB) ReadRaftState(shardID uint64, replicaID uint64, lastIndex uint64) (raftio.RaftState, error)
  • func (l *LogDB) RemoveEntriesTo(shardID uint64, replicaID uint64, index uint64) error
  • func (l *LogDB) RemoveNodeData(shardID uint64, replicaID uint64) error
  • func (l *LogDB) SaveBootstrapInfo(shardID uint64, replicaID uint64, rec pb.Bootstrap) error
  • func (l *LogDB) SaveRaftState(updates []pb.Update, shardID uint64) error
  • func (l *LogDB) SaveSnapshots(updates []pb.Update) error
• type Options
  • func (o *Options) EnsureDefaults() *Options

Constants

const (
	// MaxManifestFileSize is the default max manifest file size
	MaxManifestFileSize int64 = 1024 * 1024 * 2
	// MaxLogFileSize is the default max log file size
	MaxLogFileSize int64 = 1024 * 1024 * 64
)

Variables

var (
	// ErrClosed is the error used to indicate that the db has already been closed
	ErrClosed = errors.New("db closed")
	// ErrNoBootstrap is the error used to indicate that there is no saved
	// bootstrap record
	ErrNoBootstrap = errors.New("no bootstrap info")
	// ErrNoState is the error indicating that there is no state record in the db
	ErrNoState = errors.New("no state record")
)

var (
	// ErrNotAnIOSeeker is returned if the io.Reader underlying a Reader does not implement io.Seeker.
	ErrNotAnIOSeeker = errors.New("reader does not implement io.Seeker")
	// ErrNoLastRecord is returned if LastRecordOffset is called and there is no previous record.
	ErrNoLastRecord = errors.New("no last record exists")
	// ErrZeroedChunk is returned if a chunk is encountered that is zeroed. This
	// usually occurs due to log file preallocation.
	ErrZeroedChunk = errors.New("zeroed chunk")
	// ErrInvalidChunk is returned if a chunk is encountered with an invalid
	// header, length, or checksum. This usually occurs when a log is recycled,
	// but can also occur due to corruption.
	ErrInvalidChunk = errors.New("invalid chunk")
	// ErrCRCMismatch is returned to indicate that CRC mismatch has been found.
	ErrCRCMismatch = errors.New("tan: crc mismatch")
)

var Factory = factory{}

Factory is the default LogDB factory instance used for creating tan DB instances.

var MultiplexedLogFactory = multiplexLogFactory{}

MultiplexedLogFactory is a LogDB factory instance used for creating an tan DB with multiplexed logs.

Functions

func IsInvalidRecord

func IsInvalidRecord(err error) bool

IsInvalidRecord returns true if the error matches one of the error types returned for invalid records. These are treated in a way similar to io.EOF in recovery code.

Types

type LogDB

type LogDB struct {
	// contains filtered or unexported fields
}

LogDB is the tan ILogDB type used to interface with dragonboat.

func CreateLogMultiplexedTan

func CreateLogMultiplexedTan(cfg config.NodeHostConfig, cb config.LogDBCallback,
	dirs []string, wals []string) (*LogDB, error)

CreateLogMultiplexedTan creates and returns a tan instance that uses multiplexed log files. A multiplexed log allow multiple raft shards to share the same underlying physical log file, this is required when you want to run thousands of raft nodes on the same server without having thousands action log files.

func CreateTan

func CreateTan(cfg config.NodeHostConfig, cb config.LogDBCallback,
	dirs []string, wals []string) (*LogDB, error)

CreateTan creates and return a regular tan instance. Each raft node will be backed by a dedicated log file.

func (*LogDB) BinaryFormat

func (l *LogDB) BinaryFormat() uint32

BinaryFormat returns an constant uint32 value representing the binary format version compatible with the ILogDB instance.

func (*LogDB) Close

func (l *LogDB) Close() (err error)

Close closes the ILogDB instance.

func (*LogDB) CompactEntriesTo

func (l *LogDB) CompactEntriesTo(shardID uint64,
	replicaID uint64, index uint64) (<-chan struct{}, error)

CompactEntriesTo reclaims underlying storage space used for storing entries up to the specified index.

func (*LogDB) DeleteSnapshot

func (l *LogDB) DeleteSnapshot(shardID uint64,
	replicaID uint64, index uint64) error

DeleteSnapshot ...

func (*LogDB) GetBootstrapInfo

func (l *LogDB) GetBootstrapInfo(shardID uint64,
	replicaID uint64) (pb.Bootstrap, error)

GetBootstrapInfo returns saved bootstrap info from log DB. It returns ErrNoBootstrapInfo when there is no previously saved bootstrap info for the specified node.

func (*LogDB) GetSnapshot

func (l *LogDB) GetSnapshot(shardID uint64,
	replicaID uint64) (pb.Snapshot, error)

GetSnapshot lists available snapshots associated with the specified Raft node for index range (0, index].

func (*LogDB) ImportSnapshot

func (l *LogDB) ImportSnapshot(snapshot pb.Snapshot, replicaID uint64) error

ImportSnapshot imports the specified snapshot by creating all required metadata in the logdb.

func (*LogDB) IterateEntries

func (l *LogDB) IterateEntries(ents []pb.Entry,
	size uint64, shardID uint64, replicaID uint64, low uint64,
	high uint64, maxSize uint64) ([]pb.Entry, uint64, error)

IterateEntries returns the continuous Raft log entries of the specified Raft node between the index value range of [low, high) up to a max size limit of maxSize bytes. It returns the located log entries, their total size in bytes and the occurred error.

func (*LogDB) ListNodeInfo

func (l *LogDB) ListNodeInfo() ([]raftio.NodeInfo, error)

ListNodeInfo lists all available NodeInfo found in the log DB.

func (*LogDB) ListSnapshots

func (l *LogDB) ListSnapshots(shardID uint64,
	replicaID uint64, index uint64) ([]pb.Snapshot, error)

ListSnapshots lists available snapshots associated with the specified Raft node for index range (0, index].

func (*LogDB) Name

func (l *LogDB) Name() string

Name returns the type name of the ILogDB instance.

func (*LogDB) ReadRaftState

func (l *LogDB) ReadRaftState(shardID uint64,
	replicaID uint64, lastIndex uint64) (raftio.RaftState, error)

ReadRaftState returns the persistented raft state found in Log DB.

func (*LogDB) RemoveEntriesTo

func (l *LogDB) RemoveEntriesTo(shardID uint64,
	replicaID uint64, index uint64) error

RemoveEntriesTo removes entries between (0, index].

func (*LogDB) RemoveNodeData

func (l *LogDB) RemoveNodeData(shardID uint64, replicaID uint64) error

RemoveNodeData removes all data associated with the specified node.

func (*LogDB) SaveBootstrapInfo

func (l *LogDB) SaveBootstrapInfo(shardID uint64,
	replicaID uint64, rec pb.Bootstrap) error

SaveBootstrapInfo saves the specified bootstrap info to the log DB.

func (*LogDB) SaveRaftState

func (l *LogDB) SaveRaftState(updates []pb.Update, shardID uint64) error

SaveRaftState atomically saves the Raft states, log entries and snapshots metadata found in the pb.Update list to the log DB.

func (*LogDB) SaveSnapshots

func (l *LogDB) SaveSnapshots(updates []pb.Update) error

SaveSnapshots saves all snapshot metadata found in the pb.Update list.

type Options

type Options struct {
	MaxLogFileSize      int64
	MaxManifestFileSize int64
	FS                  vfs.FS
}

Options is the option type used by tan

func (*Options) EnsureDefaults

func (o *Options) EnsureDefaults() *Options

EnsureDefaults ensures that the default values for all options are set if a valid value was not already specified. Returns the new options.