storage

package
v0.0.0-...-959c02d Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Apr 10, 2023 License: Apache-2.0 Imports: 19 Imported by: 0

Documentation

Index

Constants

This section is empty.

Variables

View Source 
var (
	// PrewriteMaxBackoff is max sleep time of the `pre-write` command.
	PrewriteMaxBackoff = atomic.NewUint64(40000)
	// CommitMaxBackoff is max sleep time of the 'commit' command
	CommitMaxBackoff = uint64(40000)
)
View Source 
var (
	ManagedLockTTL uint64 = 20000 // 20s
)

Global variable set by config file.

Functions

func BackoffErrReporter 

func BackoffErrReporter(site string) backoff.Notify

BackoffErrReporter reports backoff error events.

func IsTombstone 

func IsTombstone(val []byte) bool

IsTombstone returns whether the value is a tombstone.

func Open 

func Open(dirname string, options ...Option) (kv.Storage, error)

Open returns a new storage instance.

Types

type BatchBufferGetter 

type BatchBufferGetter interface {
	kv.Getter
	Len() int
}

BatchBufferGetter is the interface for BatchGet.

type BufferBatchGetter 

type BufferBatchGetter struct {
	// contains filtered or unexported fields
}

BufferBatchGetter is the type for BatchGet with MemBuffer.

func NewBufferBatchGetter 

func NewBufferBatchGetter(buffer BatchBufferGetter, snapshot kv.BatchGetter) *BufferBatchGetter

NewBufferBatchGetter creates a new BufferBatchGetter.

func (*BufferBatchGetter) BatchGet 

func (b *BufferBatchGetter) BatchGet(ctx context.Context, keys []kv.Key) (map[string][]byte, error)

BatchGet gets a batch of values.

type KVSnapshot 

type KVSnapshot struct {
	// contains filtered or unexported fields
}

KVSnapshot represent the MVCC snapshot of the low-level key/value store. All values read from the KVSnapshot will be checked via mvcc.Version. And only the committed key/values can be retrieved or iterated.

func (*KVSnapshot) BatchGet 

func (s *KVSnapshot) BatchGet(_ context.Context, keys []kv.Key) (map[string][]byte, error)

func (*KVSnapshot) Get 

func (s *KVSnapshot) Get(_ context.Context, key kv.Key) ([]byte, error)

Get implements the Snapshot interface.

func (*KVSnapshot) Iter 

func (s *KVSnapshot) Iter(lowerBound kv.Key, upperBound kv.Key) (kv.Iterator, error)

Iter implements the Snapshot interface.

func (*KVSnapshot) IterReverse 

func (s *KVSnapshot) IterReverse(lowerBound kv.Key, upperBound kv.Key) (kv.Iterator, error)

IterReverse implements the Snapshot interface.

func (*KVSnapshot) StartVer 

func (s *KVSnapshot) StartVer() kv.Version

StartVer implements the Snapshot interface.

type MemDB 

type MemDB struct {
	// This RWMutex only used to ensure memdbSnapGetter.Get will not race with
	// concurrent memdb.Set, memdb.SetWithFlags, memdb.Delete and memdb.UpdateFlags.
	sync.RWMutex
	// contains filtered or unexported fields
}

MemDB is rollbackable Red-Black Tree optimized for TiDB's transaction states buffer use scenario. You can think MemDB is a combination of two separate tree map, one for key => value and another for key => keyFlags.

The value map is rollbackable, that means you can use the `Staging`, `Release` and `Cleanup` API to safely modify KVs.

The flags map is not rollbackable. There are two types of flag, persistent and non-persistent. When discarding a newly added KV in `Cleanup`, the non-persistent flags will be cleared. If there are persistent flags associated with key, we will keep this key in node without value.

func (*MemDB) Checkpoint 

func (db *MemDB) Checkpoint() *MemDBCheckpoint

Checkpoint returns a checkpoint of MemDB.

func (*MemDB) Cleanup 

func (db *MemDB) Cleanup(h int)

Cleanup cleanup the resources referenced by the StagingHandle. If the changes are not published by `Release`, they will be discarded.

func (*MemDB) Delete 

func (db *MemDB) Delete(key kv.Key) error

Delete removes the entry for key k from kv store.

func (*MemDB) DeleteWithFlags 

func (db *MemDB) DeleteWithFlags(key []byte, ops ...kv.FlagsOp) error

DeleteWithFlags delete key with the given KeyFlags

func (*MemDB) Dirty 

func (db *MemDB) Dirty() bool

Dirty returns whether the root staging buffer is updated.

func (*MemDB) DiscardValues 

func (db *MemDB) DiscardValues()

DiscardValues releases the memory used by all values. NOTE: any operation need value will panic after this function.

func (*MemDB) Get 

func (db *MemDB) Get(_ context.Context, key kv.Key) ([]byte, error)

Get gets the value for key k from kv store. If corresponding kv pair does not exist, it returns nil and ErrNotExist.

func (*MemDB) GetFlags 

func (db *MemDB) GetFlags(key []byte) (kv.KeyFlags, error)

GetFlags returns the latest flags associated with key.

func (*MemDB) GetKeyByHandle 

func (db *MemDB) GetKeyByHandle(handle MemKeyHandle) []byte

GetKeyByHandle returns key by handle.

func (*MemDB) GetValueByHandle 

func (db *MemDB) GetValueByHandle(handle MemKeyHandle) ([]byte, bool)

GetValueByHandle returns value by handle.

func (*MemDB) InspectStage 

func (db *MemDB) InspectStage(handle int, f func([]byte, kv.KeyFlags, []byte))

InspectStage used to inspect the value updates in the given stage.

func (*MemDB) Iter 

func (db *MemDB) Iter(lowerBound, upperBound kv.Key) (kv.Iterator, error)

Iter creates an Iterator positioned on the first entry that k <= entry's key. If such entry is not found, it returns an invalid Iterator with no error. It yields only keys that < upperBound. If upperBound is nil, it means the upperBound is unbounded. The Iterator must be Closed after use.

func (*MemDB) IterReverse 

func (db *MemDB) IterReverse(lowerBound, upperBound kv.Key) (kv.Iterator, error)

IterReverse creates a reversed Iterator positioned on the first entry which key is less than k. The returned SnapshotIter will iterate from greater key to smaller key. If k is nil, the returned SnapshotIter will be positioned at the last key.

func (*MemDB) IterReverseWithFlags 

func (db *MemDB) IterReverseWithFlags(k []byte) *MemDBIter

IterReverseWithFlags returns a reversed MemDBIter.

func (*MemDB) IterWithFlags 

func (db *MemDB) IterWithFlags(lowerBound []byte, upperBound []byte) *MemDBIter

IterWithFlags returns a MemDBIter.

func (*MemDB) Len 

func (db *MemDB) Len() int

Len returns the number of entries in the DB.

func (*MemDB) Mem 

func (db *MemDB) Mem() uint64

Mem returns the current memory footprint

func (*MemDB) Release 

func (db *MemDB) Release(h int)

Release publish all modifications in the latest staging buffer to upper level.

func (*MemDB) RemoveFromBuffer 

func (db *MemDB) RemoveFromBuffer(key []byte)

RemoveFromBuffer removes a record from the mem buffer. It should be only used for test.

func (*MemDB) Reset 

func (db *MemDB) Reset()

Reset resets the MemBuffer to initial states.

func (*MemDB) RevertToCheckpoint 

func (db *MemDB) RevertToCheckpoint(cp *MemDBCheckpoint)

RevertToCheckpoint reverts the MemDB to the checkpoint.

func (*MemDB) SelectValueHistory 

func (db *MemDB) SelectValueHistory(key []byte, predicate func(value []byte) bool) ([]byte, error)

SelectValueHistory select the latest value which makes `predicate` returns true from the modification history.

func (*MemDB) Set 

func (db *MemDB) Set(key []byte, value []byte) error

Set sets the value for key k as v into kv store. v must NOT be nil or empty, otherwise it returns ErrCannotSetNilValue.

func (*MemDB) SetMemoryFootprintChangeHook 

func (db *MemDB) SetMemoryFootprintChangeHook(hook func(uint64))

SetMemoryFootprintChangeHook sets the hook function that is triggered when memdb grows.

func (*MemDB) SetWithFlags 

func (db *MemDB) SetWithFlags(key []byte, value []byte, ops ...kv.FlagsOp) error

SetWithFlags put key-value into the last active staging buffer with the given KeyFlags.

func (*MemDB) Size 

func (db *MemDB) Size() int

Size returns sum of keys and values length.

func (*MemDB) SnapshotGetter 

func (db *MemDB) SnapshotGetter() kv.Getter

SnapshotGetter returns a Getter for a snapshot of MemBuffer.

func (*MemDB) SnapshotIter 

func (db *MemDB) SnapshotIter(start, end []byte) kv.Iterator

SnapshotIter returns a Iterator for a snapshot of MemBuffer.

func (*MemDB) Staging 

func (db *MemDB) Staging() int

Staging create a new staging buffer inside the MemBuffer. Subsequent writes will be temporarily stored in this new staging buffer. When you think all modifications looks good, you can call `Release` to public all of them to the upper level buffer.

func (*MemDB) UpdateFlags 

func (db *MemDB) UpdateFlags(key []byte, ops ...kv.FlagsOp)

UpdateFlags update the flags associated with key.

type MemDBCheckpoint 

type MemDBCheckpoint struct {
	// contains filtered or unexported fields
}

MemDBCheckpoint is the checkpoint of memory DB.

type MemDBIter 

type MemDBIter struct {
	// contains filtered or unexported fields
}

MemDBIter is an Iterator with KeyFlags related functions.

func (*MemDBIter) Close 

func (i *MemDBIter) Close()

Close closes the current SnapshotIter.

func (*MemDBIter) Flags 

func (i *MemDBIter) Flags() kv.KeyFlags

Flags returns flags belong to current SnapshotIter.

func (*MemDBIter) Handle 

func (i *MemDBIter) Handle() MemKeyHandle

Handle returns MemKeyHandle with the current position.

func (*MemDBIter) HasValue 

func (i *MemDBIter) HasValue() bool

HasValue returns false if it is flags only.

func (*MemDBIter) Key 

func (i *MemDBIter) Key() kv.Key

Key returns current key.

func (*MemDBIter) Next 

func (i *MemDBIter) Next() error

Next goes the next position.

func (*MemDBIter) UpdateFlags 

func (i *MemDBIter) UpdateFlags(ops ...kv.FlagsOp)

UpdateFlags updates and apply with flagsOp.

func (*MemDBIter) Valid 

func (i *MemDBIter) Valid() bool

Valid returns true if the current SnapshotIter is valid.

func (*MemDBIter) Value 

func (i *MemDBIter) Value() []byte

Value returns the value.

type MemKeyHandle 

type MemKeyHandle struct {
	// contains filtered or unexported fields
}

MemKeyHandle represents a pointer for key in MemBuffer.

type Option 

type Option func(options *pebble.Options)

type SnapshotIter 

type SnapshotIter struct {
	// contains filtered or unexported fields
}

SnapshotIter represents a iterator which provides a consistent view of key/value store.

func (*SnapshotIter) Close 

func (i *SnapshotIter) Close()

Close implements the Iterator interface.

func (*SnapshotIter) Key 

func (i *SnapshotIter) Key() kv.Key

Key implements the Iterator interface.

func (*SnapshotIter) Next 

func (i *SnapshotIter) Next() error

Next implements the Iterator interface.

func (*SnapshotIter) Valid 

func (i *SnapshotIter) Valid() bool

Valid implements the Iterator interface.

func (*SnapshotIter) Value 

func (i *SnapshotIter) Value() []byte

Value implements the Iterator interface.

type Txn 

type Txn struct {
	// contains filtered or unexported fields
}

Txn represents a transaction implemented beyond the low-level key/value storage.

func (*Txn) BatchGet 

func (txn *Txn) BatchGet(ctx context.Context, keys []kv.Key) (map[string][]byte, error)

BatchGet implements the Transaction interface. It gets kv from the memory buffer of statement and transaction, and the kv storage. Do not use len(value) == 0 or value == nil to represent non-exist. If a key doesn't exist, there shouldn't be any corresponding entry in the result map.

func (*Txn) Commit 

func (txn *Txn) Commit(_ context.Context) error

func (*Txn) Delete 

func (txn *Txn) Delete(k kv.Key) error

Delete implements the Transaction interface. It removes the entry for key k from kv store.

func (*Txn) Get 

func (txn *Txn) Get(ctx context.Context, k kv.Key) ([]byte, error)

Get implements the Transaction interface.

func (*Txn) Iter 

func (txn *Txn) Iter(lowerBound, upperBound kv.Key) (kv.Iterator, error)

Iter creates an Iterator positioned on the first entry that k <= entry's key. If such entry is not found, it returns an invalid Iterator with no error. It yields only keys that < upperBound. If upperBound is nil, it means the upperBound is unbounded. The Iterator must be Closed after use.

func (*Txn) IterReverse 

func (txn *Txn) IterReverse(lowerBound, upperBound kv.Key) (kv.Iterator, error)

IterReverse creates a reversed Iterator positioned on the first entry which key is less than k.

func (*Txn) Len 

func (txn *Txn) Len() int

Len implements the Transaction interface. It returns the number of entries in the DB.

func (*Txn) Reset 

func (txn *Txn) Reset()

Reset implements the Transaction interface. It resets the Transaction to initial states.

func (*Txn) Rollback 

func (txn *Txn) Rollback() error

Rollback implements the Transaction interface. It undoes the transaction operations to KV store.

func (*Txn) Set 

func (txn *Txn) Set(k kv.Key, v []byte) error

Set implements the Transaction interface. It sets the value for key k as v into kv store. v must NOT be nil or empty, otherwise it returns ErrCannotSetNilValue.

func (*Txn) Size 

func (txn *Txn) Size() int

Size implements the Transaction interface. It returns sum of keys and values length.

func (*Txn) Snapshot 

func (txn *Txn) Snapshot() kv.Snapshot

Snapshot implements the Transaction interface.

func (*Txn) StartVer 

func (txn *Txn) StartVer() kv.Version

StartVer implements the Transaction interface.

func (*Txn) String 

func (txn *Txn) String() string

String implements fmt.Stringer interface.

type UnionIter 

type UnionIter struct {
	// contains filtered or unexported fields
}

UnionIter is the SnapshotIter on an UnionStore.

func NewUnionIter 

func NewUnionIter(dirtyIt kv.Iterator, snapshotIt kv.Iterator, reverse bool) (*UnionIter, error)

NewUnionIter returns a union SnapshotIter for BufferStore.

func (*UnionIter) Close 

func (iter *UnionIter) Close()

Close implements the Iterator Close interface.

func (*UnionIter) Key 

func (iter *UnionIter) Key() kv.Key

Key implements the Iterator Key interface.

func (*UnionIter) Next 

func (iter *UnionIter) Next() error

Next implements the Iterator Next interface.

func (*UnionIter) Valid 

func (iter *UnionIter) Valid() bool

Valid implements the Iterator Valid interface.

func (*UnionIter) Value 

func (iter *UnionIter) Value() []byte

Value implements the Iterator Value interface. Multi columns

type UnionStore 

type UnionStore struct {
	// contains filtered or unexported fields
}

UnionStore is an in-memory Store which contains a buffer for write and a snapshot for read.

func NewUnionStore 

func NewUnionStore(snapshot kv.Snapshot) *UnionStore

NewUnionStore builds a new unionStore.

func (*UnionStore) Get 

func (us *UnionStore) Get(ctx context.Context, k kv.Key) ([]byte, error)

Get implements the Retriever interface.

func (*UnionStore) HasPresumeKeyNotExists 

func (us *UnionStore) HasPresumeKeyNotExists(k []byte) bool

HasPresumeKeyNotExists gets the key exist error info for the lazy check.

func (*UnionStore) Iter 

func (us *UnionStore) Iter(lowerBound, upperBound kv.Key) (kv.Iterator, error)

Iter implements the Retriever interface.

func (*UnionStore) IterReverse 

func (us *UnionStore) IterReverse(lowerBound, upperBound kv.Key) (kv.Iterator, error)

IterReverse implements the Retriever interface.

func (*UnionStore) MemBuffer 

func (us *UnionStore) MemBuffer() *MemDB

MemBuffer return the MemBuffer binding to this unionStore.

func (*UnionStore) SetEntrySizeLimit 

func (us *UnionStore) SetEntrySizeLimit(entryLimit, bufferLimit uint64)

SetEntrySizeLimit sets the size limit for each entry and total buffer.

func (*UnionStore) UnmarkPresumeKeyNotExists 

func (us *UnionStore) UnmarkPresumeKeyNotExists(k []byte)

UnmarkPresumeKeyNotExists deletes the key exist error info for the lazy check.

Source Files

View all Source files

Directories

Path Synopsis

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.