README
¶
toy-boltdb
(学習用)自作BoltDB - An embedded key/value database for Go
Bolt is a pure Go key/value store. This is a repository that I learn how databases work.
- Bolt uses a B+tree internally so there can be a lot of random page access.
- Bolt uses copy-on-write so old pages cannot be reclaimed while an old transaction is using them.
- Once the transaction has been committed or rolled back then the memory they point to can be reused by a new page or can be unmapped from virtual memory.
- Bolt uses an exclusive write lock on the database file so it cannot be shared by multiple processes.
References
Documentation
¶
Overview ¶
The underlying page and the page's parent pages into memory as "nodes". It's like a row, same as pageElement. We use the B+tree, so the "node" exists.
These nodes are where mutations occur during read-write transactions. These changes get flushed to disk during commit.
For more information on boltdb's page allocation, see [this comment](https://github.com/boltdb/bolt/issues/308#issuecomment-74811638).
|M|M|F|D| | | | | |
metadata pages (M), free pages (F), data (D), unallocated ( )
the page size (typically 4KB)
If you want to understand how database pages work generally, see [Database Pages — A deep dive](https://medium.com/@hnasr/database-pages-a-deep-dive-38cdb2c79eb5)
RWTx CreateBucket: db -> tx -> (page)(freelist allocate) -> bucket -> put to bucket obj
RWTx PUT: db -> tx -> bucket -> cursor -> node -> put to node obj
RWTx Commit: db -> tx -> (page)(freelist allocate, current size) -> write (node and bucket) obj to page -> write page to disk -> write meta to page and disk
Tx GET: db -> tx -> bucket -> cursor
Index ¶
- Constants
- Variables
- type Bucket
- type Cursor
- type DB
- type RWTransaction
- func (t *RWTransaction) Commit() error
- func (t *RWTransaction) CreateBucket(name string) error
- func (t *RWTransaction) CreateBucketIfNotExists(name string) error
- func (t *RWTransaction) Delete(name string, key []byte) error
- func (t *RWTransaction) DeleteBucket(name string) error
- func (t *RWTransaction) NextSequence(name string) (int, error)
- func (t *RWTransaction) Put(name string, key []byte, value []byte) error
- func (t *RWTransaction) Rollback()
- type Transaction
Constants ¶
const ( MaxKeySize = 32768 // 16bit MaxValueSize = 4294967295 // 32bit MaxBucketNameSize = 255 // 8bit )
Variables ¶
var ( // ErrInvalid is returned when a data file is not a Bolt-formatted database. ErrInvalid = errors.New("invalid database") // ErrVersionMismatch is returned when the data file was created with a // different version of Bolt. ErrVersionMismatch = errors.New("version mismatch") // ErrDatabaseNotOpen is returned when a DB instance is accessed before it // is opened or after it is closed. ErrDatabaseNotOpen = errors.New("database not open") // ErrDatabaseOpen is returned when opening a database that is // already open. ErrDatabaseOpen = errors.New("database already open") // ErrBucketNotFound is returned when trying to access a bucket that has // not been created yet. ErrBucketNotFound = errors.New("bucket not found") // ErrBucketExists is returned when creating a bucket that already exists. ErrBucketExists = errors.New("bucket already exists") // ErrBucketNameRequired is returned when creating a bucket with a blank name. ErrBucketNameRequired = errors.New("bucket name required") // ErrBucketNameTooLarge is returned when creating a bucket with a name // that is longer than MaxBucketNameSize. ErrBucketNameTooLarge = errors.New("bucket name too large") // ErrKeyRequired is returned when inserting a zero-length key. ErrKeyRequired = errors.New("key required") // ErrKeyTooLarge is returned when inserting a key that is larger than MaxKeySize. ErrKeyTooLarge = errors.New("key too large") // ErrValueTooLarge is returned when inserting a value that is larger than MaxValueSize. ErrValueTooLarge = errors.New("value too large") )
Functions ¶
This section is empty.
Types ¶
type Bucket ¶
type Bucket struct {
// contains filtered or unexported fields
}
Bucket represents a collection of key/value pairs inside the database. A bucket is simply a named collection of key/value pairs, just like Go’s map. It's also like a **table**.
All keys inside the bucket are unique. The Bucket type is not typically used directly. Instead the bucket name is typically passed into the Get(), Put(), or Delete() functions.
type Cursor ¶
type Cursor struct {
// contains filtered or unexported fields
}
Cursor: This object is simply for traversing the B+tree of on-disk pages or in-memory nodes. It can seek to a specific key, move to the first or last value, or it can move forward or backward. The cursor handles the movement up and down the B+tree transparently to the end user.
Cursor represents an iterator that can traverse over all key/value pairs in a **bucket** in sorted order. Cursors can be obtained from a Transaction and are valid as long as the Transaction is open.
func (*Cursor) First ¶
First moves the cursor to the first item in the bucket and returns its key and value. If the bucket is empty then a nil key is returned.
type DB ¶
type DB struct {
// contains filtered or unexported fields
}
DB represents a collection of buckets persisted to a file on disk. All data access is performed through transactions which can be obtained through the DB. All the functions on DB will return a ErrDatabaseNotOpen if accessed before Open() is called.
func (*DB) Close ¶
func (db *DB) Close()
Close releases all database resources. All transactions must be closed before closing the database.
func (*DB) Open ¶
Open opens a data file at the given path and initializes the database. If the file does not exist then it will be created automatically.
Open(): Initializes the reference to the database. It's responsible for creating the database if it doesn't exist, obtaining an exclusive lock on the file, reading the meta pages, & memory-mapping the file.
- read or create a (new) file - read or create meta0, meta1, freelist, (empty leaf) bucket pages - mmap - reference the above pages to the db
func (*DB) Update ¶
func (db *DB) Update(fn func(*RWTransaction) error) error
Update executes a function within the context of a RWTransaction. If no error is returned from the function then the transaction is committed. If an error is returned then the entire transaction is rolled back. Any error that is returned from the function or returned from the commit is returned from the Update() method.
type RWTransaction ¶
type RWTransaction struct {
Transaction
// contains filtered or unexported fields
}
RWTransaction represents a transaction that can read and write data. Only one read/write transaction can be active for a database at a time. RWTransaction is composed of a read-only Transaction so it can also use functions provided by Transaction.
func (*RWTransaction) Commit ¶
func (t *RWTransaction) Commit() error
Commit writes all changes to **disk** and updates the **meta page**. Returns an error if a disk write error occurs.
func (*RWTransaction) CreateBucket ¶
func (t *RWTransaction) CreateBucket(name string) error
CreateBucket creates a new bucket. Returns an error if the bucket already exists, if the bucket name is blank, or if the bucket name is too long.
func (*RWTransaction) CreateBucketIfNotExists ¶
func (t *RWTransaction) CreateBucketIfNotExists(name string) error
CreateBucketIfNotExists creates a new bucket if it doesn't already exist. Returns an error if the bucket name is blank, or if the bucket name is too long.
func (*RWTransaction) Delete ¶
func (t *RWTransaction) Delete(name string, key []byte) error
Delete removes a key from the named bucket. If the key does not exist then nothing is done and a nil error is returned. Returns an error if the bucket cannot be found.
func (*RWTransaction) DeleteBucket ¶
func (t *RWTransaction) DeleteBucket(name string) error
DeleteBucket deletes a bucket. Returns an error if the bucket cannot be found.
func (*RWTransaction) NextSequence ¶
func (t *RWTransaction) NextSequence(name string) (int, error)
NextSequence returns an autoincrementing integer for the bucket.
func (*RWTransaction) Put ¶
func (t *RWTransaction) Put(name string, key []byte, value []byte) error
Put sets the value for a key inside of the named bucket. If the key exist then its previous value will be overwritten. Returns an error if the bucket is not found, if the key is blank, if the key is too large, or if the value is too large.
func (*RWTransaction) Rollback ¶
func (t *RWTransaction) Rollback()
Rollback closes the transaction and ignores all previous updates.
type Transaction ¶
type Transaction struct {
// contains filtered or unexported fields
}
Transaction represents a read-only transaction on the database. It can be used for retrieving values for keys as well as creating cursors for iterating over the data.
IMPORTANT: You must close transactions when you are done with them. Pages can not be reclaimed by the writer until no more transactions are using them. A long running read transaction can cause the database to quickly grow.
func (*Transaction) Bucket ¶
func (t *Transaction) Bucket(name string) *Bucket
Bucket retrieves a bucket by name. Returns nil if the bucket does not exist.
func (*Transaction) Buckets ¶
func (t *Transaction) Buckets() []*Bucket
Buckets retrieves a list of all buckets.
func (*Transaction) Close ¶
func (t *Transaction) Close()
Close closes the transaction and releases any pages it is using.
Source Files