README
¶
Bolt
Overview
Bolt is a low-level key/value store written in pure Go. The goal of the project is to provide a simple, fast, and reliable database for projects that don't require a full database server such as Postgres or MySQL. It is also meant to be educational. Most of us use tools without understanding how the underlying data really works.
Since Bolt is meant to be used as such a low-level piece of functionality, simplicity is key. The API will be small and only center around getting values and setting values. That's it. If you want to see additional functionality added then we encourage you submit a Github issue and we can discuss developing it as a separate fork.
Simple is the new beautiful.
Project Status
Bolt is currently in development and is currently not functional.
API
Database
The database is the object that represents your data as a whole It is split up into buckets which is analogous to tables in a relational database.
Opening and closing a database
db := DB()
err := db.Open("/path/to/db", 0666)
...
err := db.Close()
Transactions
Versioning of data in the bucket data happens through a Transaction. These transactions can be either be read-only or read/write transactions. Transactions are what keeps Bolt consistent and allows data to be rolled back if needed.
It may seem strange that read-only access needs to be wrapped in a transaction but this is so Bolt can keep track of what version of the data is currently in use. The space used to hold data is kept until the transaction closes.
One important note is that long running transactions can cause the database to grow in size quickly. Please double check that you are appropriately closing all transactions after you're done with them.
Creating and closing a read-only transaction
t, err := db.Transaction()
t.Close()
Creating and committing a read/write transaction
t, err := db.RWTransaction()
err := t.Commit()
Creating and aborting a read/write transaction
t, err := db.RWTransaction()
err := t.Abort()
Buckets
Buckets are where your actual key/value data gets stored. You can create new buckets from the database and look them up by name.
Creating a bucket
t, err := db.RWTransaction()
err := t.CreateBucket("widgets")
Renaming a bucket
t, err := db.RWTransaction()
err := t.RenameBucket("widgets", "woojits")
Deleting a bucket
t, err := db.RWTransaction()
err := t.DeleteBucket("widgets")
Retrieve an existing bucket
t, err := db.Transaction()
b, err := t.Bucket("widgets")
Retrieve a list of all buckets
t, err := db.Transaction()
buckets, err := db.Buckets()
Key/Value Access
Retrieve a value for a specific key
t, err := db.Transaction()
value, err := t.Get("widgets", []byte("foo"))
value, err := t.GetString("widgets", "foo")
Set the value for a key
t, err := db.RWTransaction()
err := t.Put("widgets", []byte("foo"), []byte("bar"))
err := t.PutString("widgets", "foo", "bar")
Delete a given key
t, err := db.RWTransaction()
err := t.Delete("widgets", []byte("foo"))
err := t.DeleteString("widgets", "foo")
Cursors
Cursors provide fast read-only access to a specific bucket within a transaction.
Creating a read-only cursor
t, err := db.Transaction()
c, err := b.Cursor("widgets")
Iterating over a cursor
for k, v, err := c.First(); k != nil; k, v, err = c.Next() {
if err != nil {
return err
}
... DO SOMETHING ...
}
Internals
The Bolt database is meant to be a clean, readable implementation of a fast single-level key/value data store. This section gives an overview of the basic concepts and structure of the file format.
B+ Tree
Bolt uses a data structure called an append-only B+ tree to store its data. This structure allows for efficient traversal of data.
TODO: Explain better. :)
Pages
Bolt stores its data in discrete units called pages. The page size can be configured but is typically between 4KB and 32KB.
There are several different types of pages:
-
Meta pages - The first two pages in a database are meta pages. These are used to store references to root pages for system buckets as well as keep track of the last transaction identifier.
-
Branch pages - These pages store references to the location of deeper branch pages or leaf pages.
-
Leaf pages - These pages store the actual key/value data.
-
Overflow pages - These are special pages used when a key's data is too large for a leaf page and needs to spill onto additional pages.
Nodes
Within each page there are one or more elements called nodes. In branch pages, these nodes store references to other child pages in the tree. In leaf pages, these nodes store the actual key/value data.
TODO
The following is a list of items to do on the Bolt project:
- Calculate freelist on db.Open(). (Traverse branches, set bitmap index, load free pages into a list -- lazy load in the future).
- Resize map. (Make sure there are no reader txns before resizing)
- DB.Copy()
- Merge pages.
- Rebalance (after deletion).
Documentation
¶
Index ¶
- Constants
- Variables
- type Bucket
- type Cursor
- type DB
- type Error
- type RWTransaction
- func (t *RWTransaction) Commit() error
- func (t *RWTransaction) CreateBucket(name string) error
- func (t *RWTransaction) Delete(key []byte) error
- func (t *RWTransaction) DeleteBucket(b *Bucket) error
- func (t *RWTransaction) Put(name string, key []byte, value []byte) error
- func (t *RWTransaction) Rollback() error
- type Stat
- type Transaction
Constants ¶
const ( MaxKeySize = 0x8000 MaxDataSize = 0xffffffff )
const Version = 1
Variables ¶
var ( DatabaseNotOpenError = &Error{"db is not open", nil} DatabaseAlreadyOpenedError = &Error{"db already open", nil} TransactionInProgressError = &Error{"writable transaction is already in progress", nil} )
var ( InvalidError = &Error{"Invalid database", nil} VersionMismatchError = &Error{"version mismatch", nil} )
var ( InvalidTransactionError = &Error{"txn is invalid", nil} BucketAlreadyExistsError = &Error{"bucket already exists", nil} )
Functions ¶
This section is empty.
Types ¶
type Bucket ¶
type Bucket struct {
// contains filtered or unexported fields
}
type Cursor ¶
type Cursor struct {
// contains filtered or unexported fields
}
func (*Cursor) First ¶
First moves the cursor to the first item in the bucket and returns its key and data.
type DB ¶
func (*DB) Open ¶
Open opens a data file at the given path and initializes the database. If the file does not exist then it will be created automatically.
func (*DB) RWTransaction ¶
func (db *DB) RWTransaction() (*RWTransaction, error)
RWTransaction creates a read/write transaction. Only one read/write transaction is allowed at a time.
func (*DB) Transaction ¶
func (db *DB) Transaction() (*Transaction, error)
Transaction creates a read-only transaction. Multiple read-only transactions can be used concurrently.
type RWTransaction ¶
type RWTransaction struct {
Transaction
// contains filtered or unexported fields
}
RWTransaction represents a transaction that can read and write data. Only one read/write transaction can be active for a DB at a time.
func (*RWTransaction) Commit ¶
func (t *RWTransaction) Commit() error
func (*RWTransaction) CreateBucket ¶
func (t *RWTransaction) CreateBucket(name string) error
CreateBucket creates a new bucket.
func (*RWTransaction) Delete ¶
func (t *RWTransaction) Delete(key []byte) error
func (*RWTransaction) DeleteBucket ¶
func (t *RWTransaction) DeleteBucket(b *Bucket) error
DropBucket deletes a bucket.
func (*RWTransaction) Put ¶
func (t *RWTransaction) Put(name string, key []byte, value []byte) error
func (*RWTransaction) Rollback ¶
func (t *RWTransaction) Rollback() error
type Transaction ¶
type Transaction struct {
// contains filtered or unexported fields
}
func (*Transaction) Bucket ¶
func (t *Transaction) Bucket(name string) *Bucket
Bucket retrieves a bucket by name.
func (*Transaction) Close ¶
func (t *Transaction) Close() error
func (*Transaction) Cursor ¶
func (t *Transaction) Cursor(name string) *Cursor
Cursor creates a cursor associated with a given bucket.
func (*Transaction) DB ¶
func (t *Transaction) DB() *DB
func (*Transaction) Get ¶
func (t *Transaction) Get(name string, key []byte) []byte
Get retrieves the value for a key in a named bucket.
func (*Transaction) Stat ¶
func (t *Transaction) Stat(name string) *Stat
Stat returns information about a bucket's internal structure.
Source Files