bow

package module
Version: v0.0.0-...-bf1012a Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Feb 29, 2020 License: MIT Imports: 15 Imported by: 4

README

Bow GoDoc

Bow is a minimal embedded database powered by Badger.

The mission of Bow is to provide a simple, fast and reliable way to persist structured data for projects that don't need an external database server such as PostgreSQL or MongoDB.

Bow is powered by BadgerDB, implementing buckets and serialization on top of it.

Table of Contents

Why Badger and not Bolt?

Badger is more actively maintained than bbolt, allows for key-only iteration and has some very interesting performance characteristics.

Getting Started

Installing
go get -u github.com/zippoxer/bow
Opening a database
// Open database under directory "test".
db, err := bow.Open("test")
if err != nil {
    log.Fatal(err)
}
defer db.Close()

With options:

db, err := bow.Open("test",
    bow.SetCodec(msgp.Codec{}),
    bow.SetBadgerOptions(badger.DefaultOptions}))
if err != nil {
    log.Fatal(err)
}
Defining a structure

Each record in the database has an explicit or implicit unique key.

If a structure doesn't define a key or has a zero-value key, Bow stores it with a randomly generated key.

type Page struct {
    Body     []byte
    Tags     []string
    Created  time.Time
}

Structures must define a key if they wish to manipulate it.

type Page struct {
    Id      string `bow:"key"`
    Body     []byte
    Tags     []string
    Created  time.Time
}

Keys must be a string, a byte slice, any built-in integer of at least 32 bits (int32, uint32 and above) or a type that implements codec.Marshaler and codec.Unmarshaler.

Randomly generated keys

Id is a convenient placeholder for Bow's randomly generated keys.

type Page struct {
    Id bow.Id // Annotating with `bow:"key"` isn't necessary.
    // ...
}

Id.String() returns a user-friendly representation of Id.

ParseId(string) parses the user-friendly representation into an Id.

NewId() generates a random Id. Only necessary when you need to know the inserted Id.

Persisting a structure

Put persists a structure into the bucket. If a record with the same key already exists, then it will be updated.

page1 := Page{
    Id:      bow.NewId(),
    Body:    []byte("<h1>Example Domain</h1>"),
    Tags:    []string{"example", "h1"},
    Created: time.Now(),
}
err := db.Bucket("pages").Put(page1)
if err != nil {
    log.Fatal(err)
}
Retrieving a structure

Get retrieves a structure by key from a bucket, returning ErrNotFound if it doesn't exist.

var page2 Page
err := db.Bucket("pages").Get(page1.Id, &page2)
if err != nil {
    log.Fatal(err)
}
Iterating a bucket
iter := db.Bucket("pages").Iter()
defer iter.Close()
var page Page
for iter.Next(&page) {
    log.Println(page.Id.String()) // User-friendly representation of bow.Id.
}
if iter.Err() != nil {
    log.Fatal(err)
}
Prefix iteration

Iterate over records whose key starts with a given prefix.

For example, let's define Page with URL as the key:

type Page struct {
    URL  string `bow:"key"`
    Body []byte
}

Finally, let's iterate over HTTPS pages:

iter := db.Bucket("pages").Prefix("https://")
var page Page
for iter.Next(&page) {
    log.Println(page.URL)
}
Serialization

By default, Bow serializes structures with encoding/json. You can change that behaviour by passing a type that implements codec.Codec via the bow.SetCodec option.

MessagePack with tinylib/msgp

msgp is a code generation tool and serialization library for MessagePack, and it's nearly as fast as protocol buffers and about an order of magnitude faster than encoding/json (see benchmarks). Bow provides a codec.Codec implementation for msgp under the codec/msgp package. Here's how to use it:

  • Since msgp generates code to serialize structures, you must include the following directive in your Go file:
//go:generate msgp
  • Replace any use of bow.Id in your structures with string. Since bow.Id is a string, you can convert between the two without any cost.

  • Import github.com/zippoxer/bow/codec/msgp and open a database with msgp.Codec:

bow.Open("test", bow.SetCodec(msgp.Codec{}))

Read more about msgp and it's code generation settings at https://github.com/tinylib/msgp

Upcoming

Key-only iteration

Since Badger separates keys from values, key-only iteration should be orders of magnitude faster, at least in some cases, than it's equivalent with Bolt.

Bow's key-only iterator is a work in progress.

Transactions

Cross-bucket transactions are a work in progress. See branch tx.

Querying

Bow doesn't feature a querying mechanism yet. Instead, you must iterate records to query or filter them.

For example, let's say I want to perform the equivalent of

SELECT * FROM pages WHERE url LIKE '%/home%'

in Bow, I could iterate the pages bucket and filter pages with URLs containing '/home':

var matches []Page
iter := db.Bag("pages").Iter()
defer iter.Close()
var page Page
for iter.Next(&page) {
    if strings.Contains(strings.ToLower(page.URL), "/home") {
        matches = append(matches, page)
    }
}
return matches, iter.Err()

Meanwhile, you can try Storm if you want convenient querying.

Performance

Bow is nearly as fast as Badger, and in most cases faster than Storm. See Go Database Benchmarks.

Contributing

I welcome any feedback and contribution.

My priorities right now are tests, documentation and polish.

Bow lacks decent tests and documentation for types, functions and methods. Most of Bow's code was written before I had any plan to release it, and I think it needs polish.

Documentation

Index

Constants

View Source
const MaxBuckets = math.MaxUint16 - (8 * 256)

MaxBuckets is the maximum amount of buckets that can be created in a database.

Variables

View Source
var (
	ErrNotFound = errors.New("Record doesn't exist")
	ErrReadOnly = errors.New("Put and Delete aren't allowed in read-only mode")
)

Functions

This section is empty.

Types

type Bucket

type Bucket struct {
	// contains filtered or unexported fields
}

Bucket represents a collection of records in the database.

func (*Bucket) Delete

func (b *Bucket) Delete(key interface{}) error

Delete removes a record from the bucket by key.

func (*Bucket) Get

func (b *Bucket) Get(key interface{}, v interface{}) error

Get retrieves a record from the bucket by key, returning ErrNotFound if it doesn't exist.

func (*Bucket) GetBytes

func (b *Bucket) GetBytes(key interface{}, in []byte) (out []byte, err error)

func (*Bucket) Iter

func (b *Bucket) Iter() *Iter

Iter returns an iterator for all the records in the bucket.

func (*Bucket) Prefix

func (b *Bucket) Prefix(prefix interface{}) *Iter

Prefix returns an iterator for all the records whose key has the given prefix.

func (*Bucket) Put

func (b *Bucket) Put(v interface{}) error

Put persists a record into the bucket. If a record with the same key already exists, then it will be updated.

func (*Bucket) PutBytes

func (b *Bucket) PutBytes(key interface{}, data []byte) error

type DB

type DB struct {
	// contains filtered or unexported fields
}

DB is an opened Bow database.

func Open

func Open(dir string, options ...Option) (*DB, error)

Open opens a database at the given directory. If the directory doesn't exist, then it will be created.

Configure the database by passing the result of functions like SetCodec or SetBadgerOptions.

Make sure to call Close after you're done.

func (*DB) Badger

func (db *DB) Badger() *badger.DB

Badger exposes the internal Badger database. Use it to call Backup, Load or RunValueLogGC. Do NOT perform Set operations as you may corrupt Bow.

func (*DB) Bucket

func (db *DB) Bucket(name string) *Bucket

Bucket returns the named bucket, creating it if it doesn't exist. If an error has occurred during creation, it would be returned by any operation on the returned bucket.

func (*DB) Buckets

func (db *DB) Buckets() []string

Buckets returns a list of the names of all the buckets in the DB.

func (*DB) Close

func (db *DB) Close() error

Close releases all database resources.

type Id

type Id string

Id is a convenient type for randomly generated keys.

func NewId

func NewId() Id

NewId generates an 8-byte unique Id.

func ParseId

func ParseId(s string) (Id, error)

ParseId parses the user-friendly output of String to an Id.

func (Id) Marshal

func (id Id) Marshal(in []byte) ([]byte, error)

func (Id) MarshalJSON

func (id Id) MarshalJSON() ([]byte, error)

func (Id) String

func (id Id) String() string

String returns a user-friendly format of the Id.

func (*Id) Unmarshal

func (id *Id) Unmarshal(b []byte) error

func (*Id) UnmarshalJSON

func (id *Id) UnmarshalJSON(data []byte) error

type Iter

type Iter struct {
	// contains filtered or unexported fields
}

func (*Iter) Close

func (it *Iter) Close()

Close closes the Iter. If Next is called and returns false and there are no further results, Iter is closed automatically and it will suffice to check the result of Err.

func (*Iter) Err

func (it *Iter) Err() error

Err returns the error, if any, that was encountered during iteration. Err may be called after an explicit or implicit Close.

func (*Iter) Next

func (it *Iter) Next(result interface{}) bool

type Option

type Option func(db *DB) error

Option is a function that configures a DB.

func SetBadgerOptions

func SetBadgerOptions(o badger.Options) Option

func SetCodec

func SetCodec(c codec.Codec) Option

func SetLogger

func SetLogger(logger badger.Logger) Option

func SetReadOnly

func SetReadOnly(readOnly bool) Option

Directories

Path Synopsis
key
Package key implements the standard encoding and decoding of Bow keys.
Package key implements the standard encoding and decoding of Bow keys.
msgp
Package msgp implements encoding and decoding of MessagePack, relying on serialization code generated by msgp, the code generation library for MessagePack at github.com/tinylib/msgp.
Package msgp implements encoding and decoding of MessagePack, relying on serialization code generated by msgp, the code generation library for MessagePack at github.com/tinylib/msgp.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL