Back to godoc.org
github.com/timshannon/bolthold

Package bolthold

v0.0.0-...-a85bcc0
Latest Go to latest

The latest major version is .

Published: Aug 12, 2019 | License: MIT | Module: github.com/timshannon/bolthold

Overview

Package bolthold is an indexing and querying layer on top of a Bolt DB. The goal is to allow easy, persistent storage and retrieval of Go types. BoltDB is an embedded key-value store, and bolthold servers a similar use case however with a higher level interface for common uses of BoltDB.

Go Types

BoltHold deals directly with Go Types. When inserting data, you pass in your structure directly. When querying data you pass in a pointer to a slice of the type you want to return. By default Gob encoding is used. You can put multiple different types into the same DB file and they (and their indexes) will be stored separately.

err := store.Insert(1234, Item{
	Name:    "Test Name",
	Created: time.Now(),
})

var result []Item

err := store.Find(&result, query)

Indexes

BoltHold will automatically create an index for any struct fields tags with "boltholdIndex"

type Item struct {
	ID       int
	Name     string
	Category string `boltholdIndex:"Category"`
	Created  time.Time
}

The first field specified in query will be used as the index (if one exists).

Queries are chained together criteria that applies to a set of fields:

bolthold.Where("Name").Eq("John Doe").And("DOB").Lt(time.Now())
Example

Code:

package main

import (
	"fmt"
	"log"
	"os"
	"time"

	"github.com/timshannon/bolthold"
	bolt "go.etcd.io/bbolt"
)

type Item struct {
	ID       int
	Category string `boltholdIndex:"Category"`
	Created  time.Time
}

func main() {
	data := []Item{
		Item{
			ID:       0,
			Category: "blue",
			Created:  time.Now().Add(-4 * time.Hour),
		},
		Item{
			ID:       1,
			Category: "red",
			Created:  time.Now().Add(-3 * time.Hour),
		},
		Item{
			ID:       2,
			Category: "blue",
			Created:  time.Now().Add(-2 * time.Hour),
		},
		Item{
			ID:       3,
			Category: "blue",
			Created:  time.Now().Add(-20 * time.Minute),
		},
	}

	filename := tempfile()
	store, err := bolthold.Open(filename, 0666, nil)
	defer store.Close()
	defer os.Remove(filename)

	if err != nil {
		// handle error
		log.Fatal(err)
	}

	// insert the data in one transaction

	err = store.Bolt().Update(func(tx *bolt.Tx) error {
		for i := range data {
			err := store.TxInsert(tx, data[i].ID, data[i])
			if err != nil {
				return err
			}
		}
		return nil
	})

	if err != nil {
		// handle error
		log.Fatal(err)
	}

	// Find all items in the blue category that have been created in the past hour
	var result []Item

	err = store.Find(&result, bolthold.Where("Category").Eq("blue").And("Created").Ge(time.Now().Add(-1*time.Hour)))

	if err != nil {
		// handle error
		log.Fatal(err)
	}

	fmt.Println(result[0].ID)
}
3

Index

Examples

Constants

const BoltholdIndexTag = "boltholdIndex"

BoltholdIndexTag is the struct tag used to define an a field as indexable for a bolthold

const BoltholdKeyTag = "boltholdKey"

BoltholdKeyTag is the struct tag used to define an a field as a key for use in a Find query

const Key = ""

Key is shorthand for specifying a query to run again the Key in a bolthold, simply returns "" Where(bolthold.Key).Eq("testkey")

Variables

var ErrKeyExists = errors.New("This Key already exists in this bolthold for this type")

ErrKeyExists is the error returned when data is being Inserted for a Key that already exists

var ErrNotFound = errors.New("No data found for this key")

ErrNotFound is returned when no data is found for the given key

func DefaultDecode

func DefaultDecode(data []byte, value interface{}) error

DefaultDecode is the default decoding func for bolthold (Gob)

func DefaultEncode

func DefaultEncode(value interface{}) ([]byte, error)

DefaultEncode is the default encoding func for bolthold (Gob)

func NextSequence

func NextSequence() interface{}

NextSequence is used to create a sequential key for inserts Inserts a uint64 as the key store.Insert(bolthold.NextSequence(), data)

func Slice

func Slice(value interface{}) []interface{}

Slice turns a slice of any time into []interface{} by copying the slice values so it can be easily passed into queries that accept variadic parameters. Will panic if value is not a slice

type AggregateResult

type AggregateResult struct {
	// contains filtered or unexported fields
}

AggregateResult allows you to access the results of an aggregate query

func (*AggregateResult) Avg

func (a *AggregateResult) Avg(field string) float64

Avg returns the average float value of the aggregate grouping panics if the field cannot be converted to an float64

func (*AggregateResult) Count

func (a *AggregateResult) Count() int

Count returns the number of records in the aggregate grouping

func (*AggregateResult) Group

func (a *AggregateResult) Group(result ...interface{})

Group returns the field grouped by in the query

func (*AggregateResult) Max

func (a *AggregateResult) Max(field string, result interface{})

Max Returns the maxiumum value of the Aggregate Grouping, uses the Comparer interface

func (*AggregateResult) Min

func (a *AggregateResult) Min(field string, result interface{})

Min returns the minimum value of the Aggregate Grouping, uses the Comparer interface

func (*AggregateResult) Reduction

func (a *AggregateResult) Reduction(result interface{})

Reduction is the collection of records that are part of the AggregateResult Group

func (*AggregateResult) Sort

func (a *AggregateResult) Sort(field string)

Sort sorts the aggregate reduction by the passed in field in ascending order Sort is called automatically by calls to Min / Max to get the min and max values

func (*AggregateResult) Sum

func (a *AggregateResult) Sum(field string) float64

Sum returns the sum value of the aggregate grouping panics if the field cannot be converted to an float64

type Comparer

type Comparer interface {
	Compare(other interface{}) (int, error)
}

Comparer compares a type against the encoded value in the store. The result should be 0 if current==other, -1 if current < other, and +1 if current > other. If a field in a struct doesn't specify a comparer, then the default comparison is used (convert to string and compare) this interface is already handled for standard Go Types as well as more complex ones such as those in time and big an error is returned if the type cannot be compared The concrete type will always be passedin, not a pointer

type Criterion

type Criterion struct {
	// contains filtered or unexported fields
}

Criterion is an operator and a value that a given field needs to match on

func Where

func Where(field string) *Criterion

Where starts a query for specifying the criteria that an object in the bolthold needs to match to be returned in a Find result

Query API Example

s.Find(bolthold.Where("FieldName").Eq(value).And("AnotherField").Lt(AnotherValue).Or(bolthold.Where("FieldName").Eq(anotherValue)

Since Gobs only encode exported fields, this will panic if you pass in a field with a lower case first letter

func (*Criterion) Contains

func (c *Criterion) Contains(value interface{}) *Query

Contains tests if the current field is a slice that contains the passed in value

func (*Criterion) ContainsAll

func (c *Criterion) ContainsAll(values ...interface{}) *Query

ContainsAll tests if the current field is a slice that contains all of the passed in values. If any of the values are NOT contained in the slice, then no match is made

func (*Criterion) ContainsAny

func (c *Criterion) ContainsAny(values ...interface{}) *Query

ContainsAny tests if the current field is a slice that contains any of the passed in values. If any of the values are contained in the slice, then a match is made

func (*Criterion) Eq

func (c *Criterion) Eq(value interface{}) *Query

Eq tests if the current field is Equal to the passed in value

func (*Criterion) Ge

func (c *Criterion) Ge(value interface{}) *Query

Ge test if the current field is Greater Than or Equal To the passed in value

func (*Criterion) Gt

func (c *Criterion) Gt(value interface{}) *Query

Gt test if the current field is Greater Than the passed in value

func (*Criterion) In

func (c *Criterion) In(values ...interface{}) *Query

In test if the current field is a member of the slice of values passed in

func (*Criterion) IsNil

func (c *Criterion) IsNil() *Query

IsNil will test if a field is equal to nil

func (*Criterion) Le

func (c *Criterion) Le(value interface{}) *Query

Le test if the current field is Less Than or Equal To the passed in value

func (*Criterion) Lt

func (c *Criterion) Lt(value interface{}) *Query

Lt test if the current field is Less Than the passed in value

func (*Criterion) MatchFunc

func (c *Criterion) MatchFunc(match interface{}) *Query

MatchFunc will test if a field matches the passed in function

func (*Criterion) Ne

func (c *Criterion) Ne(value interface{}) *Query

Ne test if the current field is Not Equal to the passed in value

func (*Criterion) Not

func (c *Criterion) Not() *Criterion

Not will negate the following critierion

func (*Criterion) RegExp

func (c *Criterion) RegExp(expression *regexp.Regexp) *Query

RegExp will test if a field matches against the regular expression The Field Value will be converted to string (%s) before testing

func (*Criterion) String

func (c *Criterion) String() string

type DecodeFunc

type DecodeFunc func(data []byte, value interface{}) error

DecodeFunc is a function for decoding a value from bytes

type EncodeFunc

type EncodeFunc func(value interface{}) ([]byte, error)

EncodeFunc is a function for encoding a value into bytes

type ErrTypeMismatch

type ErrTypeMismatch struct {
	Value interface{}
	Other interface{}
}

ErrTypeMismatch is the error thrown when two types cannot be compared

func (*ErrTypeMismatch) Error

func (e *ErrTypeMismatch) Error() string

type Field

type Field string

Field allows for referencing a field in structure being compared

type Index

type Index func(name string, value interface{}) ([]byte, error)

Index is a function that returns the indexable, encoded bytes of the passed in value

type MatchFunc

type MatchFunc func(ra interface{}) (bool, error)

MatchFunc is a function used to test an arbitrary matching value in a query

type Options

type Options struct {
	Encoder EncodeFunc
	Decoder DecodeFunc
	*bolt.Options
}

Options allows you set different options from the defaults For example the encoding and decoding funcs which default to Gob

type Query

type Query struct {
	// contains filtered or unexported fields
}

Query is a chained collection of criteria of which an object in the bolthold needs to match to be returned an empty query matches against all records

func (*Query) And

func (q *Query) And(field string) *Criterion

And creates a nother set of criterion the needs to apply to a query

func (*Query) Index

func (q *Query) Index(indexName string) *Query

Index specifies the index to use when running this query

func (*Query) IsEmpty

func (q *Query) IsEmpty() bool

IsEmpty returns true if the query is an empty query an empty query matches against everything

func (*Query) Limit

func (q *Query) Limit(amount int) *Query

Limit sets the maximum number of records that can be returned by a query Setting Limit multiple times, or to a negative value will panic

func (*Query) Or

func (q *Query) Or(query *Query) *Query

Or creates another separate query that gets unioned with any other results in the query Or will panic if the query passed in contains a limit or skip value, as they are only allowed on top level queries

func (*Query) Reverse

func (q *Query) Reverse() *Query

Reverse will reverse the current result set useful with SortBy

func (*Query) Skip

func (q *Query) Skip(amount int) *Query

Skip skips the number of records that match all the rest of the query criteria, and does not return them in the result set. Setting skip multiple times, or to a negative value will panic

func (*Query) SortBy

func (q *Query) SortBy(fields ...string) *Query

SortBy sorts the results by the given fields name Multiple fields can be used

func (*Query) String

func (q *Query) String() string

type RecordAccess

type RecordAccess struct {
	// contains filtered or unexported fields
}

RecordAccess allows access to the current record, field or allows running a subquery within a MatchFunc

func (*RecordAccess) Field

func (r *RecordAccess) Field() interface{}

Field is the current field being queried

func (*RecordAccess) Record

func (r *RecordAccess) Record() interface{}

Record is the complete record for a given row in bolthold

func (*RecordAccess) SubAggregateQuery

func (r *RecordAccess) SubAggregateQuery(query *Query, groupBy ...string) ([]*AggregateResult, error)

SubAggregateQuery allows you to run another aggregate query in the same transaction for each record in a parent query

func (*RecordAccess) SubQuery

func (r *RecordAccess) SubQuery(result interface{}, query *Query) error

SubQuery allows you to run another query in the same transaction for each record in a parent query

type Store

type Store struct {
	// contains filtered or unexported fields
}

Store is a bolthold wrapper around a bolt DB

func Open

func Open(filename string, mode os.FileMode, options *Options) (*Store, error)

Open opens or creates a bolthold file.

func (*Store) Bolt

func (s *Store) Bolt() *bolt.DB

Bolt returns the underlying Bolt DB the bolthold is based on

func (*Store) Close

func (s *Store) Close() error

Close closes the bolt db

func (*Store) Count

func (s *Store) Count(dataType interface{}, query *Query) (int, error)

Count returns the current record count for the passed in datatype

func (*Store) Delete

func (s *Store) Delete(key, dataType interface{}) error

Delete deletes a record from the bolthold, datatype just needs to be an example of the type stored so that the proper bucket and indexes are updated

func (*Store) DeleteMatching

func (s *Store) DeleteMatching(dataType interface{}, query *Query) error

DeleteMatching deletes all of the records that match the passed in query

func (*Store) Find

func (s *Store) Find(result interface{}, query *Query) error

Find retrieves a set of values from the bolthold that matches the passed in query result must be a pointer to a slice. The result of the query will be appended to the passed in result slice, rather than the passed in slice being emptied.

func (*Store) FindAggregate

func (s *Store) FindAggregate(dataType interface{}, query *Query, groupBy ...string) ([]*AggregateResult, error)

FindAggregate returns an aggregate grouping for the passed in query groupBy is optional

func (*Store) FindOne

func (s *Store) FindOne(result interface{}, query *Query) error

FindOne returns a single record, and so result is NOT a slice, but an pointer to a struct, if no record is found that matches the query, then it returns ErrNotFound

func (*Store) Get

func (s *Store) Get(key, result interface{}) error

Get retrieves a value from bolthold and puts it into result. Result must be a pointer

func (*Store) IndexExists

func (s *Store) IndexExists(tx *bolt.Tx, typeName, indexName string) bool

IndexExists tests if an index exists for the passed in field name

func (*Store) Insert

func (s *Store) Insert(key, data interface{}) error

Insert inserts the passed in data into the the bolthold

If the the key already exists in the bolthold, then an ErrKeyExists is returned If the data struct has a field tagged as `boltholdKey` and it is the same type as the Insert key, AND the data struct is passed by reference, AND the key field is currently set to the zero-value for that type, then that field will be set to the value of the insert key.

To use this with bolthold.NextSequence() use a type of `uint64` for the key field.

func (*Store) ReIndex

func (s *Store) ReIndex(exampleType interface{}, bucketName []byte) error

ReIndex removes any existing indexes and adds all the indexes defined by the passed in datatype example This function allows you to index an already existing boltDB file, or refresh any missing indexes if bucketName is nil, then we'll assume a bucketName of storer.Type() if a bucketname is specified, then the data will be copied to the bolthold standard bucket of storer.Type()

func (*Store) RemoveIndex

func (s *Store) RemoveIndex(dataType interface{}, indexName string) error

RemoveIndex removes an index from the store.

func (*Store) TxCount

func (s *Store) TxCount(tx *bolt.Tx, dataType interface{}, query *Query) (int, error)

TxCount returns the current record count from within the given transaction for the passed in datatype

func (*Store) TxDelete

func (s *Store) TxDelete(tx *bolt.Tx, key, dataType interface{}) error

TxDelete is the same as Delete except it allows you specify your own transaction

func (*Store) TxDeleteMatching

func (s *Store) TxDeleteMatching(tx *bolt.Tx, dataType interface{}, query *Query) error

TxDeleteMatching does the same as DeleteMatching, but allows you to specify your own transaction

func (*Store) TxFind

func (s *Store) TxFind(tx *bolt.Tx, result interface{}, query *Query) error

TxFind allows you to pass in your own bolt transaction to retrieve a set of values from the bolthold

func (*Store) TxFindAggregate

func (s *Store) TxFindAggregate(tx *bolt.Tx, dataType interface{}, query *Query,
	groupBy ...string) ([]*AggregateResult, error)

TxFindAggregate is the same as FindAggregate, but you specify your own transaction groupBy is optional

func (*Store) TxFindOne

func (s *Store) TxFindOne(tx *bolt.Tx, result interface{}, query *Query) error

TxFindOne allows you to pass in your own bolt transaction to retrieve a single record from the bolthold

func (*Store) TxGet

func (s *Store) TxGet(tx *bolt.Tx, key, result interface{}) error

TxGet allows you to pass in your own bolt transaction to retrieve a value from the bolthold and puts it into result

func (*Store) TxInsert

func (s *Store) TxInsert(tx *bolt.Tx, key, data interface{}) error

TxInsert is the same as Insert except it allows you specify your own transaction

func (*Store) TxUpdate

func (s *Store) TxUpdate(tx *bolt.Tx, key interface{}, data interface{}) error

TxUpdate is the same as Update except it allows you to specify your own transaction

func (*Store) TxUpdateMatching

func (s *Store) TxUpdateMatching(tx *bolt.Tx, dataType interface{}, query *Query, update func(record interface{}) error) error

TxUpdateMatching does the same as UpdateMatching, but allows you to specify your own transaction

func (*Store) TxUpsert

func (s *Store) TxUpsert(tx *bolt.Tx, key interface{}, data interface{}) error

TxUpsert is the same as Upsert except it allows you to specify your own transaction

func (*Store) Update

func (s *Store) Update(key interface{}, data interface{}) error

Update updates an existing record in the bolthold if the Key doesn't already exist in the store, then it fails with ErrNotFound

func (*Store) UpdateMatching

func (s *Store) UpdateMatching(dataType interface{}, query *Query, update func(record interface{}) error) error

UpdateMatching runs the update function for every record that match the passed in query Note that the type of record in the update func always has to be a pointer

func (*Store) Upsert

func (s *Store) Upsert(key interface{}, data interface{}) error

Upsert inserts the record into the bolthold if it doesn't exist. If it does already exist, then it updates the existing record

type Storer

type Storer interface {
	Type() string              // used as the boltdb bucket name
	Indexes() map[string]Index //[indexname]indexFunc
}

Storer is the Interface to implement to skip reflect calls on all data passed into the bolthold

Package Files

Documentation was rendered with GOOS=linux and GOARCH=amd64.

Jump to identifier

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to identifier