bitsstore

Documentation

Index

• Variables
• func CreateStore(stype string, boltPath string) (s bits.Store, err error)
• type BoltStore
  • func NewBoltStore(p string) (s *BoltStore, err error)
  • func (s *BoltStore) Get(k bits.K) (chunk []byte, err error)
  • func (s *BoltStore) Put(k bits.K, chunk []byte) (err error)
• type MemStore
  • func NewMemStore() *MemStore
  • func (s *MemStore) Get(k bits.K) (chunk []byte, err error)
  • func (s *MemStore) Put(k bits.K, chunk []byte) (err error)
  • func (s *MemStore) ServeHTTP(w http.ResponseWriter, r *http.Request)
• type S3Remote
  • func NewS3Remote(scheme, host, prefix, accessKey, secretKey string) *S3Remote
  • func (r *S3Remote) Get(k bits.K) (chunk []byte, err error)
  • func (r *S3Remote) Index(kw bits.KeyWriter) (err error)
  • func (r *S3Remote) Put(k bits.K, chunk []byte) error

Variables

var (
	//BoltChunkBucket is the name of the bucket that holds all chunks
	BoltChunkBucket = []byte("chunks")
)

var SupportedStores = []string{"bolt", "mem"}

SupportedStores holds identifiers for all supported stores

Functions

func CreateStore

func CreateStore(stype string, boltPath string) (s bits.Store, err error)

CreateStore a store instance for any of the supported types

Types

type BoltStore

type BoltStore struct {
	DB *bolt.DB
}

BoltStore stores chunks into a mmap file using a B+tree

func NewBoltStore

func NewBoltStore(p string) (s *BoltStore, err error)

NewBoltStore creates a new store by memory-mapping the file at path 'p', the database file is created if non exists at the destination

func (*BoltStore) Get

func (s *BoltStore) Get(k bits.K) (chunk []byte, err error)

Get an existhing with 'k' from the store, returns an os.ErrNotExist if no chunk with the given key exists in this store.

func (*BoltStore) Put

func (s *BoltStore) Put(k bits.K, chunk []byte) (err error)

Put a new chunk 'chunk' with key 'k' into the store

type MemStore

type MemStore struct {
	*sync.Mutex
	Chunks map[bits.K][]byte
}

MemStore stores Chunks in a map that only exists for the duration of the process, it conveniently also implements the http.Handler interface to allow it to be used as a testing remote for the S3 remote.

func NewMemStore

func NewMemStore() *MemStore

NewMemStore sets up an empty memory store

func (*MemStore) Get

func (s *MemStore) Get(k bits.K) (chunk []byte, err error)

Get a chunk from the in-memory map

func (*MemStore) Put

func (s *MemStore) Put(k bits.K, chunk []byte) (err error)

Put a chunk into the Chunks map under the given 'k'

func (*MemStore) ServeHTTP

func (s *MemStore) ServeHTTP(w http.ResponseWriter, r *http.Request)

type S3Remote

type S3Remote struct {
	// contains filtered or unexported fields
}

S3Remote will put and get chunks from an AWS S3 (compatible) interface

func NewS3Remote

func NewS3Remote(scheme, host, prefix, accessKey, secretKey string) *S3Remote

NewS3Remote sets up a HTTP client that allows chunks to be pushed to an S3 compatible object store. Its request will take on the following template <scheme>://<host>/<prefix>/<key>. When the access_key_id is set in the credentials, request will we signed prior to sending

func (*S3Remote) Get

func (r *S3Remote) Get(k bits.K) (chunk []byte, err error)

Get attempts to download chunk 'k' from an S3 object store

func (*S3Remote) Index

func (r *S3Remote) Index(kw bits.KeyWriter) (err error)

Index will use the remoet list interface to fetch all keys in the bucket

func (*S3Remote) Put

func (r *S3Remote) Put(k bits.K, chunk []byte) error

Put uploads a chunk to an S3 object store under the provided key 'k'