storethehash

README

go-storethehash

go port of github.com/vmx/storethehash

The above repo contains most of the relevant information on design.

I also ported techniques from github.com/iand/gonudb & github.com/CPPAlliance/nudb to support parallelism

Documentation

Index

• Constants
• type ErrIndexWrongBitSize
• type HashedBlockstore
  • func OpenHashedBlockstore(indexPath string, dataPath string, options ...Option) (*HashedBlockstore, error)
  • func (bs *HashedBlockstore) AllKeysChan(ctx context.Context) (<-chan cid.Cid, error)
  • func (bs *HashedBlockstore) Close()
  • func (bs *HashedBlockstore) DeleteBlock(_ cid.Cid) error
  • func (bs *HashedBlockstore) Get(c cid.Cid) (blocks.Block, error)
  • func (bs *HashedBlockstore) GetSize(c cid.Cid) (int, error)
  • func (bs *HashedBlockstore) Has(c cid.Cid) (bool, error)
  • func (bs *HashedBlockstore) HashOnRead(enabled bool)
  • func (bs *HashedBlockstore) Put(blk blocks.Block) error
  • func (bs *HashedBlockstore) PutMany(blks []blocks.Block) error
  • func (bs *HashedBlockstore) Start()
• type Option
  • func BurstRate(burstRate uint64) Option
  • func GCInterval(gcInterval time.Duration) Option
  • func IndexBitSize(indexBitSize uint8) Option
  • func SyncInterval(syncInterval time.Duration) Option

Constants

const ErrIndexTooLarge = types.ErrIndexTooLarge

ErrIndexTooLarge indicates the maximum supported bucket size is 32-bits

const ErrKeyExists = types.ErrKeyExists

const ErrKeyTooShort = types.ErrKeyTooShort

const ErrNotSupported = errorType("Operation not supported")

ErrNotSupported indicates and error that is not supported because this store is append only

const ErrOutOfBounds = types.ErrOutOfBounds

ErrOutOfBounds indicates the bucket index was greater than the number of bucks

Types

type ErrIndexWrongBitSize

type ErrIndexWrongBitSize = types.ErrIndexWrongBitSize

type HashedBlockstore

type HashedBlockstore struct {
	// contains filtered or unexported fields
}

HashedBlockstore is a blockstore that uses a simple hash table and two files to write

func OpenHashedBlockstore

func OpenHashedBlockstore(indexPath string, dataPath string, options ...Option) (*HashedBlockstore, error)

OpenHashedBlockstore opens a HashedBlockstore with the default index size

func (*HashedBlockstore) AllKeysChan

func (bs *HashedBlockstore) AllKeysChan(ctx context.Context) (<-chan cid.Cid, error)

AllKeysChan returns a channel from which the CIDs in the Blockstore can be read. It should respect the given context, closing the channel if it becomes Done.

func (*HashedBlockstore) Close

func (bs *HashedBlockstore) Close()

func (*HashedBlockstore) DeleteBlock

func (bs *HashedBlockstore) DeleteBlock(_ cid.Cid) error

DeleteBlock is not supported for this store

func (*HashedBlockstore) Get

func (bs *HashedBlockstore) Get(c cid.Cid) (blocks.Block, error)

Get returns a block

func (*HashedBlockstore) GetSize

func (bs *HashedBlockstore) GetSize(c cid.Cid) (int, error)

GetSize returns the CIDs mapped BlockSize

func (*HashedBlockstore) Has

func (bs *HashedBlockstore) Has(c cid.Cid) (bool, error)

Has indicates if a block is present in a block store

func (*HashedBlockstore) HashOnRead

func (bs *HashedBlockstore) HashOnRead(enabled bool)

HashOnRead specifies if every read block should be rehashed to make sure it matches its CID.

func (*HashedBlockstore) Put

func (bs *HashedBlockstore) Put(blk blocks.Block) error

Put puts a given block to the underlying datastore

func (*HashedBlockstore) PutMany

func (bs *HashedBlockstore) PutMany(blks []blocks.Block) error

PutMany puts a slice of blocks at the same time using batching capabilities of the underlying datastore whenever possible.

func (*HashedBlockstore) Start

func (bs *HashedBlockstore) Start()

type Option

type Option func(*configOptions)

func BurstRate

func BurstRate(burstRate uint64) Option

func GCInterval

func GCInterval(gcInterval time.Duration) Option

func IndexBitSize

func IndexBitSize(indexBitSize uint8) Option

func SyncInterval

func SyncInterval(syncInterval time.Duration) Option