scdb

Documentation

Index

• type Store
  • func New(path string, maxKeys *uint64, redundantBlocks *uint16, poolCapacity *uint64, ...) (*Store, error)
  • func (s *Store) Clear() error
  • func (s *Store) Close() error
  • func (s *Store) Compact() error
  • func (s *Store) Delete(k []byte) error
  • func (s *Store) Get(k []byte) ([]byte, error)
  • func (s *Store) Set(k []byte, v []byte, ttl *uint64) error

Examples

• New
• Store.Delete
• Store.Get
• Store.Set

Types

type Store

type Store struct {
	// contains filtered or unexported fields
}

Store is a key-value store that persists key-value pairs to disk

Store behaves like a HashMap that saves keys and value as byte arrays on disk. It allows for specifying how long each key-value pair should be kept for i.e. the time-to-live in seconds. If None is provided, they last indefinitely.

func New

func New(path string, maxKeys *uint64, redundantBlocks *uint16, poolCapacity *uint64, compactionInterval *uint32) (*Store, error)

New creates a new Store at the given path The Store has a number of configurations that are passed into this New function

• `storePath` - required: The path to a directory where scdb should store its data

• `maxKeys` - default: 1 million: The maximum number of key-value pairs to store in store

• `redundantBlocks` - default: 1: The store has an index to hold all the keys. This index is split into a fixed number of blocks basing on the virtual memory page size and the total number of keys to be held i.e. `max_keys`. Sometimes, there may be hash collision errors as the store's current stored keys approach `max_keys`. The closer it gets, the more it becomes likely see those errors. Adding redundant blocks helps mitigate this. Just be careful to not add too many (i.e. more than 2) since the higher the number of these blocks, the slower the store becomes.

• `poolCapacity` - default: 5: The number of buffers to hold in memory as cache's for the store. Each buffer has the size equal to the virtual memory's page size, usually 4096 bytes. Increasing this number will speed this store up but of course, the machine has a limited RAM. When this number increases to a value that clogs the RAM, performance suddenly degrades, and keeps getting worse from there on.

• `compactionInterval` - default 3600s (1 hour): The interval at which the store is compacted to remove dangling keys. Dangling keys result from either getting expired or being deleted. When a `delete` operation is done, the actual key-value pair is just marked as `deleted` but is not removed. Something similar happens when a key-value is updated. A new key-value pair is created and the old one is left unindexed. Compaction is important because it reclaims this space and reduces the size of the database file.

Example

var maxKeys uint64 = 1_000_000
var redundantBlocks uint16 = 1
var poolCapacity uint64 = 10
var compactionInterval uint32 = 1_800

store, err := New(
	"testdb",
	&maxKeys,
	&redundantBlocks,
	&poolCapacity,
	&compactionInterval)
if err != nil {
	log.Fatalf("error opening store: %s", err)
}
defer func() {
	_ = store.Close()
}()

func (*Store) Clear

func (s *Store) Clear() error

Clear removes all data in the store

func (*Store) Close

func (s *Store) Close() error

Close frees up any resources occupied by store. After this, the store is unusable. You have to re-instantiate it or just run into some crazy errors

func (*Store) Compact

func (s *Store) Compact() error

Compact manually removes dangling key-value pairs in the database file

Dangling keys result from either getting expired or being deleted. When a Store.Delete operation is done, the actual key-value pair is just marked as `deleted` but is not removed.

Something similar happens when a key-value is updated. A new key-value pair is created and the old one is left un-indexed. Compaction is important because it reclaims this space and reduces the size of the database file.

This is done automatically for you at the set `compactionInterval` but you may wish to do it manually for some reason.

This is a very expensive operation so use it sparingly.

func (*Store) Delete

func (s *Store) Delete(k []byte) error

Delete removes the key-value for the given key

Example

store, err := New("testdb", nil, nil, nil, nil)
if err != nil {
	log.Fatalf("error opening store: %s", err)
}
defer func() {
	_ = store.Close()
}()

err = store.Delete([]byte("foo"))
if err != nil {
	log.Fatalf("error deleting key: %s", err)
}

func (*Store) Get

func (s *Store) Get(k []byte) ([]byte, error)

Get returns the value corresponding to the given key

Example

store, err := New("testdb", nil, nil, nil, nil)
if err != nil {
	log.Fatalf("error opening store: %s", err)
}
defer func() {
	_ = store.Close()
}()

err = store.Set([]byte("foo"), []byte("bar"), nil)
if err != nil {
	log.Fatalf("error setting key value: %s", err)
}

value, err := store.Get([]byte("foo"))
if err != nil {
	log.Fatalf("error getting key: %s", err)
}

fmt.Printf("%s", value)

Output:

bar

func (*Store) Set

func (s *Store) Set(k []byte, v []byte, ttl *uint64) error

Set sets the given key value in the store This is used to insert or update any key-value pair in the store

Example

store, err := New("testdb", nil, nil, nil, nil)
if err != nil {
	log.Fatalf("error opening store: %s", err)
}
defer func() {
	_ = store.Close()
}()

err = store.Set([]byte("foo"), []byte("bar"), nil)
if err != nil {
	log.Fatalf("error setting key value without ttl: %s", err)
}

ttl := uint64(3_600)
err = store.Set([]byte("fake"), []byte("bear"), &ttl)
if err != nil {
	log.Fatalf("error setting key value with ttl: %s", err)
}