pmap

Documentation

Overview

Package pmap provides persistent dictionaries with high-performance access and specific timestamp semantics.

Index

• Constants
• type PMap
  • func New(path string, size uint64) *PMap
  • func Open(path string) *PMap
  • func (c *PMap) BackwardsIterate(foreach func(key, value []byte) (Continue bool)) error
  • func (c *PMap) CAS(h64 uint64, key, value []byte) error
  • func (c *PMap) Checksum() uint64
  • func (c *PMap) Close()
  • func (c *PMap) CloseAndDelete()
  • func (c *PMap) Del(h64 uint64, key, value []byte) error
  • func (c *PMap) Deleted() int
  • func (c *PMap) Get(h32 uint32, key []byte) ([]byte, error)
  • func (c *PMap) Iterate(foreach func(key, value []byte) (Continue bool)) error
  • func (c *PMap) Set(h64 uint64, key, value []byte) error
  • func (c *PMap) Size() int
  • func (c *PMap) Used() int

Constants

const FilePerms = 0700

FilePerms i

Types

type PMap

type PMap struct {
	// contains filtered or unexported fields
}

A PMap is a persistent dictionary with high-performance access and specific timestamp semantics.

A PMap has some utility functions and 4 primitives: Get, Set, Del and CAS.

Set, del and cas value parameters should contain an 8 byte long header with a timestamp. Get returns a similar value (header + real value). These timestamps have specific semantics described in each operation.

They are composed by a hashmap and a list: -The hashmap is stored in memory (RAM-only). It is used to index key-value pairs. It uses 8 bytes per bucket and it is expanded at twice its size each time a load factor is reached. -The list is stored in a memory-mapped file, RAM vs disk usage is controlled by kernel. It uses an 8 byte long header.

Note: this module is *not* thread-safe.

func New

func New(path string, size uint64) *PMap

New returns an initialized PMap stored in path with a maximum store size. Set path to "" to make the PMap anonymous, it will use RAM for everything and it won't use the file system.

func Open

func Open(path string) *PMap

Open opens a previous closed pmap returning a new pmap

func (*PMap) BackwardsIterate

func (c *PMap) BackwardsIterate(foreach func(key, value []byte) (Continue bool)) error

BackwardsIterate calls foreach for each stored pair in backwards direction, it will stop iterating if the call returns false It stops early if foreach returns false

func (*PMap) CAS

func (c *PMap) CAS(h64 uint64, key, value []byte) error

CAS (compare and swap) sets a pair value if 2 tests are passed. The value should be in this format: [0:8] => CAS timestamp [8:16] => old value FNV1a64 hash [16:24] => new timestamp [24:] => new value Tests: 1. Stored value timestamp match the CAS timestamp, if the pair doesn't exists the CAS timestamp should be 0 2. Stored value hash matches the provided hash It returns nil if the new value was written

func (*PMap) Checksum

func (c *PMap) Checksum() uint64

Checksum returns a time-stable checksum

func (*PMap) Close

func (c *PMap) Close()

Close closes a PMap. The hashmap is destroyed and the store is disk synced. Close will panic if it is called more than one time.

func (*PMap) CloseAndDelete

func (c *PMap) CloseAndDelete()

CloseAndDelete closes the PMap and removes the associated file freeing disk space.

func (*PMap) Del

func (c *PMap) Del(h64 uint64, key, value []byte) error

Del marks as deleted a pair, future read instructions won't see the old value. However, it never frees the memory-mapped region associated with the deleted pair. It "leaks". The only way to free those regions is to delete the entire PMap.

func (*PMap) Deleted

func (c *PMap) Deleted() int

Deleted returns the number of bytes deleted

func (*PMap) Get

func (c *PMap) Get(h32 uint32, key []byte) ([]byte, error)

Get returns the key's associated value or nil if it doesn't exists (or was deleted) If the pair doesn't exist it will return (nil, nil), non-existance is not considered an error The first 8 bytes contain the timestamp of the pair (nanoseconds elapsed since Unix time). Returned value is a copy of the stored one

func (*PMap) Iterate

func (c *PMap) Iterate(foreach func(key, value []byte) (Continue bool)) error

BackwardsIterate calls foreach for each stored pair, it will stop iterating if the call returns false It stops early if foreach returns false

func (*PMap) Set

func (c *PMap) Set(h64 uint64, key, value []byte) error

Set sets the value of a pair if the pair doesn't exists or if the already stored pair timestamp is before the provided timestamp. The first 8 bytes of value should contain the timestamp of the pair (nanoseconds elapsed since Unix time).

func (*PMap) Size

func (c *PMap) Size() int

Size returns the size of the pmap

func (*PMap) Used

func (c *PMap) Used() int

Used returns the number of bytes used