zdb

README

This is a goinstall-able mirror of modified code already published at:
http://git.nic.cz/redmine/projects/godns/repository/show/zdb

Online godoc documentation for this package (should be) available at:
http://gopkgdoc.appspot.com/pkg/github.com/cznic/dns/zdb

Documentation

Overview

WIP: Package zdb provides a specialized DB engine for DNS zone(s)/a DNS server, i.e. some design parameters/decisions were made with the expected usage in mind wrt the forementioned usage patterns/clients. E.g. a single map key hash could be for example only 27 bits (=> max 128M buckets), as the expected maximum number of distinct domain names in a single zone is estimated to be bellow 100M domains (and in an order of magnitudes less on average).

This combined with a theoretically sound hash algorithm (FNV-1a) leads to an estimatedly acceptable, low enough collision rate, even with such short hash keys.

In the noncolliding case, fetching an item's DB "pointer" requires reading just a few (typically 4 or 5) bytes from the DB store from an offset computed directly from the key hash.

This is guessed to be fast enough to make the DNS server still perform within tolerable latency times on nowadays server grade HW (and it will get only better with the raise of SSD and simillar/better future "HD like" technologies).

Note that the mentioned latency has (mostly) loose connection with the total server's QPS throughput, it makes slower the access per one CPU core only. And the number of cores in modern CPUs are currently surging, one can expect a server grade machine to have multicore processors.

Index

• type Store
  • func New(fn string, hashWidth, ptrBytes int) (s *Store, err error)
  • func Open(fn string) (s *Store, err error)
  • func (s *Store) Close() error
  • func (s *Store) Get(partition uint32, key []byte) (value []byte, ok bool, err error)
  • func (s *Store) Set(partition uint32, key, value []byte) (err error)

Types

type Store

type Store struct {
	*hdb.Store
	HashWidth int // R/O: FVN1a 32 bit hashes are tightened to HashWidth bits (8...30)
	PtrBytes  int // R/O: hdb "pointers" size, 4...7
	// contains filtered or unexported fields
}

Store is a DNS zone(s)/DNS server underlying DB engine.

Any object ([]byte) can be used as the (hdb) DB root.

The zdb model extends the hdb model with a hashmap of []byte keys to []byte values, (sub)partitioned with a table/zone/partition numeric IDs (uint32). The hashmap has a huge (hundreds of MB) fixed index prepended to the "real" hdb data. This overhead is a tradeoff between disk space and hashmap access speed, cf. the specialization of this package as described in the package comment.

Layout:

+--------+
| Header | Fixed size (16 bytes)
+--------+
| Index  | Variable size: 2^HashWidth * PtrBytes
+--------+
| hdb    | See the cznic/fileutil/hdb package
+--------+

Header (@ 0x0000, 16):

+0x00, 6: Magic "ZONEDB"
+0x06, 1: HashWidth
+0x07, 1: PtrBytes
+0x08, 8: int64(0) // reserved

Index (@ 0x0010, 2^HashWidth * PtrBytes):

+0*PtrBytes, PtrBytes: hdb handle for hash == 0
+1*PtrBytes, PtrBytes: hdb handle for hash == 1
+2*PtrBytes, PtrBytes: hdb handle for hash == 2
...

HDB (@ 0x0010 + 2^HashWidth * PtrBytes, variable size):

See the cznic/fileutil/falloc and cznic/fileutil/hdb packages documentations.

The underlying hdb.Store is accessible as the Store field of Store. Additionaly the hdb.Store methods are emebeded into the Store method set.

Note: The minimum allowed HashWidth == 8 bits is for testing only. The recomended value is N bits, such that 2^N > expected maximum number of keys in the DB.

func New

func New(fn string, hashWidth, ptrBytes int) (s *Store, err error)

New creates the Store in named file fn, truncating it if it already exists. The hashWidth and ptrBytes arguments: see the Store type docs. If successful, methods on the returned Store can be used for I/O. It returns the Store and an error, if any.

func Open

func Open(fn string) (s *Store, err error)

Open opens a Store in/from named file fn. If successful, methods on the returned Store can be used for data exchange. The HashWidth and PtrBytes are read from the DB. Open returns the Store and an error, if any.

func (*Store) Close

func (s *Store) Close() error

Close closes the store. Further access to the store has undefined behavior and may panic. It returns an error, if any.

func (*Store) Get

func (s *Store) Get(partition uint32, key []byte) (value []byte, ok bool, err error)

Get reads the value associated with partition, key in Store. It returns the data, key exists in ok and an error, if any. Get may return a non nil err iff !ok.

func (*Store) Set

func (s *Store) Set(partition uint32, key, value []byte) (err error)

Set stores value under partition, key in Store and returns an error if any.