bitempura

README

bitempura ⌛... ⏳!

^*non-test

Bitempura.DB is a simple, in-memory, bitemporal key-value database.

Bitemporality

Temporal databases model time as a core aspect of storing and querying data. A bitemporal database is one that supports these orthogonal axes.

• Valid time: When the fact was true in the real world. This is the application domain's notion of time.
• Transaction time: When the fact was recorded in the database. This is the system's notion of time.

Because every fact in a bitemporal database has these two dimensions, it enables use cases like this:

// We initialize a DB and start using it like an ordinary key-value store.
db, err := memory.NewDB()
err := db.Set("Bob/balance", 100)
val, err := db.Get("Bob/balance")
err := db.Delete("Alice/balance")
// and so on...

// We later learn that Bob had a temporary pending charge we missed from Dec 30 to Jan 3. (VT start = Dec 30, VT end = Jan 3)
// Retroactively record it! This does not change his balance today nor does it destroy any history we had about that period.
err := db.Set("Bob/balance", 90, WithValidTime(dec30), WithEndValidTime(jan3))

// We can at any point seamlessly ask questions about the real world past AND database record past!
// "What was Bob's balance on Jan 1 as best we knew on Jan 8?" (VT = Jan 1, TT = Jan 8)
val, err := db.Get("Bob/balance", AsOfValidTime(jan1), AsOfTransactionTime(jan8))

// More time passes and more corrections are made... When trying to make sense of what happened last month, we can ask again:
// "But what was it on Jan 1 as best we now know?" (VT = Jan 1, TT = now)
val, err := db.Get("Bob/balance", AsOfValidTime(jan1))

// And while we are at it, let's double check all of our transactions and known states for Bob's balance.
versions, err := db.History("Bob/balance")

*See full examples

Using a bitemporal database allows you to offload management of temporal application data (valid time) and data versions (transaction time) from your code and onto infrastructure. This provides a universal "time travel" capability across models in the database. Adopting these capabilities proactively is valuable because by the time you realize you need to update (or have already updated) data, it may be too late. Context may already be lost or painful to reconstruct manually.

Design

// DB for bitemporal data.
//
// Temporal control options.
// ReadOpt's: AsOfValidTime, AsOfTransactionTime.
// WriteOpt's: WithValidTime, WithEndValidTime.
type DB interface {
	// Get data by key (as of optional valid and transaction times).
	Get(key string, opts ...ReadOpt) (*VersionedKV, error)
	// List all data (as of optional valid and transaction times).
	List(opts ...ReadOpt) ([]*VersionedKV, error)
	// Set stores value (with optional start and end valid time).
	Set(key string, value Value, opts ...WriteOpt) error
	// Delete removes value (with optional start and end valid time).
	Delete(key string, opts ...WriteOpt) error

	// History returns all versioned key-values for key by descending end transaction time, descending end valid time.
	History(key string) ([]*VersionedKV, error)
}

// VersionedKV is a transaction time and valid time versioned key-value. Transaction and valid time starts are inclusive
// and ends are exclusive. No two VersionedKVs for the same key can overlap both transaction time and valid time.
type VersionedKV struct {
	Key   string
	Value Value

	TxTimeStart    time.Time  // inclusive
	TxTimeEnd      *time.Time // exclusive
	ValidTimeStart time.Time  // inclusive
	ValidTimeEnd   *time.Time // exclusive
}

// Value is the user-controlled data associated with a key (and valid and transaction time information) in the database.
type Value interface{}

• DB interface is inspired by XTDB (and Datomic).
• Storage model is inspired by Snodgrass' SQL implementations.

Author

I was learning about bitemporal databases and thought the best way to build intuition about their internal design was by building a simple one for myself. My goals are:

• Sharing a viable, standalone key-value store lib
• Creating artifacts for explaining bitemporality
• Expanding scope in new tools for gracefully extending existing SQL databases with bitemporality

Bitempura was the name of my time traveling shrimp. RIP 2049-2022. 🦐

See TODO for more.

Documentation

Overview

Package bitempura defines a bitemporal key-value database. See bitempura/memory for implementation.

Index

• Variables
• type Clock
• type DB
• type DefaultClock
  • func (c *DefaultClock) Now() time.Time
• type ReadOpt
  • func AsOfTransactionTime(t time.Time) ReadOpt
  • func AsOfValidTime(t time.Time) ReadOpt
• type ReadOptions
  • func ApplyReadOpts(opts []ReadOpt) *ReadOptions
• type Value
• type VersionedKV
  • func (d *VersionedKV) Validate() error
• type WriteOpt
  • func WithEndValidTime(t time.Time) WriteOpt
  • func WithValidTime(t time.Time) WriteOpt
• type WriteOptions
  • func ApplyWriteOpts(opts []WriteOpt) *WriteOptions

Variables

var ErrNotFound = errors.New("not found")

ErrNotFound error is returned when key not found in DB (as of relevant valid and transaction times).

Types

type Clock

type Clock interface {
	Now() time.Time
}

Clock is an interface for providing the current time for database to use as transaction times.

type DB

type DB interface {
	// Get data by key (as of optional valid and transaction times).
	Get(key string, opts ...ReadOpt) (*VersionedKV, error)
	// List all data (as of optional valid and transaction times).
	List(opts ...ReadOpt) ([]*VersionedKV, error)
	// Set stores value (with optional start and end valid time).
	Set(key string, value Value, opts ...WriteOpt) error
	// Delete removes value (with optional start and end valid time).
	Delete(key string, opts ...WriteOpt) error

	// History returns all versioned key-values for key by descending end transaction time, descending end valid time.
	History(key string) ([]*VersionedKV, error)
}

DB is a key-value database for bitemporal data.

Temporal control options. ReadOpt's: AsOfValidTime, AsOfTransactionTime. WriteOpt's: WithValidTime, WithEndValidTime.

type DefaultClock

type DefaultClock struct{}

DefaultClock is a default clock that implements Now() with time.Now()

func (*DefaultClock) Now

func (c *DefaultClock) Now() time.Time

Now returns time.Now()

type ReadOpt

type ReadOpt func(*ReadOptions)

ReadOpt is an option for database reads

func AsOfTransactionTime

func AsOfTransactionTime(t time.Time) ReadOpt

AsOfTransactionTime allows reader to read as of a specified transaction time

func AsOfValidTime

func AsOfValidTime(t time.Time) ReadOpt

AsOfValidTime allows reader to read as of a specified valid time

type ReadOptions

type ReadOptions struct {
	ValidTime *time.Time
	TxTime    *time.Time
}

ReadOptions is a struct for processing ReadOpt's specified on reads.

func ApplyReadOpts

func ApplyReadOpts(opts []ReadOpt) *ReadOptions

ApplyReadOpts applies ReadOpt's to a ReadOptions struct for usage by the DB.

type Value

type Value interface{}

Value is the user-controlled data associated with a key (and valid and transaction time information) in the database.

type VersionedKV

type VersionedKV struct {
	Key   string
	Value Value

	TxTimeStart    time.Time  // inclusive
	TxTimeEnd      *time.Time // exclusive
	ValidTimeStart time.Time  // inclusive
	ValidTimeEnd   *time.Time // exclusive
}

VersionedKV is a transaction time and valid time versioned key-value. Transaction and valid time starts are inclusive and ends are exclusive. No two VersionedKVs for the same key can overlap both transaction time and valid time.

func (*VersionedKV) Validate

func (d *VersionedKV) Validate() error

Validate a versioned key-value

type WriteOpt

type WriteOpt func(*WriteOptions)

WriteOpt is an option for database writes

func WithEndValidTime

func WithEndValidTime(t time.Time) WriteOpt

WithEndValidTime allows writer to configure explicit end valid time. Valid times cannot be set in the future.

func WithValidTime

func WithValidTime(t time.Time) WriteOpt

WithValidTime allows writer to configure explicit valid time. Valid times cannot be set in the future.

type WriteOptions

type WriteOptions struct {
	ValidTime    *time.Time
	EndValidTime *time.Time
}

WriteOptions is a struct for processing WriteOpt's specified on writes.

func ApplyWriteOpts

func ApplyWriteOpts(opts []WriteOpt) *WriteOptions

ApplyWriteOpts applies WriteOpt's to a WriteOptions struct for usage by the DB.