ops

Documentation

Overview

Package ops implements network and storage for DOT

This builds on top of the https://godoc.org/github.com/dotchain/dot/changes package

Index

• type Cache
• type Op
• type Operation
  • func (o Operation) Basis() int
  • func (o Operation) Changes() changes.Change
  • func (o Operation) ID() interface{}
  • func (o Operation) Parent() interface{}
  • func (o Operation) Version() int
  • func (o Operation) WithChanges(c changes.Change) Op
  • func (o Operation) WithVersion(v int) Op
• type Store
  • func Polled(s Store) Store
  • func Transformed(raw Store, cache Cache) Store

Types

type Cache

type Cache interface {
	// Load returns nil if the items are not present in the cache.
	Load(version int) (transformed Op, mergeChain []Op)

	// Store saves the transformed op and its merge chain.
	Store(version int, transformed Op, mergeChain []Op)
}

Cache holds transformation state for each operation.

The transformed op is a version of the original op but one which can be applied directly on top of the previous version. The untransformed operation generally cannot be applied because it might have a Basis() or Parent() that is much older than the previous version as it may not have merged with those.

The mergeChain is the counterpart: the client which sent the original op can apply this to get to the same state as applying the transformed op on the previous version.

type Op

type Op interface {
	// ID is the unique identifier for the operation. It should be
	// of any type that can be compared for equality
	ID() interface{}

	// Version is the index of the operation in the sequential
	// store. For operations that have not yet been stored (such
	// as client operations), this must be negative. Once an
	// operation is stored, its version will never change.
	Version() int

	// WithVersion returns a new op with the specified version.
	WithVersion(int) Op

	// Parent is the ID of the local operation this particular
	// operation is based upon.  If this operaiton is based upon
	// an operation that has a version at this point (i.e. it has
	// been stored), then ParentID must be nil.
	Parent() interface{}

	// Basis is the last "stored" operation (i.e. with
	// non-negative version) that has been factored in before the
	// current operation.
	//
	// Example:
	//
	// If a client makes a sequence of operations (A, B, C), then
	// all of them will share the same Basis.  If, at this point,
	// the client receives a new operation from the server and
	// merges it in, its basis for future operations would be set
	// to that version but its parent would remain as the last
	// local operation.
	//
	// If the client receives its own operation back from the
	// server, there is no specific action the client has to take
	// but the next future operation will have the acknowledged
	// operation as its Basis and an empty Parent.
	Basis() int

	// Changes is the changes associated with this operation. It
	// can be nil.
	Changes() changes.Change

	// WithChanges creates a new operation with the same metadata
	// as the current but with a different set of changes.
	WithChanges(changes.Change) Op
}

Op represents a collection of changes, like a git-commit.

The ID of the op uniquely identifies it.

All operations are effectively ordered sequentually in a "master" branch. The index in this store is the "Version" of the operation.

Every operation has a "parent" operation that it is based on. That operation is identified by the ParentID. Each operation can be thought of as creating a branch. But a client can also "pull" in changes from the "master" branch. When this happens, further operations will effectively be based upon both the "Parent" and the merged in master version. The Basis tracks the lasts version of the master branch that was pulled in and so must always be non-negative.

Type Operation implements this interface though the rest of the package makes no assumptions about the concrete implementation of this interface.

type Operation

type Operation struct {
	OpID, ParentID interface{}
	VerID, BasisID int
	changes.Change
}

Operation holds the basic info needed for Op with string IDs

func (Operation) Basis

func (o Operation) Basis() int

Basis implements Op.Basis

func (Operation) Changes

func (o Operation) Changes() changes.Change

Changes implements Op.Changes

func (Operation) ID

func (o Operation) ID() interface{}

ID implements Op.ID

func (Operation) Parent

func (o Operation) Parent() interface{}

Parent implements Op.Parent

func (Operation) Version

func (o Operation) Version() int

Version implements Op.Version

func (Operation) WithChanges

func (o Operation) WithChanges(c changes.Change) Op

WithChanges implements Op.WithChanges

func (Operation) WithVersion

func (o Operation) WithVersion(v int) Op

WithVersion implements Op.WithVersion

type Store

type Store interface {
	// Append a sequence of operations.  If the operation IDs
	// already exist, those operations are ignored but do not
	// generate an error.
	Append(ctx context.Context, ops []Op) error

	// GetSince returns all operations with version atleast equal
	// to the specified parameter. If the number of operations is
	// larger than the limit, it is truncated.
	//
	// It is not an error if the version does not exist -- an
	// empty result is returned in that case.   If a timeout is
	// provided, it is used as a polling mechanism.
	//
	// Fewer than limit entries are returned if and only if there
	// are no further entries aavailable.
	GetSince(ctx context.Context, version, limit int) ([]Op, error)

	// Close releases all resources. Any ongoing calls should not
	// be canceled unless the caller cancels them via the context.
	Close()
}

Store is the interface to talk to a Op store.

An Op store is an append-only store which guarantees unique order of operations. It does not make any guarantees on the specific order beyond the requirement that all operations provided to a single Append call will always be together in that order and that a sequential calls to Append on a single client/goroutine will be stored in that order.

The store does make guarantee that operations will not be duplicated. If an operation is appended with an ID that already exists, it will silently be dropped.

See https://godoc.org/github.com/dotchain/dot/ops/pg for an example implementiation (for Postgres 9.5+)

func Polled

func Polled(s Store) Store

Polled implements a low-latency in memory poller. Any append will effectively trigger any pending polls to return.

It can be used to wrap a store that does not support long polling

Note: closing the wrapped store closes the original store as well

func Transformed

func Transformed(raw Store, cache Cache) Store

Transformed takes a store of raw operations and converts them to a transformed version that can be applied in sequence.

The transformed Store only modifies the GetSince method.

Every operation has a Basis (last server acknowledged server operation applied before) and a Parent (the previous client operation on top of which the current operation is applied)

An operation is written into the Store in the order it is received but this implies that a bunch of operations may have been appended into the store between the parent/basis and the current version.

Applying this operation directly onto the last version can lead to incorrect results. The transformed operation merges the operation against all intervening operations, so applying it on top of the previous version will reflect the intent of the original operation.

The cache is required for efficiency reasons.