phi

README

phi

Phi is a framework for build completely distributed and decentralized applications, such as a file-system, distributed lock, key-value store and object store. It is inspired from a combination of underlying mechanisms such as blockchain, content-addressable-storage, distributed-hash-tables to name a few.

The primary purpose of phi is to provide the following functionality:

• Distributed consistent Write-Ahead-Log with P2P consensus
• Distributed Hash Table
• Distributed Content-Addressable storage

Features:

• Completely distributed and decentralized architecture
• Distributed Write-Ahead-Log for consensus
• Distributed Hash Table
• Distributed Content-Addressable block based storage
• Automatic data de-duplication
• Automatic replication and healing

Architecture & Design

Phi's architecture combines several methodologies to provide a truly distributed and decentralized systems. Some of the design patterns used are:

• Content addressable storage
• Consistent hashing
• Multi-Version Concurrency Control
• Blockchain
• Gossip
• Distribute-Hash-Tables

Development

Ports

The following ports are used depending on the port configuration:

• TCP/UDP: Membership changes and fault-tolerance
• TCP/UDP: Lookups and Data access
• GRPC: Write Ahead Log

Note: When using debug mode a significant performance degrade may be seen.

Known Issues

• When using phi in docker on a Mac with persistent storage, a massive performance hit is incurred due to the way docker volumes and persistence are managed by docker on a Mac. This is only pertinent for Macs

Documentation

Index

• type BlockDevice
  • func NewBlockDevice(replicas int, hashFunc func() hash.Hash, local hexatype.Node, ...) *BlockDevice
  • func (dev *BlockDevice) BlockExists(id []byte) (bool, error)
  • func (dev *BlockDevice) Close() error
  • func (dev *BlockDevice) GetBlock(id []byte) (block.Block, error)
  • func (dev *BlockDevice) Hasher() func() hash.Hash
  • func (dev *BlockDevice) Register(blkDev *device.BlockDevice)
  • func (dev *BlockDevice) RegisterDHT(dht DHT)
  • func (dev *BlockDevice) RemoveBlock(id []byte) error
  • func (dev *BlockDevice) SetBlock(blk block.Block) ([]byte, error)
  • func (dev *BlockDevice) Stats() *device.Stats
• type Config
  • func DefaultConfig() *Config
  • func (config *Config) HashFunc() hash.Hash
  • func (config *Config) SetHashFunc(hf func() hash.Hash)
• type DHT
• type FSM
• type Hexalog
  • func NewHexalog(trans WALTransport, minVotes int, hashFunc func() hash.Hash) *Hexalog
  • func (hexlog *Hexalog) GetEntry(key, id []byte) (*hexalog.Entry, error)
  • func (hexlog *Hexalog) NewEntry(key []byte) (*hexalog.Entry, []*hexalog.Participant, error)
  • func (hexlog *Hexalog) NewEntryFrom(entry *hexalog.Entry) (*hexalog.Entry, []*hexalog.Participant, error)
  • func (hexlog *Hexalog) ProposeEntry(entry *hexalog.Entry, opts *hexalog.RequestOptions, retry *RetryOptions) (eid []byte, stats *WriteStats, err error)
  • func (hexlog *Hexalog) RegisterJury(jury Jury)
• type Jury
• type Phi
  • func Create(conf *Config, fsm FSM) (*Phi, error)
  • func (phi *Phi) BlockDevice() *BlockDevice
  • func (phi *Phi) BlockRemove(id []byte)
  • func (phi *Phi) BlockSet(index device.IndexEntry)
  • func (phi *Phi) DHT() DHT
  • func (phi *Phi) Join(existing []string) error
  • func (phi *Phi) LocalNode() hexatype.Node
  • func (phi *Phi) Shutdown() error
  • func (phi *Phi) WAL() WAL
• type RetryOptions
  • func DefaultRetryOptions() *RetryOptions
• type SimpleJury
  • func (jury *SimpleJury) Participants(key []byte, min int) ([]*hexalog.Participant, error)
  • func (jury *SimpleJury) RegisterDHT(dht DHT)
• type WAL
• type WALTransport
• type WriteStats

Types

type BlockDevice

type BlockDevice struct {
	// contains filtered or unexported fields
}

BlockDevice implements the blox.BlockDevice interface backed by a dht to distribute blocks into the cluster. It serves as a Participant as well as a client

func NewBlockDevice

func NewBlockDevice(replicas int, hashFunc func() hash.Hash, local hexatype.Node, idx device.BlockIndex, trans blox.Transport) *BlockDevice

NewBlockDevice inits a new Device that implements a BlockDevice that is leverages the dht with the given replica count, hash function and blox transport.

func (*BlockDevice) BlockExists

func (dev *BlockDevice) BlockExists(id []byte) (bool, error)

BlockExists returns true if the block exists on any one of the assigned nodes

func (*BlockDevice) Close

func (dev *BlockDevice) Close() error

Close shutdowns the underlying network transport

func (*BlockDevice) GetBlock

func (dev *BlockDevice) GetBlock(id []byte) (block.Block, error)

GetBlock gets a block from the device

func (*BlockDevice) Hasher

func (dev *BlockDevice) Hasher() func() hash.Hash

Hasher returns the hash function generator for hash ids for the device

func (*BlockDevice) Register

func (dev *BlockDevice) Register(blkDev *device.BlockDevice)

Register registers the actual block device to the transport. This is used in the case where the node is a member of the cluster rather than just a client

func (*BlockDevice) RegisterDHT

func (dev *BlockDevice) RegisterDHT(dht DHT)

RegisterDHT registers the DHT to the device. This device is only usable once a call to register has been made.

func (*BlockDevice) RemoveBlock

func (dev *BlockDevice) RemoveBlock(id []byte) error

RemoveBlock submits a request to remove a block on the device and all replicas

func (*BlockDevice) SetBlock

func (dev *BlockDevice) SetBlock(blk block.Block) ([]byte, error)

SetBlock writes the block to the device

func (*BlockDevice) Stats

func (dev *BlockDevice) Stats() *device.Stats

Stats is to satisfy the interface

type Config

type Config struct {
	// Block replicas
	Replicas int

	// Data directory
	DataDir string

	// WAL buffer size when seeding data on bootstrap
	WalSeedBuffSize int

	// WAL parallel go-routines for seeding
	WalSeedParallel int

	// Any existing peers. This will automatically cause the node to join the
	// cluster
	Peers []string

	// Membership and fault-tolerance
	Memberlist *memberlist.Config

	// WAL
	Hexalog *hexalog.Config

	DHT *kelips.Config

	// Hexalog jury selection algorithm to use
	Jury Jury

	// Grpc server to allow user services to be registered
	GRPCServer *grpc.Server
}

Config is the fidias config

func DefaultConfig

func DefaultConfig() *Config

DefaultConfig returns a minimally required config

func (*Config) HashFunc

func (config *Config) HashFunc() hash.Hash

HashFunc returns the hash function used for the fidias as a whole. These will always match between the the dht, blox and hexalog

func (*Config) SetHashFunc

func (config *Config) SetHashFunc(hf func() hash.Hash)

SetHashFunc sets the hash function for hexalog and the dht

type DHT

type DHT interface {
	LookupNodes(key []byte, min int) ([]*hexatype.Node, error)
	LookupGroupNodes(key []byte) ([]*hexatype.Node, error)
	Lookup(key []byte) ([]*hexatype.Node, error)
	Insert(key []byte, tuple kelips.TupleHost) error
	Delete(key []byte, tuple kelips.TupleHost) error
}

DHT implements a distributed hash table needed to route keys

type FSM

type FSM interface {
	Apply(entryID []byte, entry *hexalog.Entry) interface{}
	RegisterDHT(dht DHT)
}

FSM implements a phi fsm using a dht

type Hexalog

type Hexalog struct {
	// contains filtered or unexported fields
}

Hexalog is a network aware Hexalog. It implements selecting the participants from the network for consistency

func NewHexalog

func NewHexalog(trans WALTransport, minVotes int, hashFunc func() hash.Hash) *Hexalog

NewHexalog inita a new DHT aware WAL

func (*Hexalog) GetEntry

func (hexlog *Hexalog) GetEntry(key, id []byte) (*hexalog.Entry, error)

GetEntry tries to get an entry from the network from all known locations

func (*Hexalog) NewEntry

func (hexlog *Hexalog) NewEntry(key []byte) (*hexalog.Entry, []*hexalog.Participant, error)

NewEntry returns a new Entry for the given key from Hexalog. It returns an error if the node is not part of the location set or a lookup error occurs

func (*Hexalog) NewEntryFrom

func (hexlog *Hexalog) NewEntryFrom(entry *hexalog.Entry) (*hexalog.Entry, []*hexalog.Participant, error)

NewEntryFrom creates a new entry based on the given entry. It uses the given height and previous hash of the entry to determine the values for the new entry. This is essentially a compare and set

func (*Hexalog) ProposeEntry

func (hexlog *Hexalog) ProposeEntry(entry *hexalog.Entry, opts *hexalog.RequestOptions, retry *RetryOptions) (eid []byte, stats *WriteStats, err error)

ProposeEntry finds locations for the entry and proposes it to those locations It retries the specified number of times before returning. It returns a an entry id on success and error otherwise

func (*Hexalog) RegisterJury

func (hexlog *Hexalog) RegisterJury(jury Jury)

RegisterJury registers a jury interface used to get participants

type Jury

type Jury interface {
	Participants(key []byte, min int) ([]*hexalog.Participant, error)
	RegisterDHT(dht DHT)
}

Jury implements an interface to get participants for an entry proposal. These are the peers participating in the voting process

type Phi

type Phi struct {
	// contains filtered or unexported fields
}

Phi is the core engine for a participating node

func Create

func Create(conf *Config, fsm FSM) (*Phi, error)

Create creates a new Phi instance. It inits the local node, gossip layer and associated delegates

func (*Phi) BlockDevice

func (phi *Phi) BlockDevice() *BlockDevice

BlockDevice returns a cluster aware block device

func (*Phi) BlockRemove

func (phi *Phi) BlockRemove(id []byte)

BlockRemove is the blox delegate to handle block removals from the dht

func (*Phi) BlockSet

func (phi *Phi) BlockSet(index device.IndexEntry)

BlockSet is the blox delegate called when new blocks are set. It handles block inserts to the dht

func (*Phi) DHT

func (phi *Phi) DHT() DHT

DHT returns a distributed hash table interface

func (*Phi) Join

func (phi *Phi) Join(existing []string) error

Join joins the gossip networking using an existing node

func (*Phi) LocalNode

func (phi *Phi) LocalNode() hexatype.Node

LocalNode returns the local node from the dht. This will be different from the internal local which is cached

func (*Phi) Shutdown

func (phi *Phi) Shutdown() error

Shutdown performs a complete shutdown of all components

func (*Phi) WAL

func (phi *Phi) WAL() WAL

WAL returns the write-ahead-log for consistent operations

type RetryOptions

type RetryOptions struct {
	Retries       int
	RetryInterval time.Duration
}

RetryOptions for WAL proposals

func DefaultRetryOptions

func DefaultRetryOptions() *RetryOptions

DefaultRetryOptions returns a default set of RetryOptions

type SimpleJury

type SimpleJury struct {
	// contains filtered or unexported fields
}

SimpleJury implements a Jury interface. It returns the first n nodes available for a given key.

func (*SimpleJury) Participants

func (jury *SimpleJury) Participants(key []byte, min int) ([]*hexalog.Participant, error)

Participants gets the AffinityGroup group for the key and returns the nodes in that group as participants

func (*SimpleJury) RegisterDHT

func (jury *SimpleJury) RegisterDHT(dht DHT)

RegisterDHT registers a dht interface required to get participants

type WAL

type WAL interface {
	NewEntry(key []byte) (*hexalog.Entry, []*hexalog.Participant, error)
	NewEntryFrom(entry *hexalog.Entry) (*hexalog.Entry, []*hexalog.Participant, error)
	ProposeEntry(entry *hexalog.Entry, opts *hexalog.RequestOptions, retry *RetryOptions) ([]byte, *WriteStats, error)
	GetEntry(key []byte, id []byte) (*hexalog.Entry, error)
	RegisterJury(jury Jury)
}

WAL implements an interface to provide p2p distributed consensus

type WALTransport

type WALTransport interface {
	NewEntry(host string, key []byte, opts *hexalog.RequestOptions) (*hexalog.Entry, error)
	ProposeEntry(ctx context.Context, host string, entry *hexalog.Entry, opts *hexalog.RequestOptions) (*hexalog.ReqResp, error)
	GetEntry(host string, key []byte, id []byte, opts *hexalog.RequestOptions) (*hexalog.Entry, error)
}

WALTransport implements an interface for network log operations

type WriteStats

type WriteStats struct {
	BallotTime   time.Duration
	ApplyTime    time.Duration
	Participants []*hexalog.Participant
}

WriteStats contains stats regarding a write operation to the log