raft

package
v0.0.0-...-a52a6fd Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Dec 1, 2020 License: Apache-2.0, MIT, Apache-2.0, + 1 more Imports: 30 Imported by: 0

Documentation

Overview

Package raft implements a Consensus component for IPFS Cluster which uses Raft (go-libp2p-raft).

Index

Constants

View Source
const (
	LogOpPin = iota + 1
	LogOpUnpin
)

Type of consensus operation

Variables

View Source
var (
	DefaultDataSubFolder        = "raft"
	DefaultWaitForLeaderTimeout = 15 * time.Second
	DefaultCommitRetries        = 1
	DefaultNetworkTimeout       = 10 * time.Second
	DefaultCommitRetryDelay     = 200 * time.Millisecond
	DefaultBackupsRotate        = 6
	DefaultDatastoreNamespace   = "/r" // from "/raft"
)

Configuration defaults

View Source
var RaftLogCacheSize = 512

RaftLogCacheSize is the maximum number of logs to cache in-memory. This is used to reduce disk I/O for the recently committed entries.

View Source
var RaftMaxSnapshots = 5

RaftMaxSnapshots indicates how many snapshots to keep in the consensus data folder. TODO: Maybe include this in Config. Not sure how useful it is to touch this anyways.

Functions

func CleanupRaft

func CleanupRaft(cfg *Config) error

CleanupRaft moves the current data folder to a backup location

func LastStateRaw

func LastStateRaw(cfg *Config) (io.Reader, bool, error)

LastStateRaw returns the bytes of the last snapshot stored, its metadata, and a flag indicating whether any snapshot was found.

func OfflineState

func OfflineState(cfg *Config, store ds.Datastore) (state.State, error)

OfflineState state returns a cluster state by reading the Raft data and writing it to the given datastore which is then wrapped as a state.State. Usually an in-memory datastore suffices. The given datastore should be thread-safe.

func SnapshotSave

func SnapshotSave(cfg *Config, newState state.State, pids []peer.ID) error

SnapshotSave saves the provided state to a snapshot in the raft data path. Old raft data is backed up and replaced by the new snapshot. pids contains the config-specified peer ids to include in the snapshot metadata if no snapshot exists from which to copy the raft metadata

Types

type Config

type Config struct {
	config.Saver

	// A folder to store Raft's data.
	DataFolder string

	// InitPeerset provides the list of initial cluster peers for new Raft
	// peers (with no prior state). It is ignored when Raft was already
	// initialized or when starting in staging mode.
	InitPeerset []peer.ID
	// LeaderTimeout specifies how long to wait for a leader before
	// failing an operation.
	WaitForLeaderTimeout time.Duration
	// NetworkTimeout specifies how long before a Raft network
	// operation is timed out
	NetworkTimeout time.Duration
	// CommitRetries specifies how many times we retry a failed commit until
	// we give up.
	CommitRetries int
	// How long to wait between retries
	CommitRetryDelay time.Duration
	// BackupsRotate specifies the maximum number of Raft's DataFolder
	// copies that we keep as backups (renaming) after cleanup.
	BackupsRotate int
	// Namespace to use when writing keys to the datastore
	DatastoreNamespace string

	// A Hashicorp Raft's configuration object.
	RaftConfig *hraft.Config

	// Tracing enables propagation of contexts across binary boundaries.
	Tracing bool
	// contains filtered or unexported fields
}

Config allows to configure the Raft Consensus component for ipfs-cluster. The component's configuration section is represented by ConfigJSON. Config implements the ComponentConfig interface.

func (*Config) ApplyEnvVars

func (cfg *Config) ApplyEnvVars() error

ApplyEnvVars fills in any Config fields found as environment variables.

func (*Config) ConfigKey

func (cfg *Config) ConfigKey() string

ConfigKey returns a human-friendly indentifier for this Config.

func (*Config) Default

func (cfg *Config) Default() error

Default initializes this configuration with working defaults.

func (*Config) GetDataFolder

func (cfg *Config) GetDataFolder() string

GetDataFolder returns the Raft data folder that we are using.

func (*Config) LoadJSON

func (cfg *Config) LoadJSON(raw []byte) error

LoadJSON parses a json-encoded configuration (see jsonConfig). The Config will have default values for all fields not explicited in the given json object.

func (*Config) ToDisplayJSON

func (cfg *Config) ToDisplayJSON() ([]byte, error)

ToDisplayJSON returns JSON config as a string.

func (*Config) ToJSON

func (cfg *Config) ToJSON() ([]byte, error)

ToJSON returns the pretty JSON representation of a Config.

func (*Config) Validate

func (cfg *Config) Validate() error

Validate checks that this configuration has working values, at least in appearance.

type Consensus

type Consensus struct {
	// contains filtered or unexported fields
}

Consensus handles the work of keeping a shared-state between the peers of an IPFS Cluster, as well as modifying that state and applying any updates in a thread-safe manner.

func NewConsensus

func NewConsensus(
	host host.Host,
	cfg *Config,
	store ds.Datastore,
	staging bool,
) (*Consensus, error)

NewConsensus builds a new ClusterConsensus component using Raft.

Raft saves state snapshots regularly and persists log data in a bolt datastore. Therefore, unless memory usage is a concern, it is recommended to use an in-memory go-datastore as store parameter.

The staging parameter controls if the Raft peer should start in staging mode (used when joining a new Raft peerset with other peers).

The store parameter should be a thread-safe datastore.

func (*Consensus) AddPeer

func (cc *Consensus) AddPeer(ctx context.Context, pid peer.ID) error

AddPeer adds a new peer to participate in this consensus. It will forward the operation to the leader if this is not it.

func (*Consensus) Clean

func (cc *Consensus) Clean(ctx context.Context) error

Clean removes the Raft persisted state.

func (*Consensus) Distrust

func (cc *Consensus) Distrust(ctx context.Context, pid peer.ID) error

Distrust is a no-op.

func (*Consensus) IsTrustedPeer

func (cc *Consensus) IsTrustedPeer(ctx context.Context, p peer.ID) bool

IsTrustedPeer returns true. In Raft we trust all peers.

func (*Consensus) Leader

func (cc *Consensus) Leader(ctx context.Context) (peer.ID, error)

Leader returns the peerID of the Leader of the cluster. It returns an error when there is no leader.

func (*Consensus) LogPin

func (cc *Consensus) LogPin(ctx context.Context, pin *api.Pin) error

LogPin submits a Cid to the shared state of the cluster. It will forward the operation to the leader if this is not it.

func (*Consensus) LogUnpin

func (cc *Consensus) LogUnpin(ctx context.Context, pin *api.Pin) error

LogUnpin removes a Cid from the shared state of the cluster.

func (*Consensus) Peers

func (cc *Consensus) Peers(ctx context.Context) ([]peer.ID, error)

Peers return the current list of peers in the consensus. The list will be sorted alphabetically.

func (*Consensus) Ready

func (cc *Consensus) Ready(ctx context.Context) <-chan struct{}

Ready returns a channel which is signaled when the Consensus algorithm has finished bootstrapping and is ready to use

func (*Consensus) RmPeer

func (cc *Consensus) RmPeer(ctx context.Context, pid peer.ID) error

RmPeer removes a peer from this consensus. It will forward the operation to the leader if this is not it.

func (*Consensus) Rollback

func (cc *Consensus) Rollback(state state.State) error

Rollback replaces the current agreed-upon state with the state provided. Only the consensus leader can perform this operation.

func (*Consensus) SetClient

func (cc *Consensus) SetClient(c *rpc.Client)

SetClient makes the component ready to perform RPC requets

func (*Consensus) Shutdown

func (cc *Consensus) Shutdown(ctx context.Context) error

Shutdown stops the component so it will not process any more updates. The underlying consensus is permanently shutdown, along with the libp2p transport.

func (*Consensus) State

func (cc *Consensus) State(ctx context.Context) (state.ReadOnly, error)

State retrieves the current consensus State. It may error if no State has been agreed upon or the state is not consistent. The returned State is the last agreed-upon State known by this node. No writes are allowed, as all writes to the shared state should happen through the Consensus component methods.

func (*Consensus) Trust

func (cc *Consensus) Trust(ctx context.Context, pid peer.ID) error

Trust is a no-op.

func (*Consensus) WaitForSync

func (cc *Consensus) WaitForSync(ctx context.Context) error

WaitForSync waits for a leader and for the state to be up to date, then returns.

type LogOp

type LogOp struct {
	SpanCtx trace.SpanContext `codec:"s,omitempty"`
	TagCtx  []byte            `codec:"t,omitempty"`
	Cid     *api.Pin          `codec:"c,omitempty"`
	Type    LogOpType         `codec:"p,omitempty"`
	// contains filtered or unexported fields
}

LogOp represents an operation for the OpLogConsensus system. It implements the consensus.Op interface and it is used by the Consensus component.

func (*LogOp) ApplyTo

func (op *LogOp) ApplyTo(cstate consensus.State) (consensus.State, error)

ApplyTo applies the operation to the State

type LogOpType

type LogOpType int

LogOpType expresses the type of a consensus Operation

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL