crdt

package
v1.0.6 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Mar 6, 2023 License: Apache-2.0, MIT, Apache-2.0, + 1 more Imports: 28 Imported by: 3

Documentation

Overview

Package crdt implements the IPFS Cluster consensus interface using CRDT-datastore to replicate the cluster global state to every peer.

Index

Constants

This section is empty.

Variables

View Source 
var (
	DefaultClusterName          = "ipfs-cluster"
	DefaultPeersetMetric        = "ping"
	DefaultDatastoreNamespace   = "/c" // from "/crdt"
	DefaultRebroadcastInterval  = time.Minute
	DefaultTrustedPeers         = []peer.ID{}
	DefaultTrustAll             = true
	DefaultBatchingMaxQueueSize = 50000
	DefaultRepairInterval       = time.Hour
)

Default configuration values

View Source 
var (
	ErrNoLeader            = errors.New("crdt consensus component does not provide a leader")
	ErrRmPeer              = errors.New("crdt consensus component cannot remove peers")
	ErrMaxQueueSizeReached = errors.New("batching max_queue_size reached. Too many operations are waiting to be batched. Try increasing the max_queue_size or adjusting the batching options")
)

Common variables for the module.

View Source 
var (
	// BlocksNs is the namespace to use as blockstore with ipfs-lite.
	BlocksNs = "b"
)

Functions

func Clean 

func Clean(ctx context.Context, cfg *Config, store ds.Datastore) error

Clean deletes all crdt-consensus datas from the given datastore.

func OfflineState 

func OfflineState(cfg *Config, store ds.Datastore) (state.BatchingState, error)

OfflineState returns an offline, batching state using the given datastore. This allows to inspect and modify the shared state in offline mode.

Types

type BatchingConfig 

type BatchingConfig struct {
	MaxBatchSize int
	MaxBatchAge  time.Duration
	MaxQueueSize int
}

BatchingConfig configures parameters for batching multiple pins in a single CRDT-put operation.

MaxBatchSize will trigger a commit whenever the number of pins in the batch reaches the limit.

MaxBatchAge will trigger a commit when the oldest update in the batch reaches it. Setting both values to 0 means batching is disabled.

MaxQueueSize specifies how many items can be waiting to be batched before the LogPin/Unpin operations block.

type Config 

type Config struct {
	config.Saver

	// The topic we wish to subscribe to
	ClusterName string

	// TrustAll specifies whether we should trust all peers regardless of
	// the TrustedPeers contents.
	TrustAll bool

	// Any update received from a peer outside this set is ignored and not
	// forwarded. Trusted peers can also access additional RPC endpoints
	// for this peer that are forbidden for other peers.
	TrustedPeers []peer.ID

	// Specifies whether to batch CRDT updates for increased
	// performance.
	Batching BatchingConfig

	// The interval before re-announcing the current state
	// to the network when no activity is observed.
	RebroadcastInterval time.Duration

	// The name of the metric we use to obtain the peerset (every peer
	// with valid metric of this type is part of it).
	PeersetMetric string

	// All keys written to the datastore will be namespaced with this prefix
	DatastoreNamespace string

	// How often the underlying crdt store triggers a repair when the
	// datastore is marked dirty.
	RepairInterval time.Duration

	// Tracing enables propagation of contexts across binary boundaries.
	Tracing bool
	// contains filtered or unexported fields
}

Config is the configuration object for Consensus.

func (*Config) ApplyEnvVars 

func (cfg *Config) ApplyEnvVars() error

ApplyEnvVars fills in any Config fields found as environment variables.

func (*Config) ConfigKey 

func (cfg *Config) ConfigKey() string

ConfigKey returns the section name for this type of configuration.

func (*Config) Default 

func (cfg *Config) Default() error

Default sets the configuration fields to their default values.

func (*Config) LoadJSON 

func (cfg *Config) LoadJSON(raw []byte) error

LoadJSON takes a raw JSON slice and sets all the configuration fields.

func (*Config) ToDisplayJSON 

func (cfg *Config) ToDisplayJSON() ([]byte, error)

ToDisplayJSON returns JSON config as a string.

func (*Config) ToJSON 

func (cfg *Config) ToJSON() ([]byte, error)

ToJSON returns the JSON representation of this configuration.

func (*Config) Validate 

func (cfg *Config) Validate() error

Validate returns an error if the configuration has invalid values.

type Consensus 

type Consensus struct {
	// contains filtered or unexported fields
}

Consensus implement ipfscluster.Consensus and provides the facility to add and remove pins from the Cluster shared state. It uses a CRDT-backed implementation of go-datastore (go-ds-crdt).

func New 

func New(
	host host.Host,
	dht routing.Routing,
	pubsub *pubsub.PubSub,
	cfg *Config,
	store ds.Datastore,
) (*Consensus, error)

New creates a new crdt Consensus component. The given PubSub will be used to broadcast new heads. The given thread-safe datastore will be used to persist data and all will be prefixed with cfg.DatastoreNamespace.

func (*Consensus) AddPeer 

func (css *Consensus) AddPeer(ctx context.Context, pid peer.ID) error

AddPeer is a no-op as we do not need to do peerset management with Merkle-CRDTs. Therefore adding a peer to the peerset means doing nothing.

func (*Consensus) Clean 

func (css *Consensus) Clean(ctx context.Context) error

Clean deletes all crdt-consensus datas from the datastore.

func (*Consensus) Distrust 

func (css *Consensus) Distrust(ctx context.Context, pid peer.ID) error

Distrust removes a peer from the "trusted" set.

func (*Consensus) IsTrustedPeer 

func (css *Consensus) IsTrustedPeer(ctx context.Context, pid peer.ID) bool

IsTrustedPeer returns whether the given peer is taken into account when submitting updates to the consensus state.

func (*Consensus) Leader 

func (css *Consensus) Leader(ctx context.Context) (peer.ID, error)

Leader returns ErrNoLeader.

func (*Consensus) LogPin 

func (css *Consensus) LogPin(ctx context.Context, pin api.Pin) error

LogPin adds a new pin to the shared state.

func (*Consensus) LogUnpin 

func (css *Consensus) LogUnpin(ctx context.Context, pin api.Pin) error

LogUnpin removes a pin from the shared state.

func (*Consensus) Peers 

func (css *Consensus) Peers(ctx context.Context) ([]peer.ID, error)

Peers returns the current known peerset. It uses the monitor component and considers every peer with valid known metrics a member.

func (*Consensus) Ready 

func (css *Consensus) Ready(ctx context.Context) <-chan struct{}

Ready returns a channel which is signaled when the component is ready to use.

func (*Consensus) RmPeer 

func (css *Consensus) RmPeer(ctx context.Context, pid peer.ID) error

RmPeer is a no-op which always errors, as, since we do not do peerset management, we also have no ability to remove a peer from it.

func (*Consensus) SetClient 

func (css *Consensus) SetClient(c *rpc.Client)

SetClient gives the component the ability to communicate and leaves it ready to use.

func (*Consensus) Shutdown 

func (css *Consensus) Shutdown(ctx context.Context) error

Shutdown closes this component, canceling the pubsub subscription and closing the datastore.

func (*Consensus) State 

func (css *Consensus) State(ctx context.Context) (state.ReadOnly, error)

State returns the cluster shared state. It will block until the consensus component is ready, shutdown or the given context has been canceled.

func (*Consensus) Trust 

func (css *Consensus) Trust(ctx context.Context, pid peer.ID) error

Trust marks a peer as "trusted". It makes sure it is trusted as issuer for pubsub updates, it is protected in the connection manager, it has the highest priority when the peerstore is saved, and it's addresses are always remembered.

func (*Consensus) WaitForSync 

func (css *Consensus) WaitForSync(ctx context.Context) error

WaitForSync is a no-op as it is not necessary to be fully synced for the component to be usable.

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.