rapid

README

Rapid: membership service with consistent view changes.

WORK IN PROGRESS

Package with membership service with decentralized monitoring topology and consistent propagation of changes of the cluster configuration.

Monitoring topology is created from K graphs, where every node must monitor and monitored by K nodes. According to rapid technical report current monitoring topology will guaranteee almost-everywhere detection. In case when majority of the cluster detects change it will take 2 network delays for the cluster to detect new configuration. If there are conflicts it will take 3 more network delays to reach an agreement.

Service is using grpc for all network communication.

Contents

• Configuration
• API
• Specs
• License
• References

Configuration

type Config struct {
        // Expected network delay used for ticks
        NetworkDelay StringDuration

        // Paxos

        // Timeouts are expressed in number of network delays.
        // ElectionTimeout if replica doesn't receive hearbeat for ElectionTimeout ticks
        // it will start new election, by sending Prepare to other replicas.
        ElectionTimeout int
        // HeartbeatTimeout must be lower than election timeout.
        // Leader will send last sent Accept message to every replica as a heartbeat.
        HeartbeatTimeout int

        // Monitoring

        // Timeouts are expressed in number of network delays.
        // Each observer for the same subject must reinforce other observer vote after reinforce timeout.
        ReinforceTimeout int
        // RetransmitTimeout used to re-broadcast observed alerts.
        RetransmitTimeout int

        // Connectivity is a K paramter, used for monitoring topology construction.
        // Each node will have K observers and K subjects.
        Connectivity  int
        LowWatermark  int
        HighWatermark int

        // IP and Port of the seed.
        Seed *types.Node

        // IP and Port of the instance.
        IP   string
        Port uint64

        DialTimeout, SendTimeout time.Duration
}

API

API may change during development

Rapid instance needs to be initialized with logger, configuration and failure detector. In the example i am using simple prober, that dials once every period and if it can't reach the node consecutively it exits with notification. This can be replaced with application-based failure detector, phi acrual failure detector, etc.

The interface for failure detector is:

type FailureDetector interface {
        Monitor(context.Context, *types.Node) error
}

To bootstrap a cluster, start a single seed node and connect every other node to this seed:

logger := zap.NewNop()
fd := prober.New(logger.Sugar(), 2*time.Second, 2*time.Second, 3)
updates := make(chan *types.Configuration, 1)
if err := rapid.New(logger, conf, fd).Run(ctx, updates); err != nil {
   panic(err)
}

Specs

Latest specs can be foudn at specs.

License

MIT © Dmitry Shulyak

References

1. Stable and Consistent Membership at Scale with Rapid

Documentation

Index

• func Compare(a, b *types.Node) bool
• type Config
• type Network
• type Rapid
  • func New(logger *zap.Logger, conf Config, fd monitor.FailureDetector) Rapid
  • func (r Rapid) Configuration() (*types.Configuration, <-chan struct{})
  • func (r Rapid) Run(ctx context.Context) error
• type StringDuration
  • func (d StringDuration) MarshalJSON() ([]byte, error)
  • func (d *StringDuration) UnmarshalJSON(b []byte) error

Functions

func Compare

func Compare(a, b *types.Node) bool

Types

type Config

type Config struct {
	// Expected network delay. Used as a unit for ticks.
	NetworkDelay StringDuration

	// Timeout after value was proposed, replica will wait for Timeout
	// before executing fallback with classic paxos
	Timeout int

	// Each observer for the same subject must reinforce other observer vote after reinforce timeout.
	ReinforceTimeout int

	// Connectivity is a K paramter, used for monitoring topology construction.
	// Each node will have K observers and K subjects.
	Connectivity  int
	LowWatermark  int
	HighWatermark int

	// Network
	BroadcastFanout          int
	RetryPeriod              StringDuration
	DialTimeout, SendTimeout StringDuration

	// Bootstrap
	JoinTries   int
	JoinTimeout StringDuration

	Seed *types.Node

	IP   string
	Port uint64
}

type Network

type Network interface {
	network.Network
	bootstrap.Network
	Listen(ctx context.Context, node *types.Node) error
}

type Rapid

type Rapid struct {
	// contains filtered or unexported fields
}

func New

func New(
	logger *zap.Logger,
	conf Config,
	fd monitor.FailureDetector,
) Rapid

TODO maybe replace logger with interface

func (Rapid) Configuration

func (r Rapid) Configuration() (*types.Configuration, <-chan struct{})

func (Rapid) Run

func (r Rapid) Run(ctx context.Context) error

type StringDuration

type StringDuration time.Duration

func (StringDuration) MarshalJSON

func (d StringDuration) MarshalJSON() ([]byte, error)

func (*StringDuration) UnmarshalJSON

func (d *StringDuration) UnmarshalJSON(b []byte) error