Version: v0.0.0-...-a4757fd Latest Latest

This package is not in the latest version of its module.

Go to latest
Published: Jul 13, 2018 License: Apache-2.0, Apache-2.0 Imports: 31 Imported by: 0


The core consensus algorithm.

  • state.go - The state machine as detailed in the whitepaper
  • reactor.go - A reactor that connects the state machine to the gossip network

Go-routine summary

The reactor runs 2 go-routines for each added peer: gossipDataRoutine and gossipVotesRoutine.

The consensus state runs two persistent go-routines: timeoutRoutine and receiveRoutine. Go-routines are also started to trigger timeouts and to avoid blocking when the internalMsgQueue is really backed up.


A write-ahead log is used to record all messages processed by the receiveRoutine, which amounts to all inputs to the consensus state machine: messages from peers, messages from ourselves, and timeouts. They can be played back deterministically at startup or using the replay console.




View Source
const (
	StateChannel       = byte(0x20)
	DataChannel        = byte(0x21)
	VoteChannel        = byte(0x22)
	VoteSetBitsChannel = byte(0x23)
View Source
const (
	CssMsgTypeProposal  = byte(0x01)
	CssMsgTypeBlockPart = byte(0x02)
	CssMsgTypeVote      = byte(0x03)
View Source
const (
	WALMsgTypeRoundState  = byte(0x01)
	WALMsgTypeMsgInfo     = byte(0x02)
	WALMsgTypeTimeoutInfo = byte(0x03)


View Source
var (
	ErrPeerStateHeightRegression = errors.New("Error peer state height regression")
	ErrPeerStateInvalidStartTime = errors.New("Error peer state invalid startTime")
View Source
var (
	ErrInvalidProposalSignature = errors.New("Error invalid proposal signature")
	ErrInvalidProposalPOLRound  = errors.New("Error invalid proposal POL round")
	ErrAddingVote               = errors.New("Error adding vote")
	ErrVoteHeightMismatch       = errors.New("Error vote height mismatch")
View Source
var Major = "0" //
View Source
var Minor = "2" // replay refactor
View Source
var Revision = "2" // validation -> commit
View Source
var Spec = "1" // async

kind of arbitrary

View Source
var Version = Fmt("v%s/%s.%s.%s", Spec, Major, Minor, Revision)


func CompareHRS

func CompareHRS(h1, r1 def.INT, s1 csspb.RoundStepType, h2, r2 def.INT, s2 csspb.RoundStepType) int


type ConsensusReactor

type ConsensusReactor struct {
	p2p.BaseReactor // BaseService + p2p.Switch
	// contains filtered or unexported fields

func NewConsensusReactor

func NewConsensusReactor(logger *zap.Logger, consensusState *ConsensusState, fastSync bool) *ConsensusReactor

func (*ConsensusReactor) AddPeer

func (conR *ConsensusReactor) AddPeer(peer *p2p.Peer)

Implements Reactor

func (*ConsensusReactor) GetChannels

func (conR *ConsensusReactor) GetChannels() []*p2p.ChannelDescriptor

Implements Reactor

func (*ConsensusReactor) OnStart

func (conR *ConsensusReactor) OnStart() error

func (*ConsensusReactor) OnStop

func (conR *ConsensusReactor) OnStop()

func (*ConsensusReactor) Receive

func (conR *ConsensusReactor) Receive(chID byte, src *p2p.Peer, msgBytes []byte)

Implements Reactor NOTE: We process these messages even when we're fast_syncing. Messages affect either a peer state or the consensus state. Peer state updates can happen in parallel, but processing of proposals, block parts, and votes are ordered by the receiveRoutine NOTE: blocks on consensus state for proposals, block parts, and votes

func (*ConsensusReactor) RemovePeer

func (conR *ConsensusReactor) RemovePeer(peer *p2p.Peer, reason interface{})

Implements Reactor

func (*ConsensusReactor) SetEventSwitch

func (conR *ConsensusReactor) SetEventSwitch(evsw agtypes.EventSwitch)

implements events.Eventable

func (*ConsensusReactor) String

func (conR *ConsensusReactor) String() string

func (*ConsensusReactor) StringIndented

func (conR *ConsensusReactor) StringIndented(indent string) string

func (*ConsensusReactor) SwitchToConsensus

func (conR *ConsensusReactor) SwitchToConsensus(state *sm.State)

Switch from the fast_sync to the consensus: reset the state, turn off fast_sync, start the consensus-state-machine

type ConsensusState

type ConsensusState struct {

	// contains filtered or unexported fields

Tracks consensus state across block heights and rounds.

func NewConsensusState

func NewConsensusState(logger *zap.Logger, config *viper.Viper, state *sm.State, blockStore *bc.BlockStore, mempool *mempl.Mempool) *ConsensusState

func (*ConsensusState) AddProposalBlockPart

func (cs *ConsensusState) AddProposalBlockPart(height, round def.INT, part *pbtypes.Part, peerKey string) error

May block on send if queue is full.

func (*ConsensusState) AddVote

func (cs *ConsensusState) AddVote(vote *pbtypes.Vote, peerKey string) (added bool, err error)

May block on send if queue is full.

func (*ConsensusState) BindReactor

func (cs *ConsensusState) BindReactor(r *ConsensusReactor)

func (*ConsensusState) CatchupReplay

func (cs *ConsensusState) CatchupReplay(height def.INT) error

func (*ConsensusState) GetRoundState

func (cs *ConsensusState) GetRoundState() *RoundState

func (*ConsensusState) GetState

func (cs *ConsensusState) GetState() *sm.State

func (*ConsensusState) GetTotalVotingPower

func (cs *ConsensusState) GetTotalVotingPower() int64

func (*ConsensusState) GetValidatorSet

func (cs *ConsensusState) GetValidatorSet() *agtypes.ValidatorSet

func (*ConsensusState) LoadCommit

func (cs *ConsensusState) LoadCommit(height def.INT) *agtypes.CommitCache

func (*ConsensusState) OnStart

func (cs *ConsensusState) OnStart() error

func (*ConsensusState) OnStop

func (cs *ConsensusState) OnStop()

func (*ConsensusState) OpenWAL

func (cs *ConsensusState) OpenWAL(walDir string) (err error)

Open file to log all consensus messages and timeouts for deterministic accountability

func (ConsensusState) ReplayConsole

func (cs ConsensusState) ReplayConsole(file string) error

Interactive playback

func (ConsensusState) ReplayMessages

func (cs ConsensusState) ReplayMessages(file string) error

Full playback, with tests

func (*ConsensusState) SetBadVoteCollector

func (cs *ConsensusState) SetBadVoteCollector(c agtypes.IBadVoteCollector)

func (*ConsensusState) SetEventSwitch

func (cs *ConsensusState) SetEventSwitch(evsw agtypes.EventSwitch)

implements events.Eventable

func (*ConsensusState) SetPrivValidator

func (cs *ConsensusState) SetPrivValidator(priv PrivValidator)

Sets our private validator account for signing votes.

func (*ConsensusState) SetProposal

func (cs *ConsensusState) SetProposal(proposal *pbtypes.Proposal, peerKey string) error

May block on send if queue is full.

func (*ConsensusState) SetProposalAndBlock

func (cs *ConsensusState) SetProposalAndBlock(proposal *pbtypes.Proposal, block *pbtypes.Block, parts *agtypes.PartSet, peerKey string) error

May block on send if queue is full.

func (*ConsensusState) SetTimeoutTicker

func (cs *ConsensusState) SetTimeoutTicker(timeoutTicker TimeoutTicker)

Set the local timer

func (*ConsensusState) SetValSetLoader

func (cs *ConsensusState) SetValSetLoader(f agtypes.ValSetLoaderFunc)

func (*ConsensusState) String

func (cs *ConsensusState) String() string

func (*ConsensusState) Wait

func (cs *ConsensusState) Wait()

NOTE: be sure to Stop() the event switch and drain any event channels or this may deadlock

type HeightVoteSet

type HeightVoteSet struct {
	// contains filtered or unexported fields

Keeps track of all VoteSets from round 0 to round 'round'.

Also keeps track of up to one RoundVoteSet greater than 'round' from each peer, to facilitate catchup syncing of commits.

A commit is +2/3 precommits for a block at a round, but which round is not known in advance, so when a peer provides a precommit for a round greater than mtx.round, we create a new entry in roundVoteSets but also remember the peer to prevent abuse. We let each peer provide us with up to 2 unexpected "catchup" rounds. One for their LastCommit round, and another for the official commit round.

func NewHeightVoteSet

func NewHeightVoteSet(chainID string, height def.INT, valSet *agtypes.ValidatorSet) *HeightVoteSet

func (*HeightVoteSet) AddVote

func (hvs *HeightVoteSet) AddVote(vote *pbtypes.Vote, peerKey string) (added bool, err error)

Duplicate votes return added=false, err=nil. By convention, peerKey is "" if origin is self.

func (*HeightVoteSet) Height

func (hvs *HeightVoteSet) Height() def.INT

func (*HeightVoteSet) POLInfo

func (hvs *HeightVoteSet) POLInfo() (polRound def.INT, polBlockID pbtypes.BlockID)

Last round and blockID that has +2/3 prevotes for a particular block or nil. Returns -1 if no such round exists.

func (*HeightVoteSet) Precommits

func (hvs *HeightVoteSet) Precommits(round def.INT) *agtypes.VoteSet

func (*HeightVoteSet) Prevotes

func (hvs *HeightVoteSet) Prevotes(round def.INT) *agtypes.VoteSet

func (*HeightVoteSet) Reset

func (hvs *HeightVoteSet) Reset(height def.INT, valSet *agtypes.ValidatorSet)

func (*HeightVoteSet) Round

func (hvs *HeightVoteSet) Round() def.INT

func (*HeightVoteSet) SetPeerMaj23

func (hvs *HeightVoteSet) SetPeerMaj23(round def.INT, type_ pbtypes.VoteType, peerID string, blockID *pbtypes.BlockID)

If a peer claims that it has 2/3 majority for given blockKey, call this. NOTE: if there are too many peers, or too much peer churn, this can cause memory issues. TODO: implement ability to remove peers too

func (*HeightVoteSet) SetRound

func (hvs *HeightVoteSet) SetRound(round def.INT)

Create more RoundVoteSets up to round.

func (*HeightVoteSet) String

func (hvs *HeightVoteSet) String() string

func (*HeightVoteSet) StringIndented

func (hvs *HeightVoteSet) StringIndented(indent string) string

type PeerRoundState

type PeerRoundState struct {
	Height                   def.INT               // Height peer is at
	Round                    def.INT               // Round peer is at, -1 if unknown.
	Step                     csspb.RoundStepType   // Step peer is at
	StartTime                time.Time             // Estimated start of round 0 at this height
	Proposal                 bool                  // True if peer has proposal for this round
	ProposalBlockPartsHeader pbtypes.PartSetHeader //
	ProposalBlockParts       *BitArray             //
	ProposalPOLRound         def.INT               // Proposal's POL round. -1 if none.
	ProposalPOL              *BitArray             // nil until ProposalPOLMessage received.
	Prevotes                 *BitArray             // All votes peer has for this round
	Precommits               *BitArray             // All precommits peer has for this round
	LastCommitRound          def.INT               // Round of commit for last height. -1 if none.
	LastCommit               *BitArray             // All commit precommits of commit for last height.
	CatchupCommitRound       def.INT               // Round that we have commit for. Not necessarily unique. -1 if none.
	CatchupCommit            *BitArray             // All commit precommits peer has for this height & CatchupCommitRound

Read only when returned by PeerState.GetRoundState().

func (PeerRoundState) String

func (prs PeerRoundState) String() string

func (PeerRoundState) StringIndented

func (prs PeerRoundState) StringIndented(indent string) string

type PeerState

type PeerState struct {
	Peer *p2p.Peer

	// contains filtered or unexported fields

func NewPeerState

func NewPeerState(slogger *zap.SugaredLogger, peer *p2p.Peer) *PeerState

func (*PeerState) ApplyCommitStepMessage

func (ps *PeerState) ApplyCommitStepMessage(msg *csspb.CommitStepMessage)

func (*PeerState) ApplyHasVoteMessage

func (ps *PeerState) ApplyHasVoteMessage(msg *csspb.HasVoteMessage)

func (*PeerState) ApplyNewRoundStepMessage

func (ps *PeerState) ApplyNewRoundStepMessage(msg *csspb.NewRoundStepMessage)

func (*PeerState) ApplyProposalPOLMessage

func (ps *PeerState) ApplyProposalPOLMessage(msg *csspb.ProposalPOLMessage)

func (*PeerState) ApplyVoteSetBitsMessage

func (ps *PeerState) ApplyVoteSetBitsMessage(msg *csspb.VoteSetBitsMessage, ourVotes *BitArray)

The peer has responded with a bitarray of votes that it has of the corresponding BlockID. ourVotes: BitArray of votes we have for msg.BlockID NOTE: if ourVotes is nil (e.g. msg.Height < rs.Height), we conservatively overwrite ps's votes w/ msg.Votes.

func (*PeerState) EnsureVoteBitArrays

func (ps *PeerState) EnsureVoteBitArrays(height def.INT, numValidators int)

NOTE: It's important to make sure that numValidators actually matches what the node sees as the number of validators for height.

func (*PeerState) GetHeight

func (ps *PeerState) GetHeight() def.INT

Returns an atomic snapshot of the PeerRoundState's height used by the mempool to ensure peers are caught up before broadcasting new txs

func (*PeerState) GetRoundState

func (ps *PeerState) GetRoundState() *PeerRoundState

Returns an atomic snapshot of the PeerRoundState. There's no point in mutating it since it won't change PeerState.

func (*PeerState) PickSendVote

func (ps *PeerState) PickSendVote(votes agtypes.VoteSetReader) (ok bool)

Convenience function to send vote to peer. Returns true if vote was sent.

func (*PeerState) PickVoteToSend

func (ps *PeerState) PickVoteToSend(votes agtypes.VoteSetReader) (vote *pbtypes.Vote, ok bool)

votes: Must be the correct Size() for the Height().

func (*PeerState) SetHasProposal

func (ps *PeerState) SetHasProposal(proposal *pbtypes.Proposal)

func (*PeerState) SetHasProposalBlockPart

func (ps *PeerState) SetHasProposalBlockPart(height, round def.INT, index int)

func (*PeerState) SetHasVote

func (ps *PeerState) SetHasVote(vote *pbtypes.Vote)

func (*PeerState) String

func (ps *PeerState) String() string

func (*PeerState) StringIndented

func (ps *PeerState) StringIndented(indent string) string

type PrivValidator

type PrivValidator interface {
	GetAddress() []byte
	GetCoinbase() []byte
	SignVote(chainID string, vote *pbtypes.Vote) error
	SignProposal(chainID string, proposal *pbtypes.Proposal) error

type RoundState

type RoundState struct {
	Height             def.INT // Height we are working on
	Round              def.INT
	Step               csspb.RoundStepType
	StartTime          time.Time
	CommitTime         time.Time // Subjective time when +2/3 precommits for Block at Round were found
	Validators         *agtypes.ValidatorSet
	Proposal           *pbtypes.Proposal
	ProposalBlock      *agtypes.BlockCache
	ProposalBlockParts *agtypes.PartSet
	LockedRound        def.INT
	LockedBlock        *agtypes.BlockCache
	LockedBlockParts   *agtypes.PartSet
	Votes              *HeightVoteSet
	CommitRound        def.INT          //
	LastCommit         *agtypes.VoteSet // Last precommits at Height-1
	LastValidators     *agtypes.ValidatorSet

Immutable when returned from ConsensusState.GetRoundState()

func (*RoundState) RoundStateEvent

func (rs *RoundState) RoundStateEvent() agtypes.EventDataRoundState

func (*RoundState) String

func (rs *RoundState) String() string

func (*RoundState) StringIndented

func (rs *RoundState) StringIndented(indent string) string

func (*RoundState) StringShort

func (rs *RoundState) StringShort() string

type RoundVoteSet

type RoundVoteSet struct {
	Prevotes   *agtypes.VoteSet
	Precommits *agtypes.VoteSet

type StWALMessage

type StWALMessage struct {

func (StWALMessage) MarshalJSON

func (sw StWALMessage) MarshalJSON() ([]byte, error)

func (*StWALMessage) UnmarshalJSON

func (sw *StWALMessage) UnmarshalJSON(data []byte) error

type TimedWALMessage

type TimedWALMessage struct {
	Time time.Time    `json:"time"`
	Msg  StWALMessage `json:"msg"`

func GenTimedWALMessage

func GenTimedWALMessage(msg WALMessage) (retMsg TimedWALMessage)

func (*TimedWALMessage) GetMsg

func (tw *TimedWALMessage) GetMsg() WALMessage

func (*TimedWALMessage) UnmarshalJSON

func (tw *TimedWALMessage) UnmarshalJSON(data []byte) error

type TimeoutParams

type TimeoutParams struct {
	Propose0          def.INT
	ProposeDelta      def.INT
	Prevote0          def.INT
	PrevoteDelta      def.INT
	Precommit0        def.INT
	PrecommitDelta    def.INT
	Commit0           def.INT
	SkipTimeoutCommit bool

TimeoutParams holds timeouts and deltas for each round step. All timeouts and deltas in milliseconds.

func InitTimeoutParamsFromConfig

func InitTimeoutParamsFromConfig(config *viper.Viper) *TimeoutParams

InitTimeoutParamsFromConfig initializes parameters from config

func (*TimeoutParams) Commit

func (tp *TimeoutParams) Commit(t time.Time) time.Time

After receiving +2/3 precommits for a single block (a commit), wait this long for stragglers in the next height's RoundStepNewHeight

func (*TimeoutParams) Precommit

func (tp *TimeoutParams) Precommit(round def.INT) time.Duration

After receiving any +2/3 precommits, wait this long for stragglers

func (*TimeoutParams) Prevote

func (tp *TimeoutParams) Prevote(round def.INT) time.Duration

After receiving any +2/3 prevote, wait this long for stragglers

func (*TimeoutParams) Propose

func (tp *TimeoutParams) Propose(round def.INT) time.Duration

Wait this long for a proposal

type TimeoutTicker

type TimeoutTicker interface {
	Start() (bool, error)
	Stop() bool
	Chan() <-chan timeoutInfo       // on which to receive a timeout
	ScheduleTimeout(ti timeoutInfo) // reset the timer

TimeoutTicker is a timer that schedules timeouts conditional on the height/round/step in the timeoutInfo. The timeoutInfo.Duration may be non-positive.

func NewTimeoutTicker

func NewTimeoutTicker(logger *zap.Logger) TimeoutTicker

type WAL

type WAL struct {
	// contains filtered or unexported fields

Write ahead logger writes msgs to disk before they are processed. Can be used for crash-recovery and deterministic replay TODO: currently the wal is overwritten during replay catchup

give it a mode so it's either reading or appending - must read to end to start appending again

func NewWAL

func NewWAL(logger *zap.Logger, walDir string, light bool) (*WAL, error)

func (*WAL) OnStart

func (wal *WAL) OnStart() error

func (*WAL) OnStop

func (wal *WAL) OnStop()

func (*WAL) Save

func (wal *WAL) Save(wmsg WALMessage)

called in newStep and for each pass in receiveRoutine

type WALMessage

type WALMessage interface {

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
t or T : Toggle theme light dark auto
y or Y : Canonical URL