snap

package
Version: v1.9.1 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Aug 19, 2022 License: GPL-3.0 Imports: 26 Imported by: 0

Documentation

Index

Constants

View Source
const (
	GetAccountRangeMsg  = 0x00
	AccountRangeMsg     = 0x01
	GetStorageRangesMsg = 0x02
	StorageRangesMsg    = 0x03
	GetByteCodesMsg     = 0x04
	ByteCodesMsg        = 0x05
	GetTrieNodesMsg     = 0x06
	TrieNodesMsg        = 0x07
)
View Source
const ProtocolName = "snap"

ProtocolName is the official short name of the `snap` protocol used during devp2p capability negotiation.

View Source
const (
	SNAP1 = 1
)

Constants to match up protocol versions and messages

Variables

View Source
var ErrCancelled = errors.New("sync cancelled")

ErrCancelled is returned from snap syncing if the operation was prematurely terminated.

View Source
var ProtocolLengths = map[uint]uint64{SNAP1: 8}

ProtocolLengths are the number of implemented message corresponding to different protocol versions.

View Source
var ProtocolVersions = []uint{SNAP1}

ProtocolVersions are the supported versions of the `snap` protocol (first is primary).

Functions

func Handle

func Handle(reader SnapshotReader, downloader SnapshotDownloader, peer *Peer) error

Handle is the callback invoked to manage the life cycle of a `snap` peer. When this function terminates, the peer is disconnected.

func HandleMessage

func HandleMessage(reader SnapshotReader, downloader SnapshotDownloader, peer *Peer) error

HandleMessage is invoked whenever an inbound message is received from a remote peer on the `snap` protocol. The remote connection is torn down upon returning any error.

func ServiceGetByteCodesQuery

func ServiceGetByteCodesQuery(chain SnapshotReader, req *GetByteCodesPacket) [][]byte

ServiceGetByteCodesQuery assembles the response to a byte codes query. It is exposed to allow external packages to test protocol behavior.

func ServiceGetStorageRangesQuery

func ServiceGetStorageRangesQuery(chain SnapshotReader, req *GetStorageRangesPacket) ([][]*StorageData, [][]byte)

func ServiceGetTrieNodesQuery

func ServiceGetTrieNodesQuery(chain SnapshotReader, req *GetTrieNodesPacket, start time.Time) ([][]byte, error)

ServiceGetTrieNodesQuery assembles the response to a trie nodes query. It is exposed to allow external packages to test protocol behavior.

Types

type AccountData

type AccountData struct {
	Hash common.Hash // Hash of the account
	Body []byte      // Account body
}

AccountData represents a single account in a query response.

func ServiceGetAccountRangeQuery

func ServiceGetAccountRangeQuery(chain SnapshotReader, req *GetAccountRangePacket) ([]*AccountData, [][]byte)

ServiceGetAccountRangeQuery assembles the response to an account range query. It is exposed to allow external packages to test protocol behavior.

type AccountRangePacket

type AccountRangePacket struct {
	ID       uint64         // ID of the request this is a response for
	Accounts []*AccountData // List of consecutive accounts from the trie
	Proof    [][]byte       // List of trie nodes proving the account range
}

AccountRangePacket represents an account query response.

func (*AccountRangePacket) Kind

func (*AccountRangePacket) Kind() byte

func (*AccountRangePacket) Name

func (*AccountRangePacket) Name() string

func (*AccountRangePacket) Unpack

func (p *AccountRangePacket) Unpack() ([]common.Hash, [][]byte)

Unpack retrieves the accounts from the range packet and converts from slim wire representation to consensus format. The returned data is RLP encoded since it's expected to be serialized to disk without further interpretation.

Note, this method does a round of RLP decoding and reencoding, so only use it once and cache the results if need be. Ideally discard the packet afterwards to not double the memory use.

type ByteCodesPacket

type ByteCodesPacket struct {
	ID    uint64   // ID of the request this is a response for
	Codes [][]byte // Requested contract bytecodes
}

ByteCodesPacket represents a contract bytecode query response.

func (*ByteCodesPacket) Kind

func (*ByteCodesPacket) Kind() byte

func (*ByteCodesPacket) Name

func (*ByteCodesPacket) Name() string

type GetAccountRangePacket

type GetAccountRangePacket struct {
	ID     uint64      // Request ID to match up responses with
	Root   common.Hash // Root hash of the account trie to serve
	Origin common.Hash // Hash of the first account to retrieve
	Limit  common.Hash // Hash of the last account to retrieve
	Bytes  uint64      // Soft limit at which to stop returning data
}

GetAccountRangePacket represents an account query.

func (*GetAccountRangePacket) Kind

func (*GetAccountRangePacket) Kind() byte

func (*GetAccountRangePacket) Name

func (*GetAccountRangePacket) Name() string

type GetByteCodesPacket

type GetByteCodesPacket struct {
	ID     uint64        // Request ID to match up responses with
	Hashes []common.Hash // Code hashes to retrieve the code for
	Bytes  uint64        // Soft limit at which to stop returning data
}

GetByteCodesPacket represents a contract bytecode query.

func (*GetByteCodesPacket) Kind

func (*GetByteCodesPacket) Kind() byte

func (*GetByteCodesPacket) Name

func (*GetByteCodesPacket) Name() string

type GetStorageRangesPacket

type GetStorageRangesPacket struct {
	ID       uint64        // Request ID to match up responses with
	Root     common.Hash   // Root hash of the account trie to serve
	Accounts []common.Hash // Account hashes of the storage tries to serve
	Origin   []byte        // Hash of the first storage slot to retrieve (large contract mode)
	Limit    []byte        // Hash of the last storage slot to retrieve (large contract mode)
	Bytes    uint64        // Soft limit at which to stop returning data
}

GetStorageRangesPacket represents an storage slot query.

func (*GetStorageRangesPacket) Kind

func (*GetStorageRangesPacket) Kind() byte

func (*GetStorageRangesPacket) Name

type GetTrieNodesPacket

type GetTrieNodesPacket struct {
	ID    uint64            // Request ID to match up responses with
	Root  common.Hash       // Root hash of the account trie to serve
	Paths []TrieNodePathSet // Trie node hashes to retrieve the nodes for
	Bytes uint64            // Soft limit at which to stop returning data
}

GetTrieNodesPacket represents a state trie node query.

func (*GetTrieNodesPacket) Kind

func (*GetTrieNodesPacket) Kind() byte

func (*GetTrieNodesPacket) Name

func (*GetTrieNodesPacket) Name() string

type Handler

type Handler func(peer *Peer) error

Handler is a callback to invoke from an outside runner after the boilerplate exchanges have passed.

type NodeInfo

type NodeInfo struct{}

NodeInfo represents a short summary of the `snap` sub-protocol metadata known about the host peer.

type NodeList

type NodeList []rlp.RawValue

NodeList stores an ordered list of trie nodes. It implements ethdb.KeyValueWriter.

func (NodeList) DataSize

func (n NodeList) DataSize() int

DataSize returns the aggregated data size of nodes in the list

func (*NodeList) Delete

func (n *NodeList) Delete(key []byte) error

Delete panics as there's no reason to remove a node from the list.

func (NodeList) NodeSet

func (n NodeList) NodeSet() *NodeSet

NodeSet converts the node list to a NodeSet

func (*NodeList) Put

func (n *NodeList) Put(key []byte, value []byte) error

Put stores a new node at the end of the list

func (NodeList) Store

func (n NodeList) Store(db database.KeyValueWriter)

Store writes the contents of the list to the given database

type NodeSet

type NodeSet struct {
	// contains filtered or unexported fields
}

NodeSet stores a set of trie nodes. It implements trie.Database and can also act as a cache for another trie.Database.

func NewNodeSet

func NewNodeSet() *NodeSet

NewNodeSet creates an empty node set

func (*NodeSet) DataSize

func (db *NodeSet) DataSize() int

DataSize returns the aggregated data size of nodes in the set

func (*NodeSet) Delete

func (db *NodeSet) Delete(key []byte) error

Delete removes a node from the set

func (*NodeSet) Get

func (db *NodeSet) Get(key []byte) ([]byte, error)

Get returns a stored node

func (*NodeSet) Has

func (db *NodeSet) Has(key []byte) (bool, error)

Has returns true if the node set contains the given key

func (*NodeSet) KeyCount

func (db *NodeSet) KeyCount() int

KeyCount returns the number of nodes in the set

func (*NodeSet) NodeList

func (db *NodeSet) NodeList() NodeList

NodeList converts the node set to a NodeList

func (*NodeSet) Put

func (db *NodeSet) Put(key []byte, value []byte) error

Put stores a new node in the set

func (*NodeSet) ReadCachedTrieNode

func (db *NodeSet) ReadCachedTrieNode(hash common.Hash) ([]byte, error)

func (*NodeSet) Store

func (db *NodeSet) Store(target database.KeyValueWriter)

Store writes the contents of the set to the given database

func (*NodeSet) WriteMerkleProof

func (db *NodeSet) WriteMerkleProof(key, value []byte)

type Packet

type Packet interface {
	Name() string // Name returns a string corresponding to the message type.
	Kind() byte   // Kind returns the message type.
}

Packet represents a p2p message in the `snap` protocol.

type Peer

type Peer struct {
	*p2p.Peer // The embedded P2P package peer
	// contains filtered or unexported fields
}

Peer is a collection of relevant information we have about a `snap` peer.

func NewFakePeer

func NewFakePeer(version uint, id string, rw p2p.MsgReadWriter) *Peer

NewFakePeer create a fake snap peer without a backing p2p peer, for testing purposes.

func NewPeer

func NewPeer(version uint, p *p2p.Peer, rw p2p.MsgReadWriter) *Peer

NewPeer create a wrapper for a network connection and negotiated protocol version.

func (*Peer) ID

func (p *Peer) ID() string

ID retrieves the peer's unique identifier.

func (*Peer) Log

func (p *Peer) Log() log.Logger

Log overrides the P2P logget with the higher level one containing only the id.

func (*Peer) RequestAccountRange

func (p *Peer) RequestAccountRange(id uint64, root common.Hash, origin, limit common.Hash, bytes uint64) error

RequestAccountRange fetches a batch of accounts rooted in a specific account trie, starting with the origin.

func (*Peer) RequestByteCodes

func (p *Peer) RequestByteCodes(id uint64, hashes []common.Hash, bytes uint64) error

RequestByteCodes fetches a batch of bytecodes by hash.

func (*Peer) RequestStorageRanges

func (p *Peer) RequestStorageRanges(id uint64, root common.Hash, accounts []common.Hash, origin, limit []byte, bytes uint64) error

RequestStorageRanges fetches a batch of storage slots belonging to one or more accounts. If slots from only one accout is requested, an origin marker may also be used to retrieve from there.

func (*Peer) RequestTrieNodes

func (p *Peer) RequestTrieNodes(id uint64, root common.Hash, paths []TrieNodePathSet, bytes uint64) error

RequestTrieNodes fetches a batch of account or storage trie nodes rooted in a specificstate trie.

func (*Peer) Version

func (p *Peer) Version() uint

Version retrieves the peer's negoatiated `snap` protocol version.

type SnapshotDownloader

type SnapshotDownloader interface {
	// DeliverSnapPacket is invoked from a peer's message handler when it transmits a
	// data packet for the local node to consume.
	DeliverSnapPacket(peer *Peer, packet Packet) error
}

type SnapshotReader

type SnapshotReader interface {
	StateCache() state.Database
	Snapshots() *snapshot.Tree
	ContractCode(hash common.Hash) ([]byte, error)
	ContractCodeWithPrefix(hash common.Hash) ([]byte, error)
}

type StorageData

type StorageData struct {
	Hash common.Hash // Hash of the storage slot
	Body []byte      // Data content of the slot
}

StorageData represents a single storage slot in a query response.

type StorageRangesPacket

type StorageRangesPacket struct {
	ID    uint64           // ID of the request this is a response for
	Slots [][]*StorageData // Lists of consecutive storage slots for the requested accounts
	Proof [][]byte         // Merkle proofs for the *last* slot range, if it's incomplete
}

StorageRangesPacket represents a storage slot query response.

func (*StorageRangesPacket) Kind

func (*StorageRangesPacket) Kind() byte

func (*StorageRangesPacket) Name

func (*StorageRangesPacket) Name() string

func (*StorageRangesPacket) Unpack

func (p *StorageRangesPacket) Unpack() ([][]common.Hash, [][][]byte)

Unpack retrieves the storage slots from the range packet and returns them in a split flat format that's more consistent with the internal data structures.

type SyncPeer

type SyncPeer interface {
	// ID retrieves the peer's unique identifier.
	ID() string

	// RequestAccountRange fetches a batch of accounts rooted in a specific account
	// trie, starting with the origin.
	RequestAccountRange(id uint64, root, origin, limit common.Hash, bytes uint64) error

	// RequestStorageRanges fetches a batch of storage slots belonging to one or
	// more accounts. If slots from only one accout is requested, an origin marker
	// may also be used to retrieve from there.
	RequestStorageRanges(id uint64, root common.Hash, accounts []common.Hash, origin, limit []byte, bytes uint64) error

	// RequestByteCodes fetches a batch of bytecodes by hash.
	RequestByteCodes(id uint64, hashes []common.Hash, bytes uint64) error

	// RequestTrieNodes fetches a batch of account or storage trie nodes rooted in
	// a specificstate trie.
	RequestTrieNodes(id uint64, root common.Hash, paths []TrieNodePathSet, bytes uint64) error

	// Log retrieves the peer's own contextual logger.
	Log() log.Logger
}

SyncPeer abstracts out the methods required for a peer to be synced against with the goal of allowing the construction of mock peers without the full blown networking.

type SyncPending

type SyncPending struct {
	TrienodeHeal uint64 // Number of state trie nodes pending
	BytecodeHeal uint64 // Number of bytecodes pending
}

SyncPending is analogous to SyncProgress, but it's used to report on pending ephemeral sync progress that doesn't get persisted into the database.

type SyncProgress

type SyncProgress struct {
	Tasks []*accountTask // The suspended account tasks (contract tasks within)

	// Status report during syncing phase
	AccountSynced  uint64             // Number of accounts downloaded
	AccountBytes   common.StorageSize // Number of account trie bytes persisted to disk
	BytecodeSynced uint64             // Number of bytecodes downloaded
	BytecodeBytes  common.StorageSize // Number of bytecode bytes downloaded
	StorageSynced  uint64             // Number of storage slots downloaded
	StorageBytes   common.StorageSize // Number of storage trie bytes persisted to disk

	// Status report during healing phase
	TrienodeHealSynced uint64             // Number of state trie nodes downloaded
	TrienodeHealBytes  common.StorageSize // Number of state trie bytes persisted to disk
	BytecodeHealSynced uint64             // Number of bytecodes downloaded
	BytecodeHealBytes  common.StorageSize // Number of bytecodes persisted to disk
}

SyncProgress is a database entry to allow suspending and resuming a snapshot state sync. Opposed to full and fast sync, there is no way to restart a suspended snap sync without prior knowledge of the suspension point.

type Syncer

type Syncer struct {
	// contains filtered or unexported fields
}

Syncer is an Klaytn account and storage trie syncer based on snapshots and the snap protocol. It's purpose is to download all the accounts and storage slots from remote peers and reassemble chunks of the state trie, on top of which a state sync can be run to fix any gaps / overlaps.

Every network request has a variety of failure events:

  • The peer disconnects after task assignment, failing to send the request
  • The peer disconnects after sending the request, before delivering on it
  • The peer remains connected, but does not deliver a response in time
  • The peer delivers a stale response after a previous timeout
  • The peer delivers a refusal to serve the requested state

func NewSyncer

func NewSyncer(db database.DBManager) *Syncer

NewSyncer creates a new snapshot syncer to download the Ethereum state over the snap protocol.

func (*Syncer) OnAccounts

func (s *Syncer) OnAccounts(peer SyncPeer, id uint64, hashes []common.Hash, accounts [][]byte, proof [][]byte) error

OnAccounts is a callback method to invoke when a range of accounts are received from a remote peer.

func (*Syncer) OnByteCodes

func (s *Syncer) OnByteCodes(peer SyncPeer, id uint64, bytecodes [][]byte) error

OnByteCodes is a callback method to invoke when a batch of contract bytes codes are received from a remote peer.

func (*Syncer) OnStorage

func (s *Syncer) OnStorage(peer SyncPeer, id uint64, hashes [][]common.Hash, slots [][][]byte, proof [][]byte) error

OnStorage is a callback method to invoke when ranges of storage slots are received from a remote peer.

func (*Syncer) OnTrieNodes

func (s *Syncer) OnTrieNodes(peer SyncPeer, id uint64, trienodes [][]byte) error

OnTrieNodes is a callback method to invoke when a batch of trie nodes are received from a remote peer.

func (*Syncer) Progress

func (s *Syncer) Progress() (*SyncProgress, *SyncPending)

Progress returns the snap sync status statistics.

func (*Syncer) Register

func (s *Syncer) Register(peer SyncPeer) error

Register injects a new data source into the syncer's peerset.

func (*Syncer) Sync

func (s *Syncer) Sync(root common.Hash, cancel chan struct{}) error

Sync starts (or resumes a previous) sync cycle to iterate over an state trie with the given root and reconstruct the nodes based on the snapshot leaves. Previously downloaded segments will not be redownloaded of fixed, rather any errors will be healed after the leaves are fully accumulated.

func (*Syncer) Unregister

func (s *Syncer) Unregister(id string) error

Unregister injects a new data source into the syncer's peerset.

type TrieNodePathSet

type TrieNodePathSet [][]byte

TrieNodePathSet is a list of trie node paths to retrieve. A naive way to represent trie nodes would be a simple list of `account || storage` path segments concatenated, but that would be very wasteful on the network.

Instead, this array special cases the first element as the path in the account trie and the remaining elements as paths in the storage trie. To address an account node, the slice should have a length of 1 consisting of only the account path. There's no need to be able to address both an account node and a storage node in the same request as it cannot happen that a slot is accessed before the account path is fully expanded.

type TrieNodesPacket

type TrieNodesPacket struct {
	ID    uint64   // ID of the request this is a response for
	Nodes [][]byte // Requested state trie nodes
}

TrieNodesPacket represents a state trie node query response.

func (*TrieNodesPacket) Kind

func (*TrieNodesPacket) Kind() byte

func (*TrieNodesPacket) Name

func (*TrieNodesPacket) Name() string

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL