mempool

README

mempool

Package mempool provides a policy-enforced pool of unmined Decred transactions.

A key responsibility of the Decred network is mining transactions – regular transactions and stake transactions – into blocks. In order to facilitate this, the mining process relies on having a readily-available source of transactions to include in a block that is being solved.

At a high level, this package satisfies that requirement by providing an in-memory pool of fully validated transactions that can also optionally be further filtered based upon a configurable policy.

The Policy configuration options has flags that control whether or not "standard" transactions and old votes are accepted into the mempool. In essence, a "standard" transaction is one that satisfies a fairly strict set of requirements that are largely intended to help provide fair use of the system to all users. It is important to note that what is considered to be a "standard" transaction changes over time as policy and consensus rules evolve. For some insight, at the time of this writing, an example of some of the criteria that are required for a transaction to be considered standard are that it is of the most-recently supported version, finalized, does not exceed a specific size, and only consists of specific script forms.

Since this package does not deal with other Decred specifics such as network communication and transaction relay, it returns a list of transactions that were accepted which gives the caller a high level of flexibility in how they want to proceed. Typically, this will involve things such as relaying the transactions to other peers on the network and notifying the mining process that new transactions are available.

This package has intentionally been designed so it can be used as a standalone package for any projects needing the ability create an in-memory pool of Decred transactions that are not only valid by consensus rules, but also adhere to a configurable policy

Feature Overview

The following is a quick overview of the major features. It is not intended to be an exhaustive list.

• Maintain a pool of fully validated transactions
  • Reject non-fully-spent duplicate transactions
  • Reject coinbase transactions
  • Reject double spends (both from the chain and other transactions in pool)
  • Reject invalid transactions according to the network consensus rules
  • Full script execution and validation with signature cache support
  • Individual transaction query support
• Stake transaction support (ticket purchases, votes and revocations)
  • Option to accept or reject old votes
• Orphan transaction support (transactions that spend from unknown outputs)
  • Configurable limits (see transaction acceptance policy)
  • Automatic addition of orphan transactions that are no longer orphans as new transactions are added to the pool
  • Individual orphan transaction query support
• Configurable transaction acceptance policy
  • Option to accept or reject standard transactions
  • Option to accept or reject transactions based on priority calculations
  • Rate limiting of low-fee and free transactions
  • Non-zero fee threshold
  • Max signature operations per transaction
  • Max orphan transaction size
  • Max number of orphan transactions allowed
• Additional metadata tracking for each transaction
  • Timestamp when the transaction was added to the pool
  • Most recent block height when the transaction was added to the pool
  • The fee the transaction pays
  • The starting priority for the transaction
• Manual control of transaction removal
  • Recursive removal of all dependent transactions

Installation and Updating

$ go get -u github.com/decred/dcrd/mempool

License

Package mempool is licensed under the copyfree ISC License.

Documentation

Overview

Package mempool provides a policy-enforced pool of unmined Decred transactions.

A key responsibility of the Decred network is mining transactions – regular transactions and stake transactions – into blocks. In order to facilitate this, the mining process relies on having a readily-available source of transactions to include in a block that is being solved.

At a high level, this package satisfies that requirement by providing an in-memory pool of fully validated transactions that can also optionally be further filtered based upon a configurable policy.

The Policy configuration options has flags that control whether or not "standard" transactions and old votes are accepted into the mempool. In essence, a "standard" transaction is one that satisfies a fairly strict set of requirements that are largely intended to help provide fair use of the system to all users. It is important to note that what is considered to be a "standard" transaction changes over time as policy and consensus rules evolve. For some insight, at the time of this writing, an example of _some_ of the criteria that are required for a transaction to be considered standard are that it is of the most-recently supported version, finalized, does not exceed a specific size, and only consists of specific script forms.

Since this package does not deal with other Decred specifics such as network communication and transaction relay, it returns a list of transactions that were accepted which gives the caller a high level of flexibility in how they want to proceed. Typically, this will involve things such as relaying the transactions to other peers on the network and notifying the mining process that new transactions are available.

This package has intentionally been designed so it can be used as a standalone package for any projects needing the ability create an in-memory pool of Decred transactions that are not only valid by consensus rules, but also adhere to a configurable policy

## Feature Overview

The following is a quick overview of the major features. It is not intended to be an exhaustive list.

- Maintain a pool of fully validated transactions

• Reject non-fully-spent duplicate transactions
• Reject coinbase transactions
• Reject double spends (both from the chain and other transactions in pool)
• Reject invalid transactions according to the network consensus rules
• Full script execution and validation with signature cache support
• Individual transaction query support

- Stake transaction support (ticket purchases, votes and revocations)

• Option to accept or reject old votes

- Orphan transaction support (transactions that spend from unknown outputs)

• Configurable limits (see transaction acceptance policy)
• Automatic addition of orphan transactions that are no longer orphans as new transactions are added to the pool
• Individual orphan transaction query support

- Configurable transaction acceptance policy

• Option to accept or reject standard transactions
• Option to accept or reject transactions based on priority calculations
• Rate limiting of low-fee and free transactions
• Non-zero fee threshold
• Max signature operations per transaction
• Max orphan transaction size
• Max number of orphan transactions allowed

- Additional metadata tracking for each transaction

• Timestamp when the transaction was added to the pool
• Most recent block height when the transaction was added to the pool
• The fee the transaction pays
• The starting priority for the transaction

- Manual control of transaction removal

• Recursive removal of all dependent transactions

Errors

Errors returned by this package are either the raw errors provided by underlying calls or of type mempool.RuleError. Since there are two classes of rules (mempool acceptance rules and blockchain (consensus) acceptance rules), the mempool.RuleError type contains a single Err field which will, in turn, either be a mempool.TxRuleError or a blockchain.RuleError. The first indicates a violation of mempool acceptance rules while the latter indicates a violation of consensus acceptance rules. This allows the caller to easily differentiate between unexpected errors, such as database errors, versus errors due to rule violations through type assertions. In addition, callers can programmatically determine the specific rule violation by type asserting the Err field to one of the aforementioned types and examining their underlying ErrorCode field.

Index

• Constants
• func DisableLog()
• func ErrToRejectErr(err error) (wire.RejectCode, string)
• func UseLogger(logger slog.Logger)
• type Config
• type Policy
• type RuleError
  • func (e RuleError) Error() string
• type TxDesc
• type TxPool
  • func New(cfg *Config) *TxPool
  • func (mp *TxPool) Count() int
  • func (mp *TxPool) FetchTransaction(txHash *chainhash.Hash) (*dcrutil.Tx, error)
  • func (mp *TxPool) HaveAllTransactions(hashes []chainhash.Hash) bool
  • func (mp *TxPool) HaveTransaction(hash *chainhash.Hash) bool
  • func (mp *TxPool) HaveTransactions(hashes []*chainhash.Hash) []bool
  • func (mp *TxPool) IsOrphanInPool(hash *chainhash.Hash) bool
  • func (mp *TxPool) IsRegTxTreeKnownDisapproved(hash *chainhash.Hash) bool
  • func (mp *TxPool) IsTransactionInPool(hash *chainhash.Hash) bool
  • func (mp *TxPool) LastUpdated() time.Time
  • func (mp *TxPool) MaybeAcceptTransaction(tx *dcrutil.Tx, isNew, rateLimit bool) ([]*chainhash.Hash, error)
  • func (mp *TxPool) MiningDescs() []*mining.TxDesc
  • func (mp *TxPool) ProcessOrphans(acceptedTx *dcrutil.Tx) []*dcrutil.Tx
  • func (mp *TxPool) ProcessTransaction(tx *dcrutil.Tx, allowOrphan, rateLimit, allowHighFees bool) ([]*dcrutil.Tx, error)
  • func (mp *TxPool) PruneExpiredTx()
  • func (mp *TxPool) PruneStakeTx(requiredStakeDifficulty, height int64)
  • func (mp *TxPool) RawMempoolVerbose(filterType *stake.TxType) map[string]*dcrjson.GetRawMempoolVerboseResult
  • func (mp *TxPool) RemoveDoubleSpends(tx *dcrutil.Tx)
  • func (mp *TxPool) RemoveOrphan(tx *dcrutil.Tx)
  • func (mp *TxPool) RemoveTransaction(tx *dcrutil.Tx, removeRedeemers bool)
  • func (mp *TxPool) TxDescs() []*TxDesc
  • func (mp *TxPool) TxHashes() []*chainhash.Hash
  • func (mp *TxPool) VoteHashesForBlock(blockHash *chainhash.Hash) []chainhash.Hash
  • func (mp *TxPool) VotesForBlocks(hashes []chainhash.Hash) [][]mining.VoteDesc
• type TxRuleError
  • func (e TxRuleError) Error() string

Constants

const (
	// DefaultBlockPrioritySize is the default size in bytes for high-
	// priority / low-fee transactions.  It is used to help determine which
	// are allowed into the mempool and consequently affects their relay and
	// inclusion when generating block templates.
	DefaultBlockPrioritySize = 20000

	// MinHighPriority is the minimum priority value that allows a
	// transaction to be considered high priority.
	//
	// Deprecated: Use mining.MinHighPriority
	// TODO: Remove in next version update.
	MinHighPriority = mining.MinHighPriority
)

const (

	// DefaultMinRelayTxFee is the minimum fee in atoms that is required for
	// a transaction to be treated as free for relay and mining purposes.
	// It is also used to help determine if a transaction is considered dust
	// and as a base for calculating minimum required fees for larger
	// transactions.  This value is in Atoms/1000 bytes.
	DefaultMinRelayTxFee = dcrutil.Amount(1e4)

	// BaseStandardVerifyFlags defines the script flags that should be used
	// when executing transaction scripts to enforce additional checks which
	// are required for the script to be considered standard regardless of
	// the state of any agenda votes.  The full set of standard verification
	// flags must include these flags as well as any additional flags that
	// are conditionally enabled depending on the result of agenda votes.
	BaseStandardVerifyFlags = txscript.ScriptDiscourageUpgradableNops |
		txscript.ScriptVerifyCleanStack |
		txscript.ScriptVerifyCheckLockTimeVerify |
		txscript.ScriptVerifyCheckSequenceVerify
)

Functions

func DisableLog

func DisableLog()

DisableLog disables all library log output. Logging output is disabled by default until either UseLogger or SetLogWriter are called.

func ErrToRejectErr

func ErrToRejectErr(err error) (wire.RejectCode, string)

ErrToRejectErr examines the underlying type of the error and returns a reject code and string appropriate to be sent in a wire.MsgReject message.

func UseLogger

func UseLogger(logger slog.Logger)

UseLogger uses a specified Logger to output package logging info. This should be used in preference to SetLogWriter if the caller is also using slog.

Types

type Config

type Config struct {
	// Policy defines the various mempool configuration options related
	// to policy.
	Policy Policy

	// ChainParams identifies which chain parameters the txpool is
	// associated with.
	ChainParams *chaincfg.Params

	// NextStakeDifficulty defines the function to retrieve the stake
	// difficulty for the block after the current best block.
	//
	// This function must be safe for concurrent access.
	NextStakeDifficulty func() (int64, error)

	// FetchUtxoView defines the function to use to fetch unspent
	// transaction output information.
	FetchUtxoView func(*dcrutil.Tx, bool) (*blockchain.UtxoViewpoint, error)

	// BlockByHash defines the function use to fetch the block identified
	// by the given hash.
	BlockByHash func(*chainhash.Hash) (*dcrutil.Block, error)

	// BestHash defines the function to use to access the block hash of
	// the current best chain.
	BestHash func() *chainhash.Hash

	// BestHeight defines the function to use to access the block height of
	// the current best chain.
	BestHeight func() int64

	// PastMedianTime defines the function to use in order to access the
	// median time calculated from the point-of-view of the current chain
	// tip within the best chain.
	PastMedianTime func() time.Time

	// CalcSequenceLock defines the function to use in order to generate
	// the current sequence lock for the given transaction using the passed
	// utxo view.
	CalcSequenceLock func(*dcrutil.Tx, *blockchain.UtxoViewpoint) (*blockchain.SequenceLock, error)

	// SubsidyCache defines a subsidy cache to use.
	SubsidyCache *blockchain.SubsidyCache

	// SigCache defines a signature cache to use.
	SigCache *txscript.SigCache

	// AddrIndex defines the optional address index instance to use for
	// indexing the unconfirmed transactions in the memory pool.
	// This can be nil if the address index is not enabled.
	AddrIndex *indexers.AddrIndex

	// ExistsAddrIndex defines the optional exists address index instance
	// to use for indexing the unconfirmed transactions in the memory pool.
	// This can be nil if the address index is not enabled.
	ExistsAddrIndex *indexers.ExistsAddrIndex

	// AddTxToFeeEstimation defines an optional function to be called whenever a
	// new transaction is added to the mempool, which can be used to track fees
	// for the purposes of smart fee estimation.
	AddTxToFeeEstimation func(txHash *chainhash.Hash, fee, size int64, txType stake.TxType)

	// RemoveTxFromFeeEstimation defines an optional function to be called
	// whenever a transaction is removed from the mempool in order to track fee
	// estimation.
	RemoveTxFromFeeEstimation func(txHash *chainhash.Hash)
}

Config is a descriptor containing the memory pool configuration.

type Policy

type Policy struct {
	// MaxTxVersion is the max transaction version that the mempool should
	// accept.  All transactions above this version are rejected as
	// non-standard.
	MaxTxVersion uint16

	// DisableRelayPriority defines whether to relay free or low-fee
	// transactions that do not have enough priority to be relayed.
	DisableRelayPriority bool

	// AcceptNonStd defines whether to accept and relay non-standard
	// transactions to the network. If true, non-standard transactions
	// will be accepted into the mempool and relayed to the rest of the
	// network. Otherwise, all non-standard transactions will be rejected.
	AcceptNonStd bool

	// FreeTxRelayLimit defines the given amount in thousands of bytes
	// per minute that transactions with no fee are rate limited to.
	FreeTxRelayLimit float64

	// MaxOrphanTxs is the maximum number of orphan transactions
	// that can be queued.
	MaxOrphanTxs int

	// MaxOrphanTxSize is the maximum size allowed for orphan transactions.
	// This helps prevent memory exhaustion attacks from sending a lot of
	// of big orphans.
	MaxOrphanTxSize int

	// MaxSigOpsPerTx is the maximum number of signature operations
	// in a single transaction we will relay or mine.  It is a fraction
	// of the max signature operations for a block.
	MaxSigOpsPerTx int

	// MinRelayTxFee defines the minimum transaction fee in PFC/kB to be
	// considered a non-zero fee.
	MinRelayTxFee dcrutil.Amount

	// AllowOldVotes defines whether or not votes on old blocks will be
	// admitted and relayed.
	AllowOldVotes bool

	// StandardVerifyFlags defines the function to retrieve the flags to
	// use for verifying scripts for the block after the current best block.
	// It must set the verification flags properly depending on the result
	// of any agendas that affect them.
	//
	// This function must be safe for concurrent access.
	StandardVerifyFlags func() (txscript.ScriptFlags, error)

	// AcceptSequenceLocks defines the function to determine whether or not
	// to accept transactions with sequence locks.  Typically this will be
	// set depending on the result of the fix sequence locks agenda vote.
	//
	// This function must be safe for concurrent access.
	AcceptSequenceLocks func() (bool, error)
}

Policy houses the policy (configuration parameters) which is used to control the mempool.

type RuleError

type RuleError struct {
	Err error
}

RuleError identifies a rule violation. It is used to indicate that processing of a transaction failed due to one of the many validation rules. The caller can use type assertions to determine if a failure was specifically due to a rule violation and use the Err field to access the underlying error, which will be either a TxRuleError or a blockchain.RuleError.

func (RuleError) Error

func (e RuleError) Error() string

Error satisfies the error interface and prints human-readable errors.

type TxDesc

type TxDesc struct {
	mining.TxDesc

	// StartingPriority is the priority of the transaction when it was added
	// to the pool.
	StartingPriority float64
}

TxDesc is a descriptor containing a transaction in the mempool along with additional metadata.

type TxPool

type TxPool struct {
	// contains filtered or unexported fields
}

TxPool is used as a source of transactions that need to be mined into blocks and relayed to other peers. It is safe for concurrent access from multiple peers.

func New

func New(cfg *Config) *TxPool

New returns a new memory pool for validating and storing standalone transactions until they are mined into a block.

func (*TxPool) Count

func (mp *TxPool) Count() int

Count returns the number of transactions in the main pool. It does not include the orphan pool.

This function is safe for concurrent access.

func (*TxPool) FetchTransaction

func (mp *TxPool) FetchTransaction(txHash *chainhash.Hash) (*dcrutil.Tx, error)

FetchTransaction returns the requested transaction from the transaction pool. This only fetches from the main transaction pool and does not include orphans.

This function is safe for concurrent access.

func (*TxPool) HaveAllTransactions

func (mp *TxPool) HaveAllTransactions(hashes []chainhash.Hash) bool

HaveAllTransactions returns whether or not all of the passed transaction hashes exist in the mempool.

This function is safe for concurrent access.

func (*TxPool) HaveTransaction

func (mp *TxPool) HaveTransaction(hash *chainhash.Hash) bool

HaveTransaction returns whether or not the passed transaction already exists in the main pool or in the orphan pool.

This function is safe for concurrent access.

func (*TxPool) HaveTransactions

func (mp *TxPool) HaveTransactions(hashes []*chainhash.Hash) []bool

HaveTransactions returns whether or not the passed transactions already exist in the main pool or in the orphan pool.

This function is safe for concurrent access.

func (*TxPool) IsOrphanInPool

func (mp *TxPool) IsOrphanInPool(hash *chainhash.Hash) bool

IsOrphanInPool returns whether or not the passed transaction already exists in the orphan pool.

This function is safe for concurrent access.

func (*TxPool) IsRegTxTreeKnownDisapproved

func (mp *TxPool) IsRegTxTreeKnownDisapproved(hash *chainhash.Hash) bool

IsRegTxTreeKnownDisapproved returns whether or not the regular tree of the block represented by the provided hash is known to be disapproved according to the votes currently in the memory pool.

The function is safe for concurrent access.

func (*TxPool) IsTransactionInPool

func (mp *TxPool) IsTransactionInPool(hash *chainhash.Hash) bool

IsTransactionInPool returns whether or not the passed transaction already exists in the main pool.

This function is safe for concurrent access.

func (*TxPool) LastUpdated

func (mp *TxPool) LastUpdated() time.Time

LastUpdated returns the last time a transaction was added to or removed from the main pool. It does not include the orphan pool.

This function is safe for concurrent access.

func (*TxPool) MaybeAcceptTransaction

func (mp *TxPool) MaybeAcceptTransaction(tx *dcrutil.Tx, isNew, rateLimit bool) ([]*chainhash.Hash, error)

MaybeAcceptTransaction is the main workhorse for handling insertion of new free-standing transactions into a memory pool. It includes functionality such as rejecting duplicate transactions, ensuring transactions follow all rules, orphan transaction handling, and insertion into the memory pool. The isOrphan parameter can be nil if the caller does not need to know whether or not the transaction is an orphan.

This function is safe for concurrent access.

func (*TxPool) MiningDescs

func (mp *TxPool) MiningDescs() []*mining.TxDesc

MiningDescs returns a slice of mining descriptors for all the transactions in the pool.

This is part of the mining.TxSource interface implementation and is safe for concurrent access as required by the interface contract.

func (*TxPool) ProcessOrphans

func (mp *TxPool) ProcessOrphans(acceptedTx *dcrutil.Tx) []*dcrutil.Tx

ProcessOrphans determines if there are any orphans which depend on the passed transaction hash (it is possible that they are no longer orphans) and potentially accepts them to the memory pool. It repeats the process for the newly accepted transactions (to detect further orphans which may no longer be orphans) until there are no more.

It returns a slice of transactions added to the mempool. A nil slice means no transactions were moved from the orphan pool to the mempool.

This function is safe for concurrent access.

func (*TxPool) ProcessTransaction

func (mp *TxPool) ProcessTransaction(tx *dcrutil.Tx, allowOrphan, rateLimit, allowHighFees bool) ([]*dcrutil.Tx, error)

ProcessTransaction is the main workhorse for handling insertion of new free-standing transactions into the memory pool. It includes functionality such as rejecting duplicate transactions, ensuring transactions follow all rules, orphan transaction handling, and insertion into the memory pool.

It returns a slice of transactions added to the mempool. When the error is nil, the list will include the passed transaction itself along with any additional orphan transaactions that were added as a result of the passed one being accepted.

This function is safe for concurrent access.

func (*TxPool) PruneExpiredTx

func (mp *TxPool) PruneExpiredTx()

PruneExpiredTx prunes expired transactions from the mempool that may no longer be able to be included into a block.

This function is safe for concurrent access.

func (*TxPool) PruneStakeTx

func (mp *TxPool) PruneStakeTx(requiredStakeDifficulty, height int64)

PruneStakeTx is the function which is called every time a new block is processed. The idea is any outstanding SStx that hasn't been mined in a certain period of time (CoinbaseMaturity) and the submitted SStx's stake difficulty is below the current required stake difficulty should be pruned from mempool since they will never be mined. The same idea stands for SSGen and SSRtx

func (*TxPool) RawMempoolVerbose

func (mp *TxPool) RawMempoolVerbose(filterType *stake.TxType) map[string]*dcrjson.GetRawMempoolVerboseResult

RawMempoolVerbose returns all of the entries in the mempool filtered by the provided stake type as a fully populated JSON result. The filter type can be nil in which case all transactions will be returned.

This function is safe for concurrent access.

func (*TxPool) RemoveDoubleSpends

func (mp *TxPool) RemoveDoubleSpends(tx *dcrutil.Tx)

RemoveDoubleSpends removes all transactions which spend outputs spent by the passed transaction from the memory pool. Removing those transactions then leads to removing all transactions which rely on them, recursively. This is necessary when a block is connected to the main chain because the block may contain transactions which were previously unknown to the memory pool.

This function is safe for concurrent access.

func (*TxPool) RemoveOrphan

func (mp *TxPool) RemoveOrphan(tx *dcrutil.Tx)

RemoveOrphan removes the passed orphan transaction from the orphan pool and previous orphan index.

This function is safe for concurrent access.

func (*TxPool) RemoveTransaction

func (mp *TxPool) RemoveTransaction(tx *dcrutil.Tx, removeRedeemers bool)

RemoveTransaction removes the passed transaction from the mempool. When the removeRedeemers flag is set, any transactions that redeem outputs from the removed transaction will also be removed recursively from the mempool, as they would otherwise become orphans.

This function is safe for concurrent access.

func (*TxPool) TxDescs

func (mp *TxPool) TxDescs() []*TxDesc

TxDescs returns a slice of descriptors for all the transactions in the pool. The descriptors are to be treated as read only.

This function is safe for concurrent access.

func (*TxPool) TxHashes

func (mp *TxPool) TxHashes() []*chainhash.Hash

TxHashes returns a slice of hashes for all of the transactions in the memory pool.

This function is safe for concurrent access.

func (*TxPool) VoteHashesForBlock

func (mp *TxPool) VoteHashesForBlock(blockHash *chainhash.Hash) []chainhash.Hash

VoteHashesForBlock returns the hashes for all votes on the provided block hash that are currently available in the mempool.

This function is safe for concurrent access.

func (*TxPool) VotesForBlocks

func (mp *TxPool) VotesForBlocks(hashes []chainhash.Hash) [][]mining.VoteDesc

VotesForBlocks returns the vote metadata for all votes on the provided block hashes that are currently available in the mempool.

This function is safe for concurrent access.

type TxRuleError

type TxRuleError struct {
	RejectCode  wire.RejectCode // The code to send with reject messages
	Description string          // Human readable description of the issue
}

TxRuleError identifies a rule violation. It is used to indicate that processing of a transaction failed due to one of the many validation rules. The caller can use type assertions to determine if a failure was specifically due to a rule violation and access the ErrorCode field to ascertain the specific reason for the rule violation.

func (TxRuleError) Error

func (e TxRuleError) Error() string

Error satisfies the error interface and prints human-readable errors.