README

mempool

Build Status ISC License GoDoc

Package mempool provides a policy-enforced pool of unmined bitcoin transactions.

A key responsbility of the bitcoin network is mining user-generated transactions into blocks. In order to facilitate this, the mining process relies on having a readily-available source of transactions to include in a block that is being solved.

At a high level, this package satisfies that requirement by providing an in-memory pool of fully validated transactions that can also optionally be further filtered based upon a configurable policy.

One of the policy configuration options controls whether or not "standard" transactions are accepted. In essence, a "standard" transaction is one that satisfies a fairly strict set of requirements that are largely intended to help provide fair use of the system to all users. It is important to note that what is considered a "standard" transaction changes over time. For some insight, at the time of this writing, an example of some of the criteria that are required for a transaction to be considered standard are that it is of the most-recently supported version, finalized, does not exceed a specific size, and only consists of specific script forms.

Since this package does not deal with other bitcoin specifics such as network communication and transaction relay, it returns a list of transactions that were accepted which gives the caller a high level of flexibility in how they want to proceed. Typically, this will involve things such as relaying the transactions to other peers on the network and notifying the mining process that new transactions are available.

This package has intentionally been designed so it can be used as a standalone package for any projects needing the ability create an in-memory pool of bitcoin transactions that are not only valid by consensus rules, but also adhere to a configurable policy.

Feature Overview

The following is a quick overview of the major features. It is not intended to be an exhaustive list.

  • Maintain a pool of fully validated transactions
    • Reject non-fully-spent duplicate transactions
    • Reject coinbase transactions
    • Reject double spends (both from the chain and other transactions in pool)
    • Reject invalid transactions according to the network consensus rules
    • Full script execution and validation with signature cache support
    • Individual transaction query support
  • Orphan transaction support (transactions that spend from unknown outputs)
    • Configurable limits (see transaction acceptance policy)
    • Automatic addition of orphan transactions that are no longer orphans as new transactions are added to the pool
    • Individual orphan transaction query support
  • Configurable transaction acceptance policy
    • Option to accept or reject standard transactions
    • Option to accept or reject transactions based on priority calculations
    • Rate limiting of low-fee and free transactions
    • Non-zero fee threshold
    • Max signature operations per transaction
    • Max orphan transaction size
    • Max number of orphan transactions allowed
  • Additional metadata tracking for each transaction
    • Timestamp when the transaction was added to the pool
    • Most recent block height when the transaction was added to the pool
    • The fee the transaction pays
    • The starting priority for the transaction
  • Manual control of transaction removal
    • Recursive removal of all dependent transactions

Installation and Updating

$ go get -u github.com/gcash/bchd/mempool

License

Package mempool is licensed under the copyfree ISC License.

Expand ▾ Collapse ▴

Documentation

Overview

Package mempool provides a policy-enforced pool of unmined bitcoin cash transactions.

A key responsibility of the bitcoin cash network is mining user-generated transactions into blocks. In order to facilitate this, the mining process relies on having a readily-available source of transactions to include in a block that is being solved.

At a high level, this package satisfies that requirement by providing an in-memory pool of fully validated transactions that can also optionally be further filtered based upon a configurable policy.

One of the policy configuration options controls whether or not "standard" transactions are accepted. In essence, a "standard" transaction is one that satisfies a fairly strict set of requirements that are largely intended to help provide fair use of the system to all users. It is important to note that what is considered a "standard" transaction changes over time. For some insight, at the time of this writing, an example of SOME of the criteria that are required for a transaction to be considered standard are that it is of the most-recently supported version, finalized, does not exceed a specific size, and only consists of specific script forms.

Since this package does not deal with other bitcoin cash specifics such as network communication and transaction relay, it returns a list of transactions that were accepted which gives the caller a high level of flexibility in how they want to proceed. Typically, this will involve things such as relaying the transactions to other peers on the network and notifying the mining process that new transactions are available.

Feature Overview

The following is a quick overview of the major features. It is not intended to be an exhaustive list.

- Maintain a pool of fully validated transactions
  - Reject non-fully-spent duplicate transactions
  - Reject coinbase transactions
  - Reject double spends (both from the chain and other transactions in pool)
  - Reject invalid transactions according to the network consensus rules
  - Full script execution and validation with signature cache support
  - Individual transaction query support
- Orphan transaction support (transactions that spend from unknown outputs)
  - Configurable limits (see transaction acceptance policy)
  - Automatic addition of orphan transactions that are no longer orphans as new
    transactions are added to the pool
  - Individual orphan transaction query support
- Configurable transaction acceptance policy
  - Option to accept or reject standard transactions
  - Option to accept or reject transactions based on priority calculations
  - Rate limiting of low-fee and free transactions
  - Non-zero fee threshold
  - Max signature operations per transaction
  - Max orphan transaction size
  - Max number of orphan transactions allowed
- Additional metadata tracking for each transaction
  - Timestamp when the transaction was added to the pool
  - Most recent block height when the transaction was added to the pool
  - The fee the transaction pays
  - The starting priority for the transaction
- Manual control of transaction removal
  - Recursive removal of all dependent transactions

Errors

Errors returned by this package are either the raw errors provided by underlying calls or of type mempool.RuleError. Since there are two classes of rules (mempool acceptance rules and blockchain (consensus) acceptance rules), the mempool.RuleError type contains a single Err field which will, in turn, either be a mempool.TxRuleError or a blockchain.RuleError. The first indicates a violation of mempool acceptance rules while the latter indicates a violation of consensus acceptance rules. This allows the caller to easily differentiate between unexpected errors, such as database errors, versus errors due to rule violations through type assertions. In addition, callers can programmatically determine the specific rule violation by type asserting the Err field to one of the aforementioned types and examining their underlying ErrorCode field.

Index

Constants

View Source
const (

	// DefaultEstimateFeeMaxRollback is the default number of rollbacks
	// allowed by the fee estimator for orphaned blocks.
	DefaultEstimateFeeMaxRollback = 2

	// DefaultEstimateFeeMinRegisteredBlocks is the default minimum
	// number of blocks which must be observed by the fee estimator before
	// it will provide fee estimations.
	DefaultEstimateFeeMinRegisteredBlocks = 3
)
View Source
const (
	// DefaultBlockPrioritySize is the default size in bytes for high-
	// priority / low-fee transactions.  It is used to help determine which
	// are allowed into the mempool and consequently affects their relay and
	// inclusion when generating block templates.
	DefaultBlockPrioritySize = 1600000
)
View Source
const (

	// DefaultMinRelayTxFee is the minimum fee in satoshi that is required
	// for a transaction to be treated as free for relay and mining
	// purposes.  It is also used to help determine if a transaction is
	// considered dust and as a base for calculating minimum required fees
	// for larger transactions.  This value is in Satoshi/1000 bytes.
	DefaultMinRelayTxFee = bchutil.Amount(1000)
)

Variables

View Source
var (
	// EstimateFeeDatabaseKey is the key that we use to
	// store the fee estimator in the database.
	EstimateFeeDatabaseKey = []byte("estimatefee")
)

Functions

func DisableLog

func DisableLog()

    DisableLog disables all library log output. Logging output is disabled by default until either UseLogger or SetLogWriter are called.

    func ErrToRejectErr

    func ErrToRejectErr(err error) (wire.RejectCode, string)

      ErrToRejectErr examines the underlying type of the error and returns a reject code and string appropriate to be sent in a wire.MsgReject message.

      func UseLogger

      func UseLogger(logger bchlog.Logger)

        UseLogger uses a specified Logger to output package logging info. This should be used in preference to SetLogWriter if the caller is also using btclog.

        Types

        type BchPerKilobyte

        type BchPerKilobyte float64

          BchPerKilobyte is number with units of bitcoins per kilobyte.

          type Config

          type Config struct {
          	// Policy defines the various mempool configuration options related
          	// to policy.
          	Policy Policy
          
          	// ChainParams identifies which chain parameters the txpool is
          	// associated with.
          	ChainParams *chaincfg.Params
          
          	// FetchUtxoView defines the function to use to fetch unspent
          	// transaction output information.
          	FetchUtxoView func(*bchutil.Tx) (*blockchain.UtxoViewpoint, error)
          
          	// BestHeight defines the function to use to access the block height of
          	// the current best chain.
          	BestHeight func() int32
          
          	// MedianTimePast defines the function to use in order to access the
          	// median time past calculated from the point-of-view of the current
          	// chain tip within the best chain.
          	MedianTimePast func() time.Time
          
          	// CalcSequenceLock defines the function to use in order to generate
          	// the current sequence lock for the given transaction using the passed
          	// utxo view.
          	CalcSequenceLock func(*bchutil.Tx, *blockchain.UtxoViewpoint) (*blockchain.SequenceLock, error)
          
          	// IsDeploymentActive returns true if the target deploymentID is
          	// active, and false otherwise. The mempool uses this function to gauge
          	// if transactions using new to be soft-forked rules should be allowed
          	// into the mempool or not.
          	IsDeploymentActive func(deploymentID uint32) (bool, error)
          
          	// SigCache defines a signature cache to use.
          	SigCache *txscript.SigCache
          
          	// HashCache defines the transaction hash mid-state cache to use.
          	HashCache *txscript.HashCache
          
          	// AddrIndex defines the optional address index instance to use for
          	// indexing the unconfirmed transactions in the memory pool.
          	// This can be nil if the address index is not enabled.
          	AddrIndex *indexers.AddrIndex
          
          	// FeeEstimatator provides a feeEstimator. If it is not nil, the mempool
          	// records all new transactions it observes into the feeEstimator.
          	FeeEstimator *FeeEstimator
          }

            Config is a descriptor containing the memory pool configuration.

            type FeeEstimator

            type FeeEstimator struct {
            	// contains filtered or unexported fields
            }

              FeeEstimator manages the data necessary to create fee estimations. It is safe for concurrent access.

              func NewFeeEstimator

              func NewFeeEstimator(maxRollback, minRegisteredBlocks uint32) *FeeEstimator

                NewFeeEstimator creates a FeeEstimator for which at most maxRollback blocks can be unregistered and which returns an error unless minRegisteredBlocks have been registered with it.

                func RestoreFeeEstimator

                func RestoreFeeEstimator(data FeeEstimatorState) (*FeeEstimator, error)

                  RestoreFeeEstimator takes a FeeEstimatorState that was previously returned by Save and restores it to a FeeEstimator

                  func (*FeeEstimator) EstimateFee

                  func (ef *FeeEstimator) EstimateFee(numBlocks uint32) (BchPerKilobyte, error)

                    EstimateFee estimates the fee per byte to have a tx confirmed a given number of blocks from now.

                    func (*FeeEstimator) LastKnownHeight

                    func (ef *FeeEstimator) LastKnownHeight() int32

                      LastKnownHeight returns the height of the last block which was registered.

                      func (*FeeEstimator) ObserveTransaction

                      func (ef *FeeEstimator) ObserveTransaction(t *TxDesc)

                        ObserveTransaction is called when a new transaction is observed in the mempool.

                        func (*FeeEstimator) RegisterBlock

                        func (ef *FeeEstimator) RegisterBlock(block *bchutil.Block) error

                          RegisterBlock informs the fee estimator of a new block to take into account.

                          func (*FeeEstimator) Rollback

                          func (ef *FeeEstimator) Rollback(hash *chainhash.Hash) error

                            Rollback unregisters a recently registered block from the FeeEstimator. This can be used to reverse the effect of an orphaned block on the fee estimator. The maximum number of rollbacks allowed is given by maxRollbacks.

                            Note: not everything can be rolled back because some transactions are deleted if they have been observed too long ago. That means the result of Rollback won't always be exactly the same as if the last block had not happened, but it should be close enough.

                            func (*FeeEstimator) Save

                            func (ef *FeeEstimator) Save() FeeEstimatorState

                              Save records the current state of the FeeEstimator to a []byte that can be restored later.

                              type FeeEstimatorState

                              type FeeEstimatorState []byte

                                FeeEstimatorState represents a saved FeeEstimator that can be restored with data from an earlier session of the program.

                                type Policy

                                type Policy struct {
                                	// MaxTxVersion is the transaction version that the mempool should
                                	// accept.  All transactions above this version are rejected as
                                	// non-standard.
                                	MaxTxVersion int32
                                
                                	// DisableRelayPriority defines whether to relay free or low-fee
                                	// transactions that do not have enough priority to be relayed.
                                	DisableRelayPriority bool
                                
                                	// AcceptNonStd defines whether to accept non-standard transactions. If
                                	// true, non-standard transactions will be accepted into the mempool.
                                	// Otherwise, all non-standard transactions will be rejected.
                                	AcceptNonStd bool
                                
                                	// FreeTxRelayLimit defines the given amount in thousands of bytes
                                	// per minute that transactions with no fee are rate limited to.
                                	FreeTxRelayLimit float64
                                
                                	// MaxOrphanTxs is the maximum number of orphan transactions
                                	// that can be queued.
                                	MaxOrphanTxs int
                                
                                	// MaxOrphanTxSize is the maximum size allowed for orphan transactions.
                                	// This helps prevent memory exhaustion attacks from sending a lot of
                                	// of big orphans.
                                	MaxOrphanTxSize int
                                
                                	// LimitSigChecks applies an additional standardness limit to the number
                                	// of signature checks in each transaction.
                                	LimitSigChecks bool
                                
                                	// MinRelayTxFee defines the minimum transaction fee in BCH/kB to be
                                	// considered a non-zero fee.
                                	MinRelayTxFee bchutil.Amount
                                }

                                  Policy houses the policy (configuration parameters) which is used to control the mempool.

                                  type RuleError

                                  type RuleError struct {
                                  	Err error
                                  }

                                    RuleError identifies a rule violation. It is used to indicate that processing of a transaction failed due to one of the many validation rules. The caller can use type assertions to determine if a failure was specifically due to a rule violation and use the Err field to access the underlying error, which will be either a TxRuleError or a blockchain.RuleError.

                                    func (RuleError) Error

                                    func (e RuleError) Error() string

                                      Error satisfies the error interface and prints human-readable errors.

                                      type SatoshiPerByte

                                      type SatoshiPerByte float64

                                        SatoshiPerByte is number with units of satoshis per byte.

                                        func NewSatoshiPerByte

                                        func NewSatoshiPerByte(fee bchutil.Amount, size uint32) SatoshiPerByte

                                          NewSatoshiPerByte creates a SatoshiPerByte from an Amount and a size in bytes.

                                          func (SatoshiPerByte) Fee

                                          func (rate SatoshiPerByte) Fee(size uint32) bchutil.Amount

                                            Fee returns the fee for a transaction of a given size for the given fee rate.

                                            func (SatoshiPerByte) ToBchPerKb

                                            func (rate SatoshiPerByte) ToBchPerKb() BchPerKilobyte

                                              ToBchPerKb returns a float value that represents the given SatoshiPerByte converted to satoshis per kb.

                                              type Tag

                                              type Tag uint64

                                                Tag represents an identifier to use for tagging orphan transactions. The caller may choose any scheme it desires, however it is common to use peer IDs so that orphans can be identified by which peer first relayed them.

                                                type TxDesc

                                                type TxDesc struct {
                                                	mining.TxDesc
                                                
                                                	// StartingPriority is the priority of the transaction when it was added
                                                	// to the pool.
                                                	StartingPriority float64
                                                }

                                                  TxDesc is a descriptor containing a transaction in the mempool along with additional metadata.

                                                  type TxPool

                                                  type TxPool struct {
                                                  	// contains filtered or unexported fields
                                                  }

                                                    TxPool is used as a source of transactions that need to be mined into blocks and relayed to other peers. It is safe for concurrent access from multiple peers.

                                                    func New

                                                    func New(cfg *Config) *TxPool

                                                      New returns a new memory pool for validating and storing standalone transactions until they are mined into a block.

                                                      func (*TxPool) CheckSpend

                                                      func (mp *TxPool) CheckSpend(op wire.OutPoint) *bchutil.Tx

                                                        CheckSpend checks whether the passed outpoint is already spent by a transaction in the mempool. If that's the case the spending transaction will be returned, if not nil will be returned.

                                                        func (*TxPool) Count

                                                        func (mp *TxPool) Count() int

                                                          Count returns the number of transactions in the main pool. It does not include the orphan pool.

                                                          This function is safe for concurrent access.

                                                          func (*TxPool) DecodeCompressedBlock

                                                          func (mp *TxPool) DecodeCompressedBlock(iBlock interface{}) (*wire.MsgBlock, error)

                                                            DecodeCompressedBlock takes in a block interface and attempts to decode it according to know block compressing algorithms. If successful it returns the complete block.

                                                            func (*TxPool) FetchInputUtxos

                                                            func (mp *TxPool) FetchInputUtxos(tx *bchutil.Tx) (*blockchain.UtxoViewpoint, error)

                                                              FetchInputUtxos loads utxo details about the input transactions referenced by the passed transaction. First, it loads the details form the viewpoint of the main chain, then it adjusts them based upon the contents of the transaction pool.

                                                              This function is safe for concurrent access.

                                                              func (*TxPool) FetchTransaction

                                                              func (mp *TxPool) FetchTransaction(txHash *chainhash.Hash) (*bchutil.Tx, error)

                                                                FetchTransaction returns the requested transaction from the transaction pool. This only fetches from the main transaction pool and does not include orphans.

                                                                This function is safe for concurrent access.

                                                                func (*TxPool) FetchTxDesc

                                                                func (mp *TxPool) FetchTxDesc(txHash *chainhash.Hash) (*TxDesc, error)

                                                                  FetchTxDesc returns the requested transaction description from the transaction pool. This only fetches from the main transaction pool and does not include orphans.

                                                                  This function is safe for concurrent access.

                                                                  func (*TxPool) FetchUtxoView

                                                                  func (mp *TxPool) FetchUtxoView(tx *bchutil.Tx) (*blockchain.UtxoViewpoint, error)

                                                                    FetchUtxoView loads utxo details about the transactions inputs and outputs. First, it loads the details form the viewpoint of the main chain, then it adjusts them based upon the contents of the transaction pool.

                                                                    This function is safe for concurrent access.

                                                                    func (*TxPool) HaveTransaction

                                                                    func (mp *TxPool) HaveTransaction(hash *chainhash.Hash) bool

                                                                      HaveTransaction returns whether or not the passed transaction already exists in the main pool or in the orphan pool.

                                                                      This function is safe for concurrent access.

                                                                      func (*TxPool) IsOrphanInPool

                                                                      func (mp *TxPool) IsOrphanInPool(hash *chainhash.Hash) bool

                                                                        IsOrphanInPool returns whether or not the passed transaction already exists in the orphan pool.

                                                                        This function is safe for concurrent access.

                                                                        func (*TxPool) IsTransactionInPool

                                                                        func (mp *TxPool) IsTransactionInPool(hash *chainhash.Hash) bool

                                                                          IsTransactionInPool returns whether or not the passed transaction already exists in the main pool.

                                                                          This function is safe for concurrent access.

                                                                          func (*TxPool) LastUpdated

                                                                          func (mp *TxPool) LastUpdated() time.Time

                                                                            LastUpdated returns the last time a transaction was added to or removed from the main pool. It does not include the orphan pool.

                                                                            This function is safe for concurrent access.

                                                                            func (*TxPool) MaybeAcceptTransaction

                                                                            func (mp *TxPool) MaybeAcceptTransaction(tx *bchutil.Tx, isNew, rateLimit bool) ([]*chainhash.Hash, *TxDesc, error)

                                                                              MaybeAcceptTransaction is the main workhorse for handling insertion of new free-standing transactions into a memory pool. It includes functionality such as rejecting duplicate transactions, ensuring transactions follow all rules, detecting orphan transactions, and insertion into the memory pool.

                                                                              If the transaction is an orphan (missing parent transactions), the transaction is NOT added to the orphan pool, but each unknown referenced parent is returned. Use ProcessTransaction instead if new orphans should be added to the orphan pool.

                                                                              This function is safe for concurrent access.

                                                                              func (*TxPool) MiningDescs

                                                                              func (mp *TxPool) MiningDescs() []*mining.TxDesc

                                                                                MiningDescs returns a slice of mining descriptors for all the transactions in the pool.

                                                                                This is part of the mining.TxSource interface implementation and is safe for concurrent access as required by the interface contract.

                                                                                func (*TxPool) ProcessOrphans

                                                                                func (mp *TxPool) ProcessOrphans(acceptedTx *bchutil.Tx) []*TxDesc

                                                                                  ProcessOrphans determines if there are any orphans which depend on the passed transaction hash (it is possible that they are no longer orphans) and potentially accepts them to the memory pool. It repeats the process for the newly accepted transactions (to detect further orphans which may no longer be orphans) until there are no more.

                                                                                  It returns a slice of transactions added to the mempool. A nil slice means no transactions were moved from the orphan pool to the mempool.

                                                                                  This function is safe for concurrent access.

                                                                                  func (*TxPool) ProcessTransaction

                                                                                  func (mp *TxPool) ProcessTransaction(tx *bchutil.Tx, allowOrphan, rateLimit bool, tag Tag) ([]*TxDesc, error)

                                                                                    ProcessTransaction is the main workhorse for handling insertion of new free-standing transactions into the memory pool. It includes functionality such as rejecting duplicate transactions, ensuring transactions follow all rules, orphan transaction handling, and insertion into the memory pool.

                                                                                    It returns a slice of transactions added to the mempool. When the error is nil, the list will include the passed transaction itself along with any additional orphan transaactions that were added as a result of the passed one being accepted.

                                                                                    This function is safe for concurrent access.

                                                                                    func (*TxPool) RawMempoolVerbose

                                                                                    func (mp *TxPool) RawMempoolVerbose() map[string]*btcjson.GetRawMempoolVerboseResult

                                                                                      RawMempoolVerbose returns all of the entries in the mempool as a fully populated btcjson result.

                                                                                      This function is safe for concurrent access.

                                                                                      func (*TxPool) RemoveDoubleSpends

                                                                                      func (mp *TxPool) RemoveDoubleSpends(tx *bchutil.Tx)

                                                                                        RemoveDoubleSpends removes all transactions which spend outputs spent by the passed transaction from the memory pool. Removing those transactions then leads to removing all transactions which rely on them, recursively. This is necessary when a block is connected to the main chain because the block may contain transactions which were previously unknown to the memory pool.

                                                                                        This function is safe for concurrent access.

                                                                                        func (*TxPool) RemoveOrphan

                                                                                        func (mp *TxPool) RemoveOrphan(tx *bchutil.Tx)

                                                                                          RemoveOrphan removes the passed orphan transaction from the orphan pool and previous orphan index.

                                                                                          This function is safe for concurrent access.

                                                                                          func (*TxPool) RemoveOrphansByTag

                                                                                          func (mp *TxPool) RemoveOrphansByTag(tag Tag) uint64

                                                                                            RemoveOrphansByTag removes all orphan transactions tagged with the provided identifier.

                                                                                            This function is safe for concurrent access.

                                                                                            func (*TxPool) RemoveTransaction

                                                                                            func (mp *TxPool) RemoveTransaction(tx *bchutil.Tx, removeRedeemers bool)

                                                                                              RemoveTransaction removes the passed transaction from the mempool. When the removeRedeemers flag is set, any transactions that redeem outputs from the removed transaction will also be removed recursively from the mempool, as they would otherwise become orphans.

                                                                                              This function is safe for concurrent access.

                                                                                              func (*TxPool) TxDescs

                                                                                              func (mp *TxPool) TxDescs() []*TxDesc

                                                                                                TxDescs returns a slice of descriptors for all the transactions in the pool. The descriptors are to be treated as read only.

                                                                                                This function is safe for concurrent access.

                                                                                                func (*TxPool) TxHashes

                                                                                                func (mp *TxPool) TxHashes() []*chainhash.Hash

                                                                                                  TxHashes returns a slice of hashes for all of the transactions in the memory pool.

                                                                                                  This function is safe for concurrent access.

                                                                                                  type TxRuleError

                                                                                                  type TxRuleError struct {
                                                                                                  	RejectCode  wire.RejectCode // The code to send with reject messages
                                                                                                  	Description string          // Human readable description of the issue
                                                                                                  }

                                                                                                    TxRuleError identifies a rule violation. It is used to indicate that processing of a transaction failed due to one of the many validation rules. The caller can use type assertions to determine if a failure was specifically due to a rule violation and access the ErrorCode field to ascertain the specific reason for the rule violation.

                                                                                                    func (TxRuleError) Error

                                                                                                    func (e TxRuleError) Error() string

                                                                                                      Error satisfies the error interface and prints human-readable errors.