operation

package
v0.43.1-rc.1.access-me... Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Sep 30, 2025 License: AGPL-3.0 Imports: 24 Imported by: 1

Documentation

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

func AppendPrefixKeyPart added in v0.41.0 

func AppendPrefixKeyPart(buf []byte, v any) []byte

AppendPrefixKeyPart appends v in binary prefix format to buf. NOTE: this function needs to be in sync with prefixKeyPartLength.

func BatchIndexLightTransactionResult added in v0.39.4 

func BatchIndexLightTransactionResult(w storage.Writer, blockID flow.Identifier, txIndex uint32, transactionResult *flow.LightTransactionResult) error

func BatchIndexTransactionResultErrorMessage added in v0.39.4 

func BatchIndexTransactionResultErrorMessage(w storage.Writer, blockID flow.Identifier, transactionResultErrorMessage *flow.TransactionResultErrorMessage) error

BatchIndexTransactionResultErrorMessage indexes a transaction result error message by index within the block using a batch write.

func BatchInsertLightTransactionResult added in v0.39.4 

func BatchInsertLightTransactionResult(w storage.Writer, blockID flow.Identifier, transactionResult *flow.LightTransactionResult) error

func BatchInsertTransactionResultErrorMessage added in v0.39.4 

func BatchInsertTransactionResultErrorMessage(w storage.Writer, blockID flow.Identifier, transactionResultErrorMessage *flow.TransactionResultErrorMessage) error

BatchInsertTransactionResultErrorMessage inserts a transaction result error message by block ID and transaction ID into the database using a batch write.

func BatchRemoveTransactionResultsByBlockID added in v0.39.4 

func BatchRemoveTransactionResultsByBlockID(blockID flow.Identifier, batch storage.ReaderBatchWriter) error

BatchRemoveTransactionResultsByBlockID removes transaction results for the given blockID in a provided batch. No errors are expected during normal operation, but it may return generic error if badger fails to process request

func BlockExists added in v0.43.0 

func BlockExists(r storage.Reader, blockID flow.Identifier) (bool, error)

BlockExists checks whether the block exists in the database. No errors are expected during normal operation.

func CommonPrefix added in v0.42.0 

func CommonPrefix(startPrefix, endPrefix []byte) []byte

CommonPrefix returns common prefix of startPrefix and endPrefix. The common prefix is used to narrow down the SSTables that BadgerDB's iterator picks up.

func EncodeKeyPart 

func EncodeKeyPart(v interface{}) []byte

EncodeKeyPart encodes a value to be used as a part of a key to be stored in storage.

func ExistChunkLocator added in v0.39.1 

func ExistChunkLocator(r storage.Reader, locatorID flow.Identifier) (bool, error)

func ExistExecutionResult added in v0.39.4 

func ExistExecutionResult(r storage.Reader, blockID flow.Identifier) (bool, error)

ExistExecutionResult checks if the execution node has its OWN Execution Result for the specified block. No errors are expected during normal operation.

func Exists 

func Exists(key []byte, keyExists *bool) func(storage.Reader) error

func FindHeaders added in v0.43.0 

func FindHeaders(r storage.Reader, filter func(header *flow.Header) bool, found *[]flow.Header) error

FindHeaders iterates through all headers, calling `filter` on each, and adding them to the `found` slice if `filter` returned true

func FindHighestAtOrBelow 

func FindHighestAtOrBelow(prefix []byte, height uint64, entity interface{}) func(storage.Reader) error

func FindHighestAtOrBelowByPrefix 

func FindHighestAtOrBelowByPrefix(r storage.Reader, prefix []byte, height uint64, entity any) (errToReturn error)

FindHighestAtOrBelowByPrefix is for database entries that are indexed by block height. It is suitable to search keys with the format prefix` + `height` (where "+" denotes concatenation of binary strings). The height is encoded as Big-Endian (entries with numerically smaller height have lexicographically smaller key). The function finds the *highest* key with the given prefix and height equal to or below the given height.

func GetBlockIDsByStatus added in v0.39.2 

func GetBlockIDsByStatus(r storage.Reader, blockIDs *[]flow.Identifier,
	targetUploadStatus bool) error

GetBlockIDsByStatus returns all IDs of stored ComputationResult instances.

func GetComputationResultUploadStatus added in v0.39.2 

func GetComputationResultUploadStatus(r storage.Reader, blockID flow.Identifier,
	wasUploadCompleted *bool) error

GetComputationResult returns stored ComputationResult instance with given ID.

func HasExecutionForkEvidence added in v0.42.1 

func HasExecutionForkEvidence(r storage.Reader) (bool, error)

HasExecutionForkEvidence checks if conflicting seals record exists in the database. No errors are expected during normal operations.

func IndexBlockContainingCollectionGuarantee added in v0.43.0 

func IndexBlockContainingCollectionGuarantee(w storage.Writer, collID flow.Identifier, blockID flow.Identifier) error

IndexBlockContainingCollectionGuarantee produces a mapping from the ID of a flow.CollectionGuarantee to the block ID containing this guarantee.

CAUTION:

  • The caller must acquire the lock ??? and hold it until the database write has been committed. TODO: USE LOCK, we want to protect this mapping from accidental overwrites (because the key is not derived from the value via a collision-resistant hash)
  • A collection can be included in multiple *unfinalized* blocks. However, the implementation assumes a one-to-one map from collection ID to a *single* block ID. This holds for FINALIZED BLOCKS ONLY *and* only in the ABSENCE of BYZANTINE collector CLUSTERS (which the mature protocol must tolerate). Hence, this function should be treated as a temporary solution, which requires generalization (one-to-many mapping) for soft finality and the mature protocol.

Expected errors during normal operations: TODO: return storage.ErrAlreadyExists or storage.ErrDataMismatch

func IndexCertifiedBlockByView added in v0.43.0 

func IndexCertifiedBlockByView(lctx lockctx.Proof, rw storage.ReaderBatchWriter, view uint64, blockID flow.Identifier) error

IndexCertifiedBlockByView indexes a CERTIFIED block by its view. HotStuff guarantees that there is at most one certified block per view. Note that this does not hold for uncertified proposals, as a byzantine leader might produce multiple proposals for the same view.

CAUTION: The caller must acquire the storage.LockInsertBlock and hold it until the database write has been committed.

Hence, only certified blocks (i.e. blocks that have received a QC) can be indexed! Returns storage.ErrAlreadyExists if an ID has already been finalized for this view. No other errors are expected during normal operation.

func IndexClusterBlockByReferenceHeight added in v0.43.0 

func IndexClusterBlockByReferenceHeight(lctx lockctx.Proof, w storage.Writer, refHeight uint64, clusterBlockID flow.Identifier) error

IndexClusterBlockByReferenceHeight indexes a cluster block ID by its reference block height. The cluster block ID is included in the key for more efficient traversal. Only finalized cluster blocks should be included in this index. The key looks like: <prefix 0:1><ref_height 1:9><cluster_block_id 9:41>

func IndexClusterBlockHeight added in v0.43.0 

func IndexClusterBlockHeight(lctx lockctx.Proof, w storage.Writer, clusterID flow.ChainID, height uint64, blockID flow.Identifier) error

IndexClusterBlockHeight indexes a cluster block from the specified cluster by its height.

func IndexCollectionByTransaction added in v0.43.0 

func IndexCollectionByTransaction(lctx lockctx.Proof, w storage.Writer, txID flow.Identifier, collectionID flow.Identifier) error

IndexCollectionByTransaction indexes the given collection ID, keyed by the transaction ID.

CAUTION:

  • The caller must acquire the storage.LockInsertCollection and hold it until the database write has been committed.
  • OVERWRITES existing data (potential for data corruption): This method silently overrides existing data without any sanity checks whether data for the same key already exits. Note that the Flow protocol mandates that for a previously persisted key, the data is never changed to a different value. Changing data could cause the node to publish inconsistent data and to be slashed, or the protocol to be compromised as a whole. This method does not contain any safeguards to prevent such data corruption. The lock proof serves as a reminder that the CALLER is responsible to ensure that the DEDUPLICATION CHECK is done elsewhere ATOMICALLY with this write operation.

WARNING, this index is NOT BFT in its current form: Honest clusters ensure a transaction can only belong to one collection. However, in rare cases, the collector clusters can exceed byzantine thresholds -- making it possible to produce multiple finalized collections (aka guaranteed collections) containing the same transaction repeatedly. TODO: eventually we need to handle Byzantine clusters

No errors are expected during normal operation.

func IndexCollectionPayload added in v0.39.4 

func IndexCollectionPayload(lctx lockctx.Proof, w storage.Writer, clusterBlockID flow.Identifier, txIDs []flow.Identifier) error

IndexCollectionPayload populates the map from a cluster block ID to the batch of transactions it contains.

CAUTION:

  • The caller must acquire the storage.LockInsertOrFinalizeClusterBlock and hold it until the database write has been committed.
  • OVERWRITES existing data (potential for data corruption): This method silently overrides existing data without any sanity checks whether data for the same key already exits. Note that the Flow protocol mandates that for a previously persisted key, the data is never changed to a different value. Changing data could cause the node to publish inconsistent data and to be slashed, or the protocol to be compromised as a whole. This method does not contain any safeguards to prevent such data corruption. The lock proof serves as a reminder that the CALLER is responsible to ensure that the DEDUPLICATION CHECK is done elsewhere ATOMICALLY with this write operation.

No errors are expected during normal operation.

func IndexEpochProtocolState added in v0.43.0 

func IndexEpochProtocolState(w storage.Writer, blockID flow.Identifier, epochProtocolStateEntryID flow.Identifier) error

IndexEpochProtocolState indexes an epoch protocol state entry by block ID. Error returns:

  • generic error in case of unexpected failure from the database layer or encoding failure.

func IndexExecutionReceipts added in v0.39.4 

func IndexExecutionReceipts(w storage.Writer, blockID, receiptID flow.Identifier) error

IndexExecutionReceipts adds the given execution receipts to the set of all known receipts for the given block. It produces a mapping from block ID to the set of all known receipts for that block. One block could have multiple receipts, even if they are from the same executor.

This method is idempotent, and can be called repeatedly with the same block ID and receipt ID, without the risk of data corruption.

No errors are expected during normal operation.

func IndexExecutionResult added in v0.39.4 

func IndexExecutionResult(w storage.Writer, blockID flow.Identifier, resultID flow.Identifier) error

IndexExecutionResult indexes the Execution Node's OWN Execution Result by the executed block's ID.

CAUTION:

  • OVERWRITES existing data (potential for data corruption): This method silently overrides existing data without any sanity checks whether data for the same key already exits. Note that the Flow protocol mandates that for a previously persisted key, the data is never changed to a different value. Changing data could cause the node to publish inconsistent data and to be slashed, or the protocol to be compromised as a whole. This method does not contain any safeguards to prevent such data corruption.

TODO: USE LOCK, we want to protect this mapping from accidental overwrites (because the key is not derived from the value via a collision-resistant hash)

No errors are expected during normal operation.

func IndexFinalizedBlockByHeight added in v0.43.0 

func IndexFinalizedBlockByHeight(lctx lockctx.Proof, rw storage.ReaderBatchWriter, height uint64, blockID flow.Identifier) error

IndexFinalizedBlockByHeight indexes a block by its height. It must ONLY be called on FINALIZED BLOCKS.

CAUTION: The caller must acquire the storage.LockFinalizeBlock and hold it until the database write has been committed.

This function guarantees that the index is only inserted once for each height. We return storage.ErrAlreadyExists if an entry for the given height already exists in the database. No other errors are expected during normal operation.

func IndexFinalizedSealByBlockID added in v0.43.0 

func IndexFinalizedSealByBlockID(w storage.Writer, sealedBlockID flow.Identifier, sealID flow.Identifier) error

IndexFinalizedSealByBlockID indexes the _finalized_ seal by the sealed block ID. Example: A <- B <- C(SealA) when block C is finalized, we create the index `A.ID->SealA.ID`

CAUTION:

  • The caller must acquire the storage.LockFinalizeBlock and hold it until the database write has been committed. TODO: add lock proof as input and check for holding the lock in the implementation
  • OVERWRITES existing data (potential for data corruption): This method silently overrides existing data without any sanity checks whether data for the same key already exits. Note that the Flow protocol mandates that for a previously persisted key, the data is never changed to a different value. Changing data could cause the node to publish inconsistent data and to be slashed, or the protocol to be compromised as a whole. This method does not contain any safeguards to prevent such data corruption. The lock proof serves as a reminder that the CALLER is responsible to ensure that the DEDUPLICATION CHECK is done elsewhere ATOMICALLY with this write operation.

No errors are expected during normal operation.

func IndexGuarantee added in v0.43.0 

func IndexGuarantee(lctx lockctx.Proof, rw storage.ReaderBatchWriter, collectionID flow.Identifier, guaranteeID flow.Identifier) error

IndexGuarantee inserts a flow.CollectionGuarantee into the database, keyed by the collection ID.

Expected errors during normal operations:

func IndexLatestSealAtBlock added in v0.43.0 

func IndexLatestSealAtBlock(lctx lockctx.Proof, w storage.Writer, blockID flow.Identifier, sealID flow.Identifier) error

IndexLatestSealAtBlock persists the highest seal that was included in the fork with head blockID. Frequently, the highest seal included in this block's payload. However, if there are no seals in this block, sealID should reference the highest seal in blockID's ancestors.

CAUTION:

  • The caller must acquire the storage.LockInsertBlock and hold it until the database write has been committed.
  • OVERWRITES existing data (potential for data corruption): This method silently overrides existing data without any sanity checks whether data for the same key already exits. Note that the Flow protocol mandates that for a previously persisted key, the data is never changed to a different value. Changing data could cause the node to publish inconsistent data and to be slashed, or the protocol to be compromised as a whole. This method does not contain any safeguards to prevent such data corruption. The lock proof serves as a reminder that the CALLER is responsible to ensure that the DEDUPLICATION CHECK is done elsewhere ATOMICALLY with this write operation.

No errors are expected during normal operation.

func IndexOwnExecutionReceipt added in v0.39.4 

func IndexOwnExecutionReceipt(w storage.Writer, blockID flow.Identifier, receiptID flow.Identifier) error

IndexOwnExecutionReceipt indexes the Execution Node's OWN execution receipt by the executed block ID.

CAUTION:

  • OVERWRITES existing data (potential for data corruption): This method silently overrides existing data without any sanity checks whether data for the same key already exits. Note that the Flow protocol mandates that for a previously persisted key, the data is never changed to a different value. Changing data could cause the node to publish inconsistent data and to be slashed, or the protocol to be compromised as a whole. This method does not contain any safeguards to prevent such data corruption. The caller is responsible to ensure that the DEDUPLICATION CHECK is done elsewhere ATOMICALLY with this write operation.

No errors are expected during normal operation.

func IndexPayloadGuarantees added in v0.43.0 

func IndexPayloadGuarantees(lctx lockctx.Proof, w storage.Writer, blockID flow.Identifier, guarIDs []flow.Identifier) error

IndexPayloadGuarantees indexes the list of collection guarantees that were included in the specified block, keyed by the block ID. It produces a mapping from block ID to the list of collection guarantees contained in the block's payload. The collection guarantees are represented by their respective IDs.

CAUTION:

  • The caller must acquire the storage.LockInsertBlock and hold it until the database write has been committed.
  • OVERWRITES existing data (potential for data corruption): This method silently overrides existing data without any sanity checks whether data for the same key already exits. Note that the Flow protocol mandates that for a previously persisted key, the data is never changed to a different value. Changing data could cause the node to publish inconsistent data and to be slashed, or the protocol to be compromised as a whole. This method does not contain any safeguards to prevent such data corruption. The lock proof serves as a reminder that the CALLER is responsible to ensure that the DEDUPLICATION CHECK is done elsewhere ATOMICALLY with this write operation.

No errors expected during normal operations.

func IndexPayloadProtocolStateID added in v0.43.0 

func IndexPayloadProtocolStateID(lctx lockctx.Proof, w storage.Writer, blockID flow.Identifier, stateID flow.Identifier) error

IndexPayloadProtocolStateID indexes the given Protocol State ID by the block ID. The Protocol State ID represents the configuration, which the block proposes to become active *after* the block's certification. Every block states the ID of the Protocol State it proposes as part of the payload.

CAUTION:

  • The caller must acquire the storage.LockInsertBlock and hold it until the database write has been committed.
  • OVERWRITES existing data (potential for data corruption): This method silently overrides existing data without any sanity checks whether data for the same key already exits. Note that the Flow protocol mandates that for a previously persisted key, the data is never changed to a different value. Changing data could cause the node to publish inconsistent data and to be slashed, or the protocol to be compromised as a whole. This method does not contain any safeguards to prevent such data corruption. The lock proof serves as a reminder that the CALLER is responsible to ensure that the DEDUPLICATION CHECK is done elsewhere ATOMICALLY with this write operation.

No errors are expected during normal operation.

func IndexPayloadReceipts added in v0.43.0 

func IndexPayloadReceipts(lctx lockctx.Proof, w storage.Writer, blockID flow.Identifier, receiptIDs []flow.Identifier) error

IndexPayloadReceipts indexes the list of Execution Receipts that were included in the specified block by the block ID. It produces a mapping from block ID to the list of Receipts contained in the block's payload. Execution Receipts are represented by their respective IDs.

CAUTION:

  • The caller must acquire the storage.LockInsertBlock and hold it until the database write has been committed.
  • OVERWRITES existing data (potential for data corruption): This method silently overrides existing data without any sanity checks whether data for the same key already exits. Note that the Flow protocol mandates that for a previously persisted key, the data is never changed to a different value. Changing data could cause the node to publish inconsistent data and to be slashed, or the protocol to be compromised as a whole. This method does not contain any safeguards to prevent such data corruption. The lock proof serves as a reminder that the CALLER is responsible to ensure that the DEDUPLICATION CHECK is done elsewhere ATOMICALLY with this write operation.

No errors are expected during normal operation.

func IndexPayloadResults added in v0.43.0 

func IndexPayloadResults(lctx lockctx.Proof, w storage.Writer, blockID flow.Identifier, resultIDs []flow.Identifier) error

IndexPayloadResults indexes the list of Execution Results that were included in the specified block by the block ID. It produces a mapping from block ID to the list of Results contained in the block's payload. Execution Results are represented by their respective IDs.

CAUTION:

  • The caller must acquire the storage.LockInsertBlock and hold it until the database write has been committed.
  • OVERWRITES existing data (potential for data corruption): This method silently overrides existing data without any sanity checks whether data for the same key already exits. Note that the Flow protocol mandates that for a previously persisted key, the data is never changed to a different value. Changing data could cause the node to publish inconsistent data and to be slashed, or the protocol to be compromised as a whole. This method does not contain any safeguards to prevent such data corruption. The lock proof serves as a reminder that the CALLER is responsible to ensure that the DEDUPLICATION CHECK is done elsewhere ATOMICALLY with this write operation.

No errors are expected during normal operation.

func IndexPayloadSeals added in v0.43.0 

func IndexPayloadSeals(lctx lockctx.Proof, w storage.Writer, blockID flow.Identifier, sealIDs []flow.Identifier) error

IndexPayloadSeals indexes the list of Seals that were included in the specified block by the block ID. It produces a mapping from block ID to the list of seals contained in the block's payload. The seals are represented by their respective IDs.

CAUTION:

  • The caller must acquire the storage.LockInsertBlock and hold it until the database write has been committed.
  • OVERWRITES existing data (potential for data corruption): This method silently overrides existing data without any sanity checks whether data for the same key already exits. Note that the Flow protocol mandates that for a previously persisted key, the data is never changed to a different value. Changing data could cause the node to publish inconsistent data and to be slashed, or the protocol to be compromised as a whole. This method does not contain any safeguards to prevent such data corruption. The lock proof serves as a reminder that the CALLER is responsible to ensure that the DEDUPLICATION CHECK is done elsewhere ATOMICALLY with this write operation.

No errors are expected during normal operation.

func IndexProtocolKVStore added in v0.43.0 

func IndexProtocolKVStore(w storage.Writer, blockID flow.Identifier, protocolKVStoreID flow.Identifier) error

IndexProtocolKVStore indexes a protocol KV store by block ID. Error returns:

  • storage.ErrAlreadyExists if the key already exists in the database.
  • generic error in case of unexpected failure from the database layer

func IndexReferenceBlockByClusterBlock added in v0.43.0 

func IndexReferenceBlockByClusterBlock(lctx lockctx.Proof, w storage.Writer, clusterBlockID, refID flow.Identifier) error

IndexReferenceBlockByClusterBlock updates the reference block ID for the given cluster block ID. While each cluster block specifies a reference block in its payload, we maintain this additional lookup for performance reasons.

func IndexStateCommitment added in v0.39.4 

func IndexStateCommitment(lctx lockctx.Proof, rw storage.ReaderBatchWriter, blockID flow.Identifier, commit flow.StateCommitment) error

IndexStateCommitment indexes a state commitment by the block ID whose execution results in that state. The function ensures data integrity by first checking if a commitment already exists for the given block and rejecting overwrites with different values. This function is idempotent, i.e. repeated calls with the *initially* indexed value are no-ops.

CAUTION:

  • Confirming that no value is already stored and the subsequent write must be atomic to prevent data corruption. The caller must acquire the storage.LockInsertOwnReceipt and hold it until the database write has been committed.

Expected error returns during normal operations:

func IndexTransactionResult added in v0.39.4 

func IndexTransactionResult(w storage.Writer, blockID flow.Identifier, txIndex uint32, transactionResult *flow.TransactionResult) error

func IndexVersionBeaconByHeight added in v0.39.2 

func IndexVersionBeaconByHeight(
	w storage.Writer,
	beacon *flow.SealedVersionBeacon,
) error

IndexVersionBeaconByHeight stores a sealed version beacon indexed by flow.SealedVersionBeacon.SealHeight.

No errors are expected during normal operation.

func InsertAndIndexResultApproval added in v0.43.0 

func InsertAndIndexResultApproval(approval *flow.ResultApproval) func(lctx lockctx.Proof, rw storage.ReaderBatchWriter) error

InsertAndIndexResultApproval atomically performs the following storage operations:

  1. Store ResultApproval by its ID (in this step, accidental overwrites with inconsistent values are prevented by using a collision-resistant hash to derive the key from the value)
  2. Index approval by the executed chunk, specifically the key pair (ExecutionResultID, chunk index). - first, we ensure that no _different_ approval has already been indexed for the same key pair - only if the prior check succeeds, we write the index to the database

CAUTION:

  • In general, the Flow protocol requires multiple approvals for the same chunk from different verification nodes. In other words, there are multiple different approvals for the same chunk. Therefore, this index Executed Chunk ➜ ResultApproval ID is *only safe* to be used by Verification Nodes for tracking their own approvals (for the same ExecutionResult, a Verifier will always produce the same approval)
  • In order to make sure only one approval is indexed for the chunk, _all calls_ to `InsertAndIndexResultApproval` must be synchronized by the higher-logic. Currently, we have the lockctx.Proof to prove the higher logic is holding the lock inserting the approval after checking that the approval is not already indexed.

Expected error returns:

  • `storage.ErrDataMismatch` if a *different* approval for the same key pair (ExecutionResultID, chunk index) is already indexed

func InsertChunkDataPack 

func InsertChunkDataPack(w storage.Writer, c *storage.StoredChunkDataPack) error

InsertChunkDataPack inserts a chunk data pack keyed by chunk ID. any error are exceptions

func InsertChunkLocator added in v0.39.1 

func InsertChunkLocator(w storage.Writer, locator *chunks.Locator) error

func InsertEpochCommit added in v0.43.0 

func InsertEpochCommit(w storage.Writer, eventID flow.Identifier, event *flow.EpochCommit) error

func InsertEpochFirstHeight added in v0.43.0 

func InsertEpochFirstHeight(lctx lockctx.Proof, rw storage.ReaderBatchWriter, epoch, height uint64) error

InsertEpochFirstHeight inserts the height of the first block in the given epoch. The first block of an epoch E is the finalized block with view >= E.FirstView. Although we don't store the final height of an epoch, it can be inferred from this index. The caller must hold storage.LockFinalizeBlock. This function enforces each index is written exactly once. Returns storage.ErrAlreadyExists if the height has already been indexed.

func InsertEpochProtocolState added in v0.43.0 

func InsertEpochProtocolState(w storage.Writer, entryID flow.Identifier, entry *flow.MinEpochStateEntry) error

InsertEpochProtocolState inserts an epoch protocol state entry by ID. Error returns:

  • generic error in case of unexpected failure from the database layer or encoding failure.

func InsertEpochSetup added in v0.43.0 

func InsertEpochSetup(w storage.Writer, eventID flow.Identifier, event *flow.EpochSetup) error

func InsertEvent added in v0.39.4 

func InsertEvent(w storage.Writer, blockID flow.Identifier, event flow.Event) error

func InsertExecutionForkEvidence added in v0.42.1 

func InsertExecutionForkEvidence(w storage.Writer, conflictingSeals []*flow.IncorporatedResultSeal) error

InsertExecutionForkEvidence upserts conflicting seals to the database. If a record already exists, it is overwritten; otherwise a new record is created. No errors are expected during normal operations.

func InsertExecutionReceiptStub added in v0.43.0 

func InsertExecutionReceiptStub(w storage.Writer, receiptID flow.Identifier, meta *flow.ExecutionReceiptStub) error

InsertExecutionReceiptStub inserts a flow.ExecutionReceiptStub into the database, keyed by its ID.

CAUTION: The caller must ensure receiptID is a collision-resistant hash of the provided flow.ExecutionReceiptMeta! This method silently overrides existing data, which is safe only if for the same key, we always write the same value.

func InsertExecutionResult added in v0.39.4 

func InsertExecutionResult(w storage.Writer, result *flow.ExecutionResult) error

InsertExecutionResult inserts a flow.ExecutionResult into the storage, keyed by its ID.

If the result already exists, it will be overwritten. Note that here, the key (result ID) is derived from the value (result) via a collision-resistant hash function. Hence, unchecked overwrites pose no risk of data corruption, because for the same key, we expect the same value.

No errors are expected during normal operation.

func InsertExecutionStateInteractions added in v0.39.4 

func InsertExecutionStateInteractions(
	w storage.Writer,
	blockID flow.Identifier,
	executionSnapshots []*snapshot.ExecutionSnapshot,
) error

func InsertGuarantee added in v0.43.0 

func InsertGuarantee(w storage.Writer, guaranteeID flow.Identifier, guarantee *flow.CollectionGuarantee) error

InsertGuarantee inserts a collection guarantee by ID.

CAUTION: The caller must ensure guaranteeID is a collision-resistant hash of the provided guarantee! This method silently overrides existing data, which is safe only if for the same key, we always write the same value.

No errors expected during normal operations.

func InsertHeader added in v0.43.0 

func InsertHeader(lctx lockctx.Proof, rw storage.ReaderBatchWriter, headerID flow.Identifier, header *flow.Header) error

InsertHeader inserts a block header into the database.

CAUTION:

  • The caller must ensure that headerID is a collision-resistant hash of the provided header! Otherwise, data corruption may occur.
  • The caller must acquire one (but not both) of the following locks and hold it until the database write has been committed: either storage.LockInsertBlock or storage.LockInsertOrFinalizeClusterBlock.

It returns storage.ErrAlreadyExists if the header already exists, i.e. we only insert a new header once. This error allows the caller to detect duplicate inserts. If the header is stored along with other parts of the block in the same batch, similar duplication checks can be skipped for storing other parts of the block. No other errors are expected during normal operation.

func InsertInstanceParams added in v0.43.0 

func InsertInstanceParams(lctx lockctx.Proof, rw storage.ReaderBatchWriter, params flow.VersionedInstanceParams) error

InsertInstanceParams stores the consolidated instance params under a single key.

CAUTION:

  • This function is intended to be called exactly once during bootstrapping. Overwrites are prevented by an explicit existence check; if data is already present, error is returned.
  • To guarantee atomicity of existence-check plus database write, we require the caller to acquire the storage.LockBootstrapping lock and hold it until the database write has been committed.

Expected errors during normal operations:

func InsertJobAtIndex added in v0.39.1 

func InsertJobAtIndex(w storage.Writer, queue string, index uint64, entity flow.Identifier) error

InsertJobAtIndex insert an entity ID at the given index

func InsertLightTransactionResult added in v0.39.4 

func InsertLightTransactionResult(w storage.Writer, blockID flow.Identifier, transactionResult *flow.LightTransactionResult) error

deprecated

func InsertProposalSignature added in v0.43.0 

func InsertProposalSignature(w storage.Writer, blockID flow.Identifier, sig *[]byte) error

InsertProposalSignature inserts a proposal signature by block ID. Returns storage.ErrAlreadyExists if a proposal signature has already been inserted for the block.

func InsertProtocolKVStore added in v0.43.0 

func InsertProtocolKVStore(w storage.Writer, protocolKVStoreID flow.Identifier, kvStore *flow.PSKeyValueStoreData) error

InsertProtocolKVStore inserts a protocol KV store by ID. Error returns:

  • storage.ErrAlreadyExists if the key already exists in the database.
  • generic error in case of unexpected failure from the database layer or encoding failure.

func InsertQuorumCertificate added in v0.43.0 

func InsertQuorumCertificate(lctx lockctx.Proof, rw storage.ReaderBatchWriter, qc *flow.QuorumCertificate) error

InsertQuorumCertificate atomically performs the following storage operations for the given QuorumCertificate [QC]:

  1. Check if a QC certifying the same block is already stored.
  2. Only if no QC exists for the block, append the storage operations for indexing the QC by the block ID it certifies.

CAUTION:

  • For the same block, different QCs can easily be constructed by selecting different sub-sets of the received votes. In most cases, it is only important that a block has been certified, but it is irrelevant who specifically contributed to the QC. Therefore, we only store the first QC.
  • In order to make sure only one QC is stored per block, _all calls_ to `InsertQuorumCertificate` must be synchronized by the higher-logic. Currently, we have the lockctx.Proof to prove the higher logic is holding the storage.LockInsertBlock when inserting the QC after checking that no QC is already stored.

Expected error returns:

func InsertSeal added in v0.43.0 

func InsertSeal(w storage.Writer, sealID flow.Identifier, seal *flow.Seal) error

InsertSeal inserts a flow.Seal into the database, keyed by its ID.

CAUTION: The caller must ensure sealID is a collision-resistant hash of the provided seal! This method silently overrides existing data, which is safe only if for the same key, we always write the same value.

No errors are expected during normal operation.

func InsertServiceEvent added in v0.39.4 

func InsertServiceEvent(w storage.Writer, blockID flow.Identifier, event flow.Event) error

func InsertTransactionResult added in v0.39.4 

func InsertTransactionResult(w storage.Writer, blockID flow.Identifier, transactionResult *flow.TransactionResult) error

func Iterate 

func Iterate(startPrefix []byte, endPrefix []byte, check func(key []byte) error) func(storage.Reader) error

func IterateKeys 

func IterateKeys(r storage.Reader, startPrefix []byte, endPrefix []byte, iterFunc IterationFunc, opt storage.IteratorOption) (errToReturn error)

IterateKeys will iterate over all entries in the database, where the key starts with a prefixes in the range [startPrefix, endPrefix] (both inclusive). No errors expected during normal operations.

func IterateKeysByPrefixRange 

func IterateKeysByPrefixRange(r storage.Reader, startPrefix []byte, endPrefix []byte, check func(key []byte) error) error

IterateKeysByPrefixRange will iterate over all entries in the database, where the key starts with a prefixes in the range [startPrefix, endPrefix] (both inclusive). We require that startPrefix <= endPrefix (otherwise this function errors). On every such key, the `check` function is called. If `check` errors, iteration is aborted. In other words, error returned by the iteration functions will be propagated to the caller. No errors expected during normal operations.

func KeyExists 

func KeyExists(r storage.Reader, key []byte) (exist bool, errToReturn error)

KeyExists returns true if a key exists in the database. When this returned function is executed (and only then), it will write into the `keyExists` whether the key exists. No errors are expected during normal operation.

func LookupBlockContainingCollectionGuarantee added in v0.43.0 

func LookupBlockContainingCollectionGuarantee(r storage.Reader, collID flow.Identifier, blockID *flow.Identifier) error

LookupBlockContainingCollectionGuarantee retrieves the block containing the flow.CollectionGuarantee with the given ID.

CAUTION: A collection can be included in multiple *unfinalized* blocks. However, the implementation assumes a one-to-one map from collection ID to a *single* block ID. This holds for FINALIZED BLOCKS ONLY *and* only in the ABSENCE of BYZANTINE collector CLUSTERS (which the mature protocol must tolerate). Hence, this function should be treated as a temporary solution, which requires generalization (one-to-many mapping) for soft finality and the mature protocol.

Expected errors during normal operations:

func LookupBlockHeight added in v0.43.0 

func LookupBlockHeight(r storage.Reader, height uint64, blockID *flow.Identifier) error

LookupBlockHeight retrieves finalized blocks by height. Expected errors during normal operations:

func LookupBySealedBlockID added in v0.43.0 

func LookupBySealedBlockID(r storage.Reader, blockID flow.Identifier, sealID *flow.Identifier) error

LookupBySealedBlockID returns the finalized seal for the specified FINALIZED block ID. In order for a block to have a seal in a finalized block, it must itself be finalized. Hence, this function only works for finalized blocks. However, note that there might be finalized for which no seal exits (or the block containing the seal might not yet be finalized).

Expected errors during normal operations:

func LookupCertifiedBlockByView added in v0.43.0 

func LookupCertifiedBlockByView(r storage.Reader, view uint64, blockID *flow.Identifier) error

LookupCertifiedBlockByView retrieves the certified block by view. (Certified blocks are blocks that have received QC.) Expected errors during normal operations:

func LookupClusterBlockHeight added in v0.43.0 

func LookupClusterBlockHeight(r storage.Reader, clusterID flow.ChainID, height uint64, blockID *flow.Identifier) error

LookupClusterBlockHeight retrieves a block ID by height for the given cluster (only finalized cluster blocks are indexed by height to guarantee uniqueness).

func LookupClusterBlocksByReferenceHeightRange added in v0.43.0 

func LookupClusterBlocksByReferenceHeightRange(lctx lockctx.Proof, r storage.Reader, start, end uint64, clusterBlockIDs *[]flow.Identifier) error

LookupClusterBlocksByReferenceHeightRange traverses the ref_height->cluster_block index and returns any finalized cluster blocks which have a reference block with height in the given range. This is used to avoid including duplicate transaction when building or validating a new collection.

func LookupCollectionByTransaction added in v0.41.0 

func LookupCollectionByTransaction(r storage.Reader, txID flow.Identifier, collectionID *flow.Identifier) error

LookupCollectionByTransaction retrieves the collection ID for the collection that contains the specified transaction. For every known transaction, this index should be populated.

WARNING, this index is NOT BFT in its current form: Honest clusters ensure a transaction can only belong to one collection. However, in rare cases, the collector clusters can exceed byzantine thresholds -- making it possible to produce multiple finalized collections (aka guaranteed collections) containing the same transaction repeatedly.

Expected errors during normal operations:

func LookupCollectionPayload added in v0.39.4 

func LookupCollectionPayload(r storage.Reader, clusterBlockID flow.Identifier, txIDs *[]flow.Identifier) error

LookupCollectionPayload retrieves the list of transaction IDs that constitute the payload of the specified cluster block. For every known cluster block, this index should be populated.

Expected errors during normal operations:

func LookupEpochProtocolState added in v0.43.0 

func LookupEpochProtocolState(r storage.Reader, blockID flow.Identifier, epochProtocolStateEntryID *flow.Identifier) error

LookupEpochProtocolState finds an epoch protocol state entry ID by block ID. Error returns:

  • storage.ErrNotFound if the key does not exist in the database
  • generic error in case of unexpected failure from the database layer

func LookupEventsByBlockID added in v0.39.4 

func LookupEventsByBlockID(r storage.Reader, blockID flow.Identifier, events *[]flow.Event) error

func LookupEventsByBlockIDEventType added in v0.39.4 

func LookupEventsByBlockIDEventType(r storage.Reader, blockID flow.Identifier, eventType flow.EventType, events *[]flow.Event) error

func LookupExecutionReceipts added in v0.39.4 

func LookupExecutionReceipts(r storage.Reader, blockID flow.Identifier, receiptIDs *[]flow.Identifier) error

LookupExecutionReceipts retrieves the set of all execution receipts for the specified block. For every known block (at or above the root block height), this index should be populated with all known receipts for that block.

Expected errors during normal operations:

func LookupExecutionResult added in v0.39.4 

func LookupExecutionResult(r storage.Reader, blockID flow.Identifier, resultID *flow.Identifier) error

LookupExecutionResult retrieves the Execution Node's OWN Execution Result ID for the specified block. Intended for Execution Node only. For every block executed by this node, this index should be populated.

Expected errors during normal operations:

func LookupGuarantee added in v0.43.0 

func LookupGuarantee(r storage.Reader, collectionID flow.Identifier, guaranteeID *flow.Identifier) error

LookupGuarantee finds collection guarantee ID by collection ID. Error returns:

  • storage.ErrNotFound if the key does not exist in the database
  • All other errors have to be treated as unexpected failures from the database layer.

func LookupLastVersionBeaconByHeight added in v0.39.2 

func LookupLastVersionBeaconByHeight(
	r storage.Reader,
	maxHeight uint64,
	versionBeacon *flow.SealedVersionBeacon,
) error

Returns storage.ErrNotFound if no version beacon exists at or below the given height.

func LookupLatestSealAtBlock added in v0.43.0 

func LookupLatestSealAtBlock(r storage.Reader, blockID flow.Identifier, sealID *flow.Identifier) error

LookupLatestSealAtBlock finds the highest seal that was included in the fork up to (and including) blockID. Frequently, the highest seal included in this block's payload. However, if there are no seals in this block, sealID should reference the highest seal in blockID's ancestors.

Expected errors during normal operations:

func LookupLightTransactionResultsByBlockIDUsingIndex added in v0.39.4 

func LookupLightTransactionResultsByBlockIDUsingIndex(r storage.Reader, blockID flow.Identifier, txResults *[]flow.LightTransactionResult) error

LookupLightTransactionResultsByBlockIDUsingIndex retrieves all tx results for a block, but using tx_index index. This correctly handles cases of duplicate transactions within block.

func LookupOwnExecutionReceipt added in v0.39.4 

func LookupOwnExecutionReceipt(r storage.Reader, blockID flow.Identifier, receiptID *flow.Identifier) error

LookupOwnExecutionReceipt retrieves the Execution Node's OWN execution receipt ID for the specified block. Intended for Execution Node only. For every block executed by this node, this index should be populated.

Expected errors during normal operations:

func LookupPayloadGuarantees added in v0.43.0 

func LookupPayloadGuarantees(r storage.Reader, blockID flow.Identifier, guarIDs *[]flow.Identifier) error

LookupPayloadGuarantees retrieves the list of guarantee IDs that were included in the payload of the specified block. For every known block (at or above the root block height), this index should be populated.

Expected errors during normal operations:

  • storage.ErrNotFound if `blockID` does not refer to a known block
  • All other errors have to be treated as unexpected failures from the database layer.

func LookupPayloadProtocolStateID added in v0.43.0 

func LookupPayloadProtocolStateID(r storage.Reader, blockID flow.Identifier, stateID *flow.Identifier) error

LookupPayloadProtocolStateID retrieves the Protocol State ID for the specified block. The Protocol State ID represents the configuration, which the block proposes to become active *after* the block's certification. For every known block (at or above the root block height), the protocol state at the end of the block should be specified in the payload, and hence be indexed. Expected errors during normal operations:

func LookupPayloadReceipts added in v0.43.0 

func LookupPayloadReceipts(r storage.Reader, blockID flow.Identifier, receiptIDs *[]flow.Identifier) error

LookupPayloadReceipts retrieves the list of Execution Receipts that were included in the payload of the specified block. For every known block (at or above the root block height), this index should be populated. Expected errors during normal operations:

func LookupPayloadResults added in v0.43.0 

func LookupPayloadResults(r storage.Reader, blockID flow.Identifier, resultIDs *[]flow.Identifier) error

LookupPayloadResults retrieves the list of Execution Results that were included in the payload of the specified block. For every known block (at or above the root block height), this index should be populated. Expected errors during normal operations:

func LookupPayloadSeals added in v0.43.0 

func LookupPayloadSeals(r storage.Reader, blockID flow.Identifier, sealIDs *[]flow.Identifier) error

LookupPayloadSeals retrieves the list of Seals that were included in the payload of the specified block. For every known block (at or above the root block height), this index should be populated.

Expected errors during normal operations:

func LookupProtocolKVStore added in v0.43.0 

func LookupProtocolKVStore(r storage.Reader, blockID flow.Identifier, protocolKVStoreID *flow.Identifier) error

LookupProtocolKVStore finds protocol KV store ID by block ID. Error returns:

  • storage.ErrNotFound if the key does not exist in the database
  • generic error in case of unexpected failure from the database layer

func LookupReferenceBlockByClusterBlock added in v0.43.0 

func LookupReferenceBlockByClusterBlock(r storage.Reader, clusterBlockID flow.Identifier, refID *flow.Identifier) error

LookupReferenceBlockByClusterBlock looks up the reference block ID for the given cluster block ID. While each cluster block specifies a reference block in its payload, we maintain this additional lookup for performance reasons.

func LookupResultApproval 

func LookupResultApproval(r storage.Reader, resultID flow.Identifier, chunkIndex uint64, approvalID *flow.Identifier) error

LookupResultApproval finds a ResultApproval by result ID and chunk index. Returns `storage.ErrNotFound` if no Approval for the given key (resultID, chunkIndex) has been stored.

NOTE that the Flow protocol requires multiple approvals for the same chunk from different verification nodes. In other words, there are multiple different approvals for the same chunk. Therefore, the index Executed Chunk ➜ ResultApproval ID (queried here) is *only safe* to be used by Verification Nodes for tracking their own approvals (for the same ExecutionResult, a Verifier will always produce the same approval)

func LookupServiceEventsByBlockID added in v0.39.4 

func LookupServiceEventsByBlockID(r storage.Reader, blockID flow.Identifier, events *[]flow.Event) error

func LookupStateCommitment added in v0.39.4 

func LookupStateCommitment(r storage.Reader, blockID flow.Identifier, commit *flow.StateCommitment) error

LookupStateCommitment retrieves a state commitment by the block ID whose execution results in that state. Expected error returns during normal operations:

func LookupTransactionResultErrorMessagesByBlockIDUsingIndex added in v0.39.4 

func LookupTransactionResultErrorMessagesByBlockIDUsingIndex(r storage.Reader, blockID flow.Identifier, txResultErrorMessages *[]flow.TransactionResultErrorMessage) error

LookupTransactionResultErrorMessagesByBlockIDUsingIndex retrieves all tx result error messages for a block, by using tx_index index. This correctly handles cases of duplicate transactions within block.

func LookupTransactionResultsByBlockIDUsingIndex added in v0.39.4 

func LookupTransactionResultsByBlockIDUsingIndex(r storage.Reader, blockID flow.Identifier, txResults *[]flow.TransactionResult) error

LookupTransactionResultsByBlockIDUsingIndex retrieves all tx results for a block, by using tx_index index. This correctly handles cases of duplicate transactions within block.

func MakePrefix 

func MakePrefix(code byte, keys ...any) []byte

func NewMultiDBStore added in v0.41.0 

func NewMultiDBStore(rwStore storage.DB, rStore storage.DB) storage.DB

NewMultiDBStore returns a DB store that consists of a primary read-and-write store, and a secondary read-only store.

func NewMultiIterator added in v0.41.0 

func NewMultiIterator(iterators ...storage.Iterator) (storage.Iterator, error)

NewMultiIterator returns an Iterator that is a logical concatenation of multiple iterators in the provided sequence. The returned iterator iterates items in the first iterator, and then iterates items in the second iterator, etc. NewMultiIterator panics if 0 iterators are provided.

func NewMultiReader added in v0.41.0 

func NewMultiReader(readers ...storage.Reader) storage.Reader

NewMultiReader returns a Reader that consists of multiple readers in the provided order. Readers are read sequentially until - a reader succeeds or - a reader returns an error that is not ErrNotFound If all readers return ErrNotFound, Reader.Get will return ErrNotFound. NewMultiReader panics if 0 readers are provided.

func NewMultiSeeker added in v0.41.0 

func NewMultiSeeker(seekers ...storage.Seeker) storage.Seeker

NewMultiSeeker returns a Seeker that consists of multiple seekers in the provided order. NewMultiSeeker panics if 0 seekers are provided.

func PersistNodeDisallowList added in v0.41.0 

func PersistNodeDisallowList(w storage.Writer, disallowList map[flow.Identifier]struct{}) error

PersistNodeDisallowList writes the set of disallowed nodes IDs into the database. If an entry already exists, it is overwritten; otherwise a new entry is created. No errors are expected during normal operations.

TODO: TEMPORARY manual override for adding node IDs to list of ejected nodes, applies to networking layer only

func PrintStats added in v0.40.0 

func PrintStats(log zerolog.Logger, stats map[byte]Stats)

PrintStats logs the statistics for each prefix in ascending order. Each prefix is shown in hex, along with count, min, max, total, and average sizes.

func PurgeNodeDisallowList added in v0.41.0 

func PurgeNodeDisallowList(w storage.Writer) error

PurgeNodeDisallowList removes the set of disallowed nodes IDs from the database. If no corresponding entry exists, this function is a no-op. No errors are expected during normal operations.

TODO: TEMPORARY manual override for adding node IDs to list of ejected nodes, applies to networking layer only

func Remove 

func Remove(key []byte) func(storage.Writer) error

func RemoveByKey 

func RemoveByKey(w storage.Writer, key []byte) error

RemoveByKey removes the entity with the given key, if it exists. If it doesn't exist, this is a no-op. Error returns: * generic error in case of unexpected database error

func RemoveByKeyPrefix 

func RemoveByKeyPrefix(reader storage.Reader, w storage.Writer, prefix []byte) error

RemoveByKeyPrefix removes all keys with the given prefix Error returns: * generic error in case of unexpected database error

func RemoveByKeyRange 

func RemoveByKeyRange(reader storage.Reader, w storage.Writer, startPrefix []byte, endPrefix []byte) error

RemoveByKeyRange removes all keys with a prefix that falls within the range [start, end], both inclusive. It returns error if endPrefix < startPrefix no other errors are expected during normal operation

func RemoveByPrefix 

func RemoveByPrefix(reader storage.Reader, key []byte) func(storage.Writer) error

func RemoveByRange 

func RemoveByRange(reader storage.Reader, startPrefix []byte, endPrefix []byte) func(storage.Writer) error

func RemoveChunkDataPack 

func RemoveChunkDataPack(w storage.Writer, chunkID flow.Identifier) error

RemoveChunkDataPack removes the chunk data pack with the given chunk ID. any error are exceptions

func RemoveCollection added in v0.39.4 

func RemoveCollection(w storage.Writer, collID flow.Identifier) error

RemoveCollection removes a collection from the storage. CAUTION: this is for recovery purposes only, and should not be used during normal operations! It returns nil if the collection does not exist. No errors are expected during normal operation.

func RemoveCollectionPayloadIndices added in v0.41.0 

func RemoveCollectionPayloadIndices(w storage.Writer, collID flow.Identifier) error

RemoveCollectionPayloadIndices removes a collection id indexed by a block id. CAUTION: this is for recovery purposes only, and should not be used during normal operations! It returns nil if the collection does not exist. No errors are expected during normal operation.

func RemoveCollectionTransactionIndices added in v0.41.0 

func RemoveCollectionTransactionIndices(w storage.Writer, txID flow.Identifier) error

RemoveCollectionByTransactionIndex removes an entry in the index from transaction ID to collection containing the transaction. CAUTION: this is for recovery purposes only, and should not be used during normal operations! It returns nil if the collection does not exist. No errors are expected during normal operation.

func RemoveComputationResultUploadStatus added in v0.39.2 

func RemoveComputationResultUploadStatus(
	w storage.Writer,
	blockID flow.Identifier) error

RemoveComputationResult removes an instance of ComputationResult with given ID.

func RemoveEventsByBlockID added in v0.39.4 

func RemoveEventsByBlockID(r storage.Reader, w storage.Writer, blockID flow.Identifier) error

func RemoveExecutionForkEvidence added in v0.43.0 

func RemoveExecutionForkEvidence(w storage.Writer) error

RemoveExecutionForkEvidence deletes conflicting seals record from the database. No errors are expected during normal operations.

func RemoveExecutionResultIndex added in v0.39.4 

func RemoveExecutionResultIndex(w storage.Writer, blockID flow.Identifier) error

RemoveExecutionResultIndex removes Execution Node's OWN Execution Result for the given blockID. CAUTION: this is for recovery purposes only, and should not be used during normal operations It returns nil if the collection does not exist. No errors are expected during normal operation.

func RemoveOwnExecutionReceipt added in v0.39.4 

func RemoveOwnExecutionReceipt(w storage.Writer, blockID flow.Identifier) error

RemoveOwnExecutionReceipt removes the Execution Node's OWN execution receipt index for the given block ID. CAUTION: this is for recovery purposes only, and should not be used during normal operations! It returns nil if the collection does not exist.

No errors are expected during normal operation.

func RemoveServiceEventsByBlockID added in v0.39.4 

func RemoveServiceEventsByBlockID(r storage.Reader, w storage.Writer, blockID flow.Identifier) error

func RemoveStateCommitment added in v0.39.4 

func RemoveStateCommitment(w storage.Writer, blockID flow.Identifier) error

RemoveStateCommitment removes the state commitment by block ID CAUTION: this is for recovery purposes only, and should not be used during normal operations! It returns nil if no execution result for the given blockID was previously indexed. No errors are expected during normal operation.

func RemoveTransaction added in v0.41.0 

func RemoveTransaction(r storage.Writer, txID flow.Identifier) error

RemoveTransaction removes a transaction by ID.

func RemoveTransactionResultsByBlockID added in v0.39.4 

func RemoveTransactionResultsByBlockID(r storage.Reader, w storage.Writer, blockID flow.Identifier) error

RemoveTransactionResultsByBlockID removes the transaction results for the given blockID

func Retrieve 

func Retrieve(key []byte, entity interface{}) func(storage.Reader) error

func RetrieveBlockChildren added in v0.43.0 

func RetrieveBlockChildren(r storage.Reader, blockID flow.Identifier, childrenIDs *flow.IdentifierList) error

RetrieveBlockChildren retrieves the list of child block IDs for the specified parent block. For every known block (at or above the root block height), this index should be populated.

Expected errors during normal operations:

func RetrieveByKey 

func RetrieveByKey(r storage.Reader, key []byte, entity any) (errToReturn error)

RetrieveByKey will retrieve the binary data under the given key from the database and decode it into the given entity. The provided entity needs to be a pointer to an initialized entity of the correct type. Error returns:

  • storage.ErrNotFound if the key does not exist in the database
  • generic error in case of unexpected failure from the database layer, or failure to decode an existing database value

func RetrieveChunkDataPack 

func RetrieveChunkDataPack(r storage.Reader, chunkID flow.Identifier, c *storage.StoredChunkDataPack) error

RetrieveChunkDataPack retrieves a chunk data pack by chunk ID. it returns storage.ErrNotFound if the chunk data pack is not found

func RetrieveChunkLocator added in v0.39.1 

func RetrieveChunkLocator(r storage.Reader, locatorID flow.Identifier, locator *chunks.Locator) error

func RetrieveClusterFinalizedHeight added in v0.43.0 

func RetrieveClusterFinalizedHeight(r storage.Reader, clusterID flow.ChainID, height *uint64) error

RetrieveClusterFinalizedHeight retrieves the latest finalized cluster block height of the given cluster.

func RetrieveCollection added in v0.39.4 

func RetrieveCollection(r storage.Reader, collID flow.Identifier, collection *flow.LightCollection) error

RetrieveCollection retrieves a flow.LightCollection by its ID.

Expected errors during normal operations:

func RetrieveEpochCommit added in v0.43.0 

func RetrieveEpochCommit(r storage.Reader, eventID flow.Identifier, event *flow.EpochCommit) error

func RetrieveEpochFirstHeight added in v0.43.0 

func RetrieveEpochFirstHeight(r storage.Reader, epoch uint64, height *uint64) error

RetrieveEpochFirstHeight retrieves the height of the first block in the given epoch. This operation does not require any locks, because the first height of an epoch does not change once set. Returns storage.ErrNotFound if the first block of the epoch has not yet been finalized.

func RetrieveEpochLastHeight added in v0.43.0 

func RetrieveEpochLastHeight(r storage.Reader, epoch uint64, height *uint64) error

RetrieveEpochLastHeight retrieves the height of the last block in the given epoch. This operation does not require any locks, because the first height of an epoch does not change once set. It's a more readable, but equivalent query to RetrieveEpochFirstHeight when interested in the last height of an epoch. Returns storage.ErrNotFound if the first block of the epoch has not yet been finalized.

func RetrieveEpochProtocolState added in v0.43.0 

func RetrieveEpochProtocolState(r storage.Reader, entryID flow.Identifier, entry *flow.MinEpochStateEntry) error

RetrieveEpochProtocolState retrieves an epoch protocol state entry by ID. Error returns:

  • storage.ErrNotFound if the key does not exist in the database
  • generic error in case of unexpected failure from the database layer

func RetrieveEpochSetup added in v0.43.0 

func RetrieveEpochSetup(r storage.Reader, eventID flow.Identifier, event *flow.EpochSetup) error

func RetrieveEvents added in v0.39.4 

func RetrieveEvents(r storage.Reader, blockID flow.Identifier, transactionID flow.Identifier, events *[]flow.Event) error

func RetrieveExecutedBlock added in v0.39.4 

func RetrieveExecutedBlock(r storage.Reader, blockID *flow.Identifier) error

func RetrieveExecutionForkEvidence added in v0.42.1 

func RetrieveExecutionForkEvidence(r storage.Reader, conflictingSeals *[]*flow.IncorporatedResultSeal) error

RetrieveExecutionForkEvidence reads conflicting seals from the database. It returns `storage.ErrNotFound` error if no database record is present.

func RetrieveExecutionReceiptStub added in v0.43.0 

func RetrieveExecutionReceiptStub(r storage.Reader, receiptID flow.Identifier, meta *flow.ExecutionReceiptStub) error

RetrieveExecutionReceiptStub retrieves a flow.ExecutionReceiptStub by its ID.

Expected errors during normal operations:

func RetrieveExecutionResult added in v0.39.4 

func RetrieveExecutionResult(r storage.Reader, resultID flow.Identifier, result *flow.ExecutionResult) error

RetrieveExecutionResult retrieves an Execution Result by its ID. Expected errors during normal operations:

func RetrieveExecutionStateInteractions added in v0.39.4 

func RetrieveExecutionStateInteractions(
	r storage.Reader,
	blockID flow.Identifier,
	executionSnapshots *[]*snapshot.ExecutionSnapshot,
) error

func RetrieveFinalizedHeight added in v0.43.0 

func RetrieveFinalizedHeight(r storage.Reader, height *uint64) error

func RetrieveGuarantee added in v0.43.0 

func RetrieveGuarantee(r storage.Reader, collID flow.Identifier, guarantee *flow.CollectionGuarantee) error

RetrieveGuarantee retrieves a flow.CollectionGuarantee by the collection ID. For every collection that has been guaranteed, this data should be populated.

Expected errors during normal operations:

  • storage.ErrNotFound if `collID` does not refer to a known guaranteed collection
  • All other errors have to be treated as unexpected failures from the database layer.

func RetrieveHeader added in v0.43.0 

func RetrieveHeader(r storage.Reader, blockID flow.Identifier, header *flow.Header) error

RetrieveHeader retrieves the header of the block with the specified ID. Expected errors during normal operations:

  • storage.ErrNotFound if no block with the specified `blockID` is known.
  • generic error in case of unexpected failure from the database layer

func RetrieveInstanceParams added in v0.43.0 

func RetrieveInstanceParams(r storage.Reader, params *flow.VersionedInstanceParams) error

RetrieveInstanceParams retrieves the consolidated instance params from storage.

Expected errors during normal operations:

func RetrieveJobAtIndex added in v0.39.1 

func RetrieveJobAtIndex(r storage.Reader, queue string, index uint64, entity *flow.Identifier) error

RetrieveJobAtIndex returns the entity at the given index

func RetrieveJobLatestIndex added in v0.39.1 

func RetrieveJobLatestIndex(r storage.Reader, queue string, index *uint64) error

func RetrieveLightTransactionResult added in v0.39.4 

func RetrieveLightTransactionResult(r storage.Reader, blockID flow.Identifier, transactionID flow.Identifier, transactionResult *flow.LightTransactionResult) error

func RetrieveLightTransactionResultByIndex added in v0.39.4 

func RetrieveLightTransactionResultByIndex(r storage.Reader, blockID flow.Identifier, txIndex uint32, transactionResult *flow.LightTransactionResult) error

func RetrieveLivenessData added in v0.42.0 

func RetrieveLivenessData(r storage.Reader, chainID flow.ChainID, livenessData *hotstuff.LivenessData) error

RetrieveSafetyData retrieves the safety data for this node. Intended for consensus participants only (consensus and collector nodes). Here, `chainID` specifies which consensus instance specifically the node participates in. For consensus and collector nodes, this value should always exist (for the correct chainID). No errors are expected during normal operation.

func RetrieveNodeDisallowList added in v0.41.0 

func RetrieveNodeDisallowList(r storage.Reader, disallowList *map[flow.Identifier]struct{}) error

RetrieveNodeDisallowList reads the set of disallowed node IDs from the database. Returns `storage.ErrNotFound` error in case no respective database entry is present.

TODO: TEMPORARY manual override for adding node IDs to list of ejected nodes, applies to networking layer only

func RetrieveProcessedIndex 

func RetrieveProcessedIndex(r storage.Reader, jobName string, processed *uint64) error

RetrieveProcessedIndex returns the processed index for a job consumer

func RetrieveProposalSignature added in v0.43.0 

func RetrieveProposalSignature(r storage.Reader, blockID flow.Identifier, sig *[]byte) error

RetrieveProposalSignature retrieves a proposal signature by blockID. Returns storage.ErrNotFound if no proposal signature is stored for the block.

func RetrieveProtocolKVStore added in v0.43.0 

func RetrieveProtocolKVStore(r storage.Reader, protocolKVStoreID flow.Identifier, kvStore *flow.PSKeyValueStoreData) error

RetrieveProtocolKVStore retrieves a protocol KV store by ID. Error returns:

  • storage.ErrNotFound if the key does not exist in the database
  • generic error in case of unexpected failure from the database layer

func RetrieveQuorumCertificate added in v0.43.0 

func RetrieveQuorumCertificate(r storage.Reader, blockID flow.Identifier, qc *flow.QuorumCertificate) error

RetrieveQuorumCertificate retrieves the QuorumCertificate for the specified block. For every block that has been certified, this index should be populated.

Expected errors during normal operations:

func RetrieveResultApproval 

func RetrieveResultApproval(r storage.Reader, approvalID flow.Identifier, approval *flow.ResultApproval) error

RetrieveResultApproval retrieves an approval by ID. Returns `storage.ErrNotFound` if no Approval with the given ID has been stored.

func RetrieveSafetyData added in v0.42.0 

func RetrieveSafetyData(r storage.Reader, chainID flow.ChainID, safetyData *hotstuff.SafetyData) error

RetrieveSafetyData retrieves the safety data for this node. Intended for consensus participants only (consensus and collector nodes). Here, `chainID` specifies which consensus instance specifically the node participates in. For consensus and collector nodes, this value should always exist (for the correct chainID). No errors are expected during normal operation.

func RetrieveSeal added in v0.43.0 

func RetrieveSeal(r storage.Reader, sealID flow.Identifier, seal *flow.Seal) error

RetrieveSeal retrieves the flow.Seal by its ID. Expected errors during normal operations:

func RetrieveSealedHeight added in v0.43.0 

func RetrieveSealedHeight(r storage.Reader, height *uint64) error

func RetrieveTransaction added in v0.39.4 

func RetrieveTransaction(r storage.Reader, txID flow.Identifier, tx *flow.TransactionBody) error

RetrieveTransaction retrieves a transaction by fingerprint.

func RetrieveTransactionResult added in v0.39.4 

func RetrieveTransactionResult(r storage.Reader, blockID flow.Identifier, transactionID flow.Identifier, transactionResult *flow.TransactionResult) error

func RetrieveTransactionResultByIndex added in v0.39.4 

func RetrieveTransactionResultByIndex(r storage.Reader, blockID flow.Identifier, txIndex uint32, transactionResult *flow.TransactionResult) error

func RetrieveTransactionResultErrorMessage added in v0.39.4 

func RetrieveTransactionResultErrorMessage(r storage.Reader, blockID flow.Identifier, transactionID flow.Identifier, transactionResultErrorMessage *flow.TransactionResultErrorMessage) error

RetrieveTransactionResultErrorMessage retrieves a transaction result error message by block ID and transaction ID.

func RetrieveTransactionResultErrorMessageByIndex added in v0.39.4 

func RetrieveTransactionResultErrorMessageByIndex(r storage.Reader, blockID flow.Identifier, txIndex uint32, transactionResultErrorMessage *flow.TransactionResultErrorMessage) error

RetrieveTransactionResultErrorMessageByIndex retrieves a transaction result error message by block ID and index.

func SetJobLatestIndex added in v0.39.1 

func SetJobLatestIndex(w storage.Writer, queue string, index uint64) error

func SetProcessedIndex 

func SetProcessedIndex(w storage.Writer, jobName string, processed uint64) error

SetProcessedIndex updates the processed index for a job consumer with given index

func SummarizeKeysByFirstByteConcurrent added in v0.40.0 

func SummarizeKeysByFirstByteConcurrent(log zerolog.Logger, r storage.Reader, nWorker int) (map[byte]Stats, error)

SummarizeKeysByFirstByteConcurrent iterates over all prefixes [0x00..0xFF] in parallel using nWorker goroutines. Each worker handles one prefix at a time until all are processed.

The storage.Reader must be able to create multiple iterators concurrently.

func TransactionResultErrorMessagesExists added in v0.39.4 

func TransactionResultErrorMessagesExists(r storage.Reader, blockID flow.Identifier, blockExists *bool) error

TransactionResultErrorMessagesExists checks whether tx result error messages exist in the database.

func Traverse 

func Traverse(prefix []byte, iterFunc IterationFunc, opt storage.IteratorOption) func(storage.Reader) error

func TraverseByPrefix 

func TraverseByPrefix(r storage.Reader, prefix []byte, iterFunc IterationFunc, opt storage.IteratorOption) error

TraverseByPrefix will iterate over all keys with the given prefix error returned by the iteration functions will be propagated to the caller. No other errors are expected during normal operation.

func UpdateExecutedBlock added in v0.39.4 

func UpdateExecutedBlock(w storage.Writer, blockID flow.Identifier) error

UpdateExecutedBlock updates the pointer to the Execution Node's OWN highest executed block. We overwrite the block ID of the most recently executed block, regardless of whether this block may later be orphaned or is already orphaned.

## Usage Context

  • The stored "last executed block" may reference a block on a fork that is later orphaned.
  • This is acceptable and expected: the index is intended for reporting execution metrics and for optimizing the loading of unexecuted blocks on node startup.
  • On startup, the Execution Node may use the latest executed block as a hint on where to restart the execution. It MUST traverse from the last executed in the direction of decreasing height. It will eventually reach a block with a finalized seal. From this block, the Execution Node should restart its execution and cover _all_ descendants (that are not orphaned). Thereby, we guarantee that even if the stored block is on a fork, we eventually also cover blocks are finalized or and the most recent still unfinalized blocks.
  • If the block referenced as "highest executed block" is not on the canonical chain, the Execution Node may (re-)execute some blocks unnecessarily, but this does not affect correctness.

## Limitations & Edge Cases - The value is not guaranteed to be on the finalized chain. - Forks of arbitrary length may occur; the stored block may be on any such fork.

## Correct Usage - Use for metrics (e.g., reporting latest executed block height). - Use for optimizing block execution on startup (as a performance hint).

## Incorrect Usage - Do not use as a source of truth for canonical chain state. - Do not disregard blocks with lower heights as not needing execution.

See project documentation in `engine/execution/ingestion/loader/unexecuted_loader.go` for details on startup traversal logic.

func Upsert 

func Upsert(key []byte, val interface{}) func(storage.Writer) error

func UpsertBlockChildren added in v0.43.0 

func UpsertBlockChildren(lctx lockctx.Proof, w storage.Writer, blockID flow.Identifier, childrenIDs flow.IdentifierList) error

UpsertBlockChildren updates the children of the specified parent block ID.

CAUTION:

  • The caller must acquire either the lock storage.LockInsertBlock or storage.LockInsertOrFinalizeClusterBlock (but not both) and hold it until the database write has been committed.
  • OVERWRITES existing data (potential for data corruption): This method silently overrides existing data without any sanity checks whether data for the same key already exits. Note that the Flow protocol mandates that for a previously persisted key, the data is never changed to a different value. Changing data could cause the node to publish inconsistent data and to be slashed, or the protocol to be compromised as a whole. This method does not contain any safeguards to prevent such data corruption. The lock proof serves as a reminder that the CALLER is responsible to ensure that the DEDUPLICATION CHECK is done elsewhere ATOMICALLY with this write operation.

No errors are expected during normal operation.

func UpsertByKey 

func UpsertByKey(w storage.Writer, key []byte, val interface{}) error

UpsertByKey will encode the given entity using msgpack and will insert the resulting binary data under the provided key. If the key already exists, the value will be overwritten. Error returns:

  • generic error in case of unexpected failure from the database layer or encoding failure.

func UpsertClusterFinalizedHeight added in v0.43.0 

func UpsertClusterFinalizedHeight(lctx lockctx.Proof, w storage.Writer, clusterID flow.ChainID, number uint64) error

UpsertClusterFinalizedHeight updates (overwrites!) the latest finalized cluster block height for the given cluster.

func UpsertCollection added in v0.39.4 

func UpsertCollection(w storage.Writer, collection *flow.LightCollection) error

UpsertCollection inserts a flow.LightCollection into the storage, keyed by its ID.

If the collection already exists, it will be overwritten. Note that here, the key (collection ID) is derived from the value (collection) via a collision-resistant hash function. Hence, unchecked overwrites pose no risk of data corruption, because for the same key, we expect the same value.

No errors are expected during normal operation.

func UpsertComputationResultUploadStatus added in v0.39.2 

func UpsertComputationResultUploadStatus(w storage.Writer, blockID flow.Identifier,
	wasUploadCompleted bool) error

UpsertComputationResult upserts given existing instance of ComputationResult in local BadgerDB.

func UpsertFinalizedHeight added in v0.43.0 

func UpsertFinalizedHeight(lctx lockctx.Proof, w storage.Writer, height uint64) error

UpsertFinalizedHeight upserts the finalized height index, overwriting the current value. Updates to this index must strictly increase the finalized height. To enforce this, the caller must check the current finalized height while holding storage.LockFinalizeBlock.

func UpsertLivenessData added in v0.42.0 

func UpsertLivenessData(w storage.Writer, chainID flow.ChainID, livenessData *hotstuff.LivenessData) error

UpsertLivenessData inserts or updates the given liveness data for this node. Intended for consensus participants only (consensus and collector nodes). Here, `chainID` specifies which consensus instance specifically the node participates in. CAUTION: OVERWRITES existing data (potential for data corruption).

No errors are expected during normal operation.

func UpsertSafetyData added in v0.42.0 

func UpsertSafetyData(w storage.Writer, chainID flow.ChainID, safetyData *hotstuff.SafetyData) error

UpsertSafetyData inserts or updates the given safety data for this node. Intended for consensus participants only (consensus and collector nodes). Here, `chainID` specifies which consensus instance specifically the node participates in. CAUTION: OVERWRITES existing data (potential for data corruption).

No errors are expected during normal operation.

func UpsertSealedHeight added in v0.43.0 

func UpsertSealedHeight(lctx lockctx.Proof, w storage.Writer, height uint64) error

UpsertSealedHeight upserts the latest sealed height, OVERWRITING the current value. Updates to this index must strictly increase the sealed height. To enforce this, the caller must check the current sealed height while holding storage.LockFinalizeBlock.

func UpsertTransaction added in v0.39.4 

func UpsertTransaction(w storage.Writer, txID flow.Identifier, tx *flow.TransactionBody) error

UpsertTransaction inserts a transaction keyed by transaction fingerprint. It overwrites any existing transaction, which is ok because tx is unique by its ID

func Upserting added in v0.43.0 

func Upserting(key []byte, val interface{}) func(storage.Writer) error

Upserting returns a functor, whose execution will append the given key-value-pair to the provided storage writer (typically a pending batch of database writes).

Types

type Callbacks 

type Callbacks struct {
	// contains filtered or unexported fields
}

Callbacks represents a collection of callbacks to be executed. Callbacks are not concurrent safe. Since Callbacks is only used in ReaderBatchWriter, which isn't concurrent safe, there isn't a need to add locking overhead to Callbacks.

func NewCallbacks added in v0.41.0 

func NewCallbacks() *Callbacks

func (*Callbacks) AddCallback 

func (b *Callbacks) AddCallback(callback func(error))

func (*Callbacks) NotifyCallbacks 

func (b *Callbacks) NotifyCallbacks(err error)

type IterationFunc 

type IterationFunc func(keyCopy []byte, getValue func(destVal any) error) (bail bool, err error)

IterationFunc is a callback function that will be called on each key-value pair during the iteration. The key is copied and passed to the function, so key can be modified or retained after iteration. The `getValue` function can be called to retrieve the value of the current key and decode value into destVal object. The caller can return (true, nil) to stop the iteration early.

func KeyOnlyIterateFunc 

func KeyOnlyIterateFunc(fn func(key []byte) error) IterationFunc

KeyOnlyIterateFunc returns an IterationFunc that only iterates over keys

type Stats added in v0.40.0 

type Stats struct {
	Count       int     `json:"count"`
	MinSize     int     `json:"min_size"`
	MaxSize     int     `json:"max_size"`
	TotalSize   int     `json:"total_size"`
	AverageSize float64 `json:"avg_size"`
}

Stats holds statistics for a single prefix group.

Source Files

View all Source files

Directories

Path Synopsis

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.