Documentation ¶
Index ¶
- Variables
- type ArchiveLedgerReader
- type Change
- type DBLedgerReader
- func (dblrc *DBLedgerReader) Close() error
- func (dblrc *DBLedgerReader) GetHeader() xdr.LedgerHeaderHistoryEntry
- func (dblrc *DBLedgerReader) GetSequence() uint32
- func (dblrc *DBLedgerReader) GetUpgradeChanges() []Change
- func (dblrc *DBLedgerReader) IgnoreUpgradeChanges()
- func (dblrc *DBLedgerReader) Read() (LedgerTransaction, error)
- func (dblrc *DBLedgerReader) ReadUpgradeChange() (Change, error)
- type LedgerEntryChangeCache
- type LedgerReader
- type LedgerTransaction
- type LedgerWriter
- type MemoryTempSet
- type MockLedgerReader
- func (m *MockLedgerReader) Close() error
- func (m *MockLedgerReader) GetHeader() xdr.LedgerHeaderHistoryEntry
- func (m *MockLedgerReader) GetSequence() uint32
- func (m *MockLedgerReader) GetUpgradeChanges() []Change
- func (m *MockLedgerReader) IgnoreUpgradeChanges()
- func (m *MockLedgerReader) Read() (LedgerTransaction, error)
- func (m *MockLedgerReader) ReadUpgradeChange() (Change, error)
- type MockLedgerWriter
- type MockStateReader
- type MockStateWriter
- type PostgresTempSet
- type SingleLedgerStateReader
- type StateReader
- type StateWriter
- type TempSet
- type UpgradeChangesContainer
Constants ¶
This section is empty.
Variables ¶
var ErrNotFound = errors.New("not found")
Functions ¶
This section is empty.
Types ¶
type ArchiveLedgerReader ¶
type ArchiveLedgerReader interface { GetSequence() uint32 Read() (bool, xdr.Transaction, xdr.TransactionResult, error) }
ArchiveLedgerReader placeholder
type Change ¶
type Change struct { Type xdr.LedgerEntryType Pre *xdr.LedgerEntry Post *xdr.LedgerEntry }
Change is a developer friendly representation of LedgerEntryChanges. It also provides some helper functions to quickly check if a given change has occured in an entry.
If an entry is created: Pre is nil and Post is not nil. If an entry is updated: Pre is not nil and Post is not nil. If an entry is removed: Pre is not nil and Post is nil.
func (*Change) AccountChangedExceptSigners ¶
AccountChangedExceptSigners returns true if account has changed WITHOUT checking the signers (except master key weight!). In other words, if the only change is connected to signers, this function will return false.
func (*Change) AccountSignersChanged ¶
AccountSignersChanged returns true if account signers have changed. Notice: this will return true on master key changes too!
func (*Change) LedgerEntryChangeType ¶
func (c *Change) LedgerEntryChangeType() xdr.LedgerEntryChangeType
LedgerEntryChangeType returns type in terms of LedgerEntryChangeType.
type DBLedgerReader ¶
type DBLedgerReader struct {
// contains filtered or unexported fields
}
DBLedgerReader is a database-backed implementation of the io.LedgerReader interface. Use NewDBLedgerReader to create a new instance.
func NewDBLedgerReader ¶
func NewDBLedgerReader(sequence uint32, backend ledgerbackend.LedgerBackend) (*DBLedgerReader, error)
NewDBLedgerReader is a factory method for LedgerReader.
func (*DBLedgerReader) Close ¶
func (dblrc *DBLedgerReader) Close() error
Close moves the read pointer so that subsequent calls to Read() will return EOF.
func (*DBLedgerReader) GetHeader ¶
func (dblrc *DBLedgerReader) GetHeader() xdr.LedgerHeaderHistoryEntry
GetHeader returns the XDR Header data associated with the stored ledger.
func (*DBLedgerReader) GetSequence ¶
func (dblrc *DBLedgerReader) GetSequence() uint32
GetSequence returns the sequence number of the ledger data stored by this object.
func (*DBLedgerReader) GetUpgradeChanges ¶
func (dblrc *DBLedgerReader) GetUpgradeChanges() []Change
GetUpgradeChanges returns all ledger upgrade changes.
func (*DBLedgerReader) IgnoreUpgradeChanges ¶
func (dblrc *DBLedgerReader) IgnoreUpgradeChanges()
func (*DBLedgerReader) Read ¶
func (dblrc *DBLedgerReader) Read() (LedgerTransaction, error)
Read returns the next transaction in the ledger, ordered by tx number, each time it is called. When there are no more transactions to return, an EOF error is returned.
func (*DBLedgerReader) ReadUpgradeChange ¶
func (dblrc *DBLedgerReader) ReadUpgradeChange() (Change, error)
ReadUpgradeChange returns the next upgrade change in the ledger, each time it is called. When there are no more upgrades to return, an EOF error is returned.
type LedgerEntryChangeCache ¶
type LedgerEntryChangeCache struct {
// contains filtered or unexported fields
}
LedgerEntryChangeCache is a cache of ledger entry changes that squashes all changes within a single ledger. By doing this, it decreases number of DB queries sent to a DB to update the current state of the ledger. It has integrity checks built in so ex. removing an account that was previously removed returns an error. In such case verify.StateError is returned.
It applies changes to the cache using the following algorithm:
- If the change is CREATED it checks if any change connected to given entry is already in the cache. If not, it adds CREATED change. Otherwise, if existing change is: a. CREATED it returns error because we can't add an entry that already exists. b. UPDATED it returns error because we can't add an entry that already exists. c. REMOVED it means that due to previous transitions we want to remove this from a DB what means that it already exists in a DB so we need to update the type of change to UPDATED.
- If the change is UPDATE it checks if any change connected to given entry is already in the cache. If not, it adds UPDATE change. Otherwise, if existing change is: a. CREATED it means that due to previous transitions we want to create this in a DB what means that it doesn't exist in a DB so we need to update the entry but stay with CREATED type. b. UPDATED we simply update it with the new value. c. REMOVED it means that at this point in the ledger the entry is removed so updating it returns an error.
- If the change is REMOVE it checks if any change connected to given entry is already in the cache. If not, it adds REMOVE change. Otherwise, if existing change is: a. CREATED it means that due to previous transitions we want to create this in a DB what means that it doesn't exist in a DB. If it was created and removed in the same ledger it's a noop so we remove entry from the cache. b. UPDATED we simply update it to be a REMOVE change because the UPDATE change means the entry exists in a DB. c. REMOVED it returns error because we can't remove an entry that was already removed.
func NewLedgerEntryChangeCache ¶
func NewLedgerEntryChangeCache() *LedgerEntryChangeCache
NewLedgerEntryChangeCache returns a new LedgerEntryChangeCache.
func (*LedgerEntryChangeCache) AddChange ¶
func (c *LedgerEntryChangeCache) AddChange(change Change) error
AddChange adds a change to LedgerEntryChangeCache. All changes are stored in memory. To get the final, squashed changes call GetChanges.
Please note that the current ledger capacity in pubnet (max 1000 ops/ledger) makes LedgerEntryChangeCache safe to use in terms of memory usage. If the cache takes too much memory, you apply changes returned by GetChanges and create a new LedgerEntryChangeCache object to continue ingestion.
func (*LedgerEntryChangeCache) GetChanges ¶
func (c *LedgerEntryChangeCache) GetChanges() []Change
GetChanges returns a slice of Changes in the cache. The order of changes is random but each change is connected to a separate entry.
type LedgerReader ¶
type LedgerReader interface { GetSequence() uint32 GetHeader() xdr.LedgerHeaderHistoryEntry // Read should return the next transaction. If there are no more // transactions it should return `io.EOF` error. Read() (LedgerTransaction, error) // Read should return the next ledger entry change from ledger upgrades. If // there are no more changes it should return `io.EOF` error. // Ledger upgrades MUST be processed AFTER all transactions and only ONCE. // If app is tracking state in more than one store, all of them need to // be updated with upgrade changes. // Values returned by this method must not be modified. ReadUpgradeChange() (Change, error) // IgnoreLedgerEntryChanges will change `Close()“ behaviour to not error // when changes returned by `ReadUpgradeChange` are not fully read. IgnoreUpgradeChanges() // Close should be called when reading is finished. This is especially // helpful when there are still some transactions available so reader can stop // streaming them. // Close should return error if `ReadUpgradeChange` are not fully read or // `ReadUpgradeChange` was not called even once. However, this behaviour can // be disabled by calling `IgnoreUpgradeChanges()`. Close() error }
LedgerReader provides convenient, streaming access to the transactions within a ledger.
type LedgerTransaction ¶
type LedgerTransaction struct { Index uint32 Envelope xdr.TransactionEnvelope Result xdr.TransactionResultPair // FeeChanges and Meta are low level values. // Use LedgerTransaction.GetChanges() for higher level access to ledger // entry changes. FeeChanges xdr.LedgerEntryChanges Meta xdr.TransactionMeta }
LedgerTransaction represents the data for a single transaction within a ledger.
func (*LedgerTransaction) GetChanges ¶
func (t *LedgerTransaction) GetChanges() []Change
GetChanges returns a developer friendly representation of LedgerEntryChanges. It contains transaction changes and operation changes in that order.
func (*LedgerTransaction) GetFeeChanges ¶
func (t *LedgerTransaction) GetFeeChanges() []Change
GetFeeChanges returns a developer friendly representation of LedgerEntryChanges connected to fees.
type LedgerWriter ¶
type LedgerWriter interface { // Write is used to pass a transaction to the next processor. It can return // `io.ErrClosedPipe` when the pipe between processors has been closed meaning // that next processor does not need more data. In such situation the current // processor can terminate as sending more transactions to a `LedgerWriter` // does not make sense (will not be read). Write(LedgerTransaction) error // Close should be called when reading is finished. This is especially // helpful when there are still some transactions available so the reader can stop // streaming them. Close() error }
LedgerWriter provides convenient, streaming access to the transactions within a ledger.
type MemoryTempSet ¶
type MemoryTempSet struct {
// contains filtered or unexported fields
}
MemoryTempSet is an in-memory implementation of TempSet interface. As of July 2019 this requires up to ~4GB of memory for pubnet ledger state processing. The internal structure is dereferenced after the store is closed.
func (*MemoryTempSet) Add ¶
func (s *MemoryTempSet) Add(key string) error
Add adds a key to TempSet.
func (*MemoryTempSet) Close ¶
func (s *MemoryTempSet) Close() error
Close removes reference to internal data structure.
func (*MemoryTempSet) Exist ¶
func (s *MemoryTempSet) Exist(key string) (bool, error)
Exist check if the key exists in a TempSet.
func (*MemoryTempSet) Open ¶
func (s *MemoryTempSet) Open() error
Open initialize internals data structure.
func (*MemoryTempSet) Preload ¶
func (s *MemoryTempSet) Preload(keys []string) error
Preload does not do anything. This TempSet keeps everything in memory so no preloading needed.
type MockLedgerReader ¶
func (*MockLedgerReader) Close ¶
func (m *MockLedgerReader) Close() error
func (*MockLedgerReader) GetHeader ¶
func (m *MockLedgerReader) GetHeader() xdr.LedgerHeaderHistoryEntry
func (*MockLedgerReader) GetSequence ¶
func (m *MockLedgerReader) GetSequence() uint32
func (*MockLedgerReader) GetUpgradeChanges ¶
func (m *MockLedgerReader) GetUpgradeChanges() []Change
func (*MockLedgerReader) IgnoreUpgradeChanges ¶
func (m *MockLedgerReader) IgnoreUpgradeChanges()
func (*MockLedgerReader) Read ¶
func (m *MockLedgerReader) Read() (LedgerTransaction, error)
func (*MockLedgerReader) ReadUpgradeChange ¶
func (m *MockLedgerReader) ReadUpgradeChange() (Change, error)
type MockLedgerWriter ¶
func (*MockLedgerWriter) Close ¶
func (m *MockLedgerWriter) Close() error
func (*MockLedgerWriter) Write ¶
func (m *MockLedgerWriter) Write(tx LedgerTransaction) error
type MockStateReader ¶
func (*MockStateReader) Close ¶
func (m *MockStateReader) Close() error
func (*MockStateReader) GetSequence ¶
func (m *MockStateReader) GetSequence() uint32
func (*MockStateReader) Read ¶
func (m *MockStateReader) Read() (xdr.LedgerEntryChange, error)
type MockStateWriter ¶
func (*MockStateWriter) Close ¶
func (m *MockStateWriter) Close() error
func (*MockStateWriter) Write ¶
func (m *MockStateWriter) Write(entryChange xdr.LedgerEntryChange) error
type PostgresTempSet ¶
type PostgresTempSet struct { DSN string Session *db.Session // contains filtered or unexported fields }
PostgresTempSet is a postgres implementation of TempSet interface. It's around 4x slower than MemoryStateReaderTempStore but has much lower memory requirements. `postgresTempSetCacheSize` can be changed to achieve better speed at the cost of higher memory usage. If `DSN` is passed, a new `db.Session` will be created. If `Session` is passed, it will be cloned.
func (*PostgresTempSet) Add ¶
func (s *PostgresTempSet) Add(key string) error
Add adds a key to TempSet.
func (*PostgresTempSet) Close ¶
func (s *PostgresTempSet) Close() error
Close closes a database connection what also removes a temporary table.
func (*PostgresTempSet) Exist ¶
func (s *PostgresTempSet) Exist(key string) (bool, error)
Exist check if the key exists in a TempSet.
func (*PostgresTempSet) Open ¶
func (s *PostgresTempSet) Open() error
Open connects to a DB and creates a temporary table and data structures.
func (*PostgresTempSet) Preload ¶
func (s *PostgresTempSet) Preload(keys []string) error
Preload preloads `keys` from a database. It ignores the keys that are already in cache because these are the most recent values.
type SingleLedgerStateReader ¶
type SingleLedgerStateReader struct {
// contains filtered or unexported fields
}
SingleLedgerStateReader is a streaming implementation that reads ledger entries from buckets for a given HistoryArchiveState (single ledger/checkpoint). SingleLedgerStateReader hides internal structure of buckets from the user so entries returned by `Read()` are exactly the ledger entries present at the given ledger.
func MakeSingleLedgerStateReader ¶
func MakeSingleLedgerStateReader( archive historyarchive.ArchiveInterface, tempStore TempSet, sequence uint32, maxStreamRetries int, ) (*SingleLedgerStateReader, error)
MakeSingleLedgerStateReader is a factory method for SingleLedgerStateReader. `maxStreamRetries` determines how many times the reader will retry when encountering errors while streaming xdr bucket entries from the history archive. Set `maxStreamRetries` to 0 if there should be no retry attempts
func (*SingleLedgerStateReader) Close ¶
func (msr *SingleLedgerStateReader) Close() error
Close should be called when reading is finished.
func (*SingleLedgerStateReader) GetSequence ¶
func (msr *SingleLedgerStateReader) GetSequence() uint32
GetSequence impl.
func (*SingleLedgerStateReader) Read ¶
func (msr *SingleLedgerStateReader) Read() (xdr.LedgerEntryChange, error)
Read returns a new ledger entry change on each call, returning io.EOF when the stream ends.
type StateReader ¶
type StateReader interface { // GetSequence returns the checkpoint ledger sequence this reader is // reading. GetSequence() uint32 // Read should return next ledger entry. If there are no more // entries it should return `io.EOF` error. Read() (xdr.LedgerEntryChange, error) // Close should be called when reading is finished. This is especially // helpful when there are still some entries available so reader can stop // streaming them. Close() error }
StateReader reads state data from history archive buckets for a single checkpoint ledger / HAS.
type StateWriter ¶
type StateWriter interface { // Write is used to pass ledger entry change to the next processor. It can return // `ErrClosedPipe` when the pipe between processors has been closed meaning // that next processor does not need more data. In such situation the current // processor can terminate as sending more entries to a `StateWriter` // does not make sense (will not be read). Write(xdr.LedgerEntryChange) error // Close should be called when there are no more entries // to write. Close() error }
StateWriter interface placeholder
type TempSet ¶
type TempSet interface { Open() error // Preload batch-loads keys into internal cache (if a store has any) to // improve execution time by removing many round-trips. Preload(keys []string) error // Add adds key to the store. Add(key string) error // Exist returns value true if the value is found in the store. // If the value has not been set, it should return false. Exist(key string) (bool, error) Close() error }
TempSet is an interface that must be implemented by stores that hold temporary set of objects for state reader. The implementation does not need to be thread-safe.
type UpgradeChangesContainer ¶
type UpgradeChangesContainer interface {
GetUpgradeChanges() []Change
}