ledgerbackend

package
v0.0.0-...-474a86a Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Sep 12, 2025 License: Apache-2.0, Apache-2.0 Imports: 41 Imported by: 16

Documentation

Index

Constants

This section is empty.

Variables

View Source 
var (
	//go:embed configs/captive-core-pubnet.cfg
	PubnetDefaultConfig []byte

	//go:embed configs/captive-core-testnet.cfg
	TestnetDefaultConfig []byte
)
View Source 
var ErrCannotStartFromGenesis = errors.New("CaptiveCore is unable to start from ledger 1, start from ledger 2")

ErrCannotStartFromGenesis is returned when attempting to prepare a range from ledger 1

Functions

func CoreBuildVersion 

func CoreBuildVersion(coreBinaryPath string) (string, error)

CoreBuildVersion executes the "stellar-core version" command and parses its output to extract the core version The output of the "version" command is expected to be a multi-line string where the first line is the core version in format "vX.Y.Z-*".

func CoreProtocolVersion 

func CoreProtocolVersion(coreBinaryPath string) (uint, error)

CoreProtocolVersion retrieves the ledger protocol version from the specified stellar-core binary. It executes the "stellar-core version" command and parses the output to extract the protocol version.

Types

type BufferedStorageBackend 

type BufferedStorageBackend struct {
	// contains filtered or unexported fields
}

BufferedStorageBackend is a ledger backend that reads from a storage service. The storage service contains files generated from the ledgerExporter.

func NewBufferedStorageBackend 

func NewBufferedStorageBackend(config BufferedStorageBackendConfig, dataStore datastore.DataStore, schema datastore.DataStoreSchema) (*BufferedStorageBackend, error)

NewBufferedStorageBackend returns a new BufferedStorageBackend instance.

func (*BufferedStorageBackend) Close 

func (bsb *BufferedStorageBackend) Close() error

Close closes existing BufferedStorageBackend processes. Note, once a BufferedStorageBackend instance is closed it can no longer be used and all subsequent calls to PrepareRange(), GetLedger(), etc will fail. Close is thread-safe and can be called from another go routine.

func (*BufferedStorageBackend) GetLatestLedgerSequence 

func (bsb *BufferedStorageBackend) GetLatestLedgerSequence(ctx context.Context) (uint32, error)

GetLatestLedgerSequence returns the most recent ledger sequence number available in the buffer.

func (*BufferedStorageBackend) GetLedger 

func (bsb *BufferedStorageBackend) GetLedger(ctx context.Context, sequence uint32) (xdr.LedgerCloseMeta, error)

GetLedger returns the LedgerCloseMeta for the specified ledger sequence number

func (*BufferedStorageBackend) IsPrepared 

func (bsb *BufferedStorageBackend) IsPrepared(ctx context.Context, ledgerRange Range) (bool, error)

IsPrepared returns true if a given ledgerRange is prepared.

func (*BufferedStorageBackend) PrepareRange 

func (bsb *BufferedStorageBackend) PrepareRange(ctx context.Context, ledgerRange Range) error

PrepareRange checks if the starting and ending (if bounded) ledgers exist.

type BufferedStorageBackendConfig 

type BufferedStorageBackendConfig struct {
	BufferSize uint32        `toml:"buffer_size"`
	NumWorkers uint32        `toml:"num_workers"`
	RetryLimit uint32        `toml:"retry_limit"`
	RetryWait  time.Duration `toml:"retry_wait"`
}

type CaptiveCoreConfig 

type CaptiveCoreConfig struct {
	// BinaryPath is the file path to the Stellar Core binary
	BinaryPath string
	// NetworkPassphrase is the Stellar network passphrase used by captive core when connecting to the Stellar network
	NetworkPassphrase string
	// HistoryArchiveURLs are a list of history archive urls
	HistoryArchiveURLs []string
	// UserAgent is the value of `User-Agent` header that will be send along http archive requests.
	UserAgent string
	Toml      *CaptiveCoreToml

	// CheckpointFrequency is the number of ledgers between checkpoints
	// if unset, DefaultCheckpointFrequency will be used
	CheckpointFrequency uint32
	// Log is an (optional) custom logger which will capture any output from the Stellar Core process.
	// If Log is omitted then all output will be printed to stdout.
	Log *log.Entry
	// Context is the (optional) context which controls the lifetime of a CaptiveStellarCore instance. Once the context is done
	// the CaptiveStellarCore instance will not be able to stream ledgers from Stellar Core or spawn new
	// instances of Stellar Core. If Context is omitted CaptiveStellarCore will default to using context.Background.
	Context context.Context
	// StoragePath is the (optional) base path passed along to Core's
	// BUCKET_DIR_PATH which specifies where various bucket data should be
	// stored. We always append /captive-core to this directory, since we clean
	// it up entirely on shutdown.
	StoragePath string

	// CoreProtocolVersionFn is a function that returns the protocol version of the stellar-core binary.
	CoreProtocolVersionFn CoreProtocolVersionFunc

	// CoreBuildVersionFn is a function that returns the build version of the stellar-core binary.
	CoreBuildVersionFn CoreBuildVersionFunc
}

CaptiveCoreConfig contains all the parameters required to create a CaptiveStellarCore instance

type CaptiveCoreToml 

type CaptiveCoreToml struct {
	// contains filtered or unexported fields
}

CaptiveCoreToml represents a parsed captive core configuration.

func NewCaptiveCoreToml 

func NewCaptiveCoreToml(params CaptiveCoreTomlParams) (*CaptiveCoreToml, error)

NewCaptiveCoreToml constructs a new CaptiveCoreToml instance based off the configuration in `params`.

func NewCaptiveCoreTomlFromData 

func NewCaptiveCoreTomlFromData(data []byte, params CaptiveCoreTomlParams) (*CaptiveCoreToml, error)

NewCaptiveCoreTomlFromData constructs a new CaptiveCoreToml instance by merging configuration from the toml data and the configuration provided by `params`.

func NewCaptiveCoreTomlFromFile 

func NewCaptiveCoreTomlFromFile(configPath string, params CaptiveCoreTomlParams) (*CaptiveCoreToml, error)

NewCaptiveCoreTomlFromFile constructs a new CaptiveCoreToml instance by merging configuration from the toml file located at `configPath` and the configuration provided by `params`.

func (*CaptiveCoreToml) AddExamplePubnetValidators 

func (c *CaptiveCoreToml) AddExamplePubnetValidators()

AddExamplePubnetQuorum adds example pubnet validators to toml file

func (*CaptiveCoreToml) CatchupToml 

func (c *CaptiveCoreToml) CatchupToml() (*CaptiveCoreToml, error)

CatchupToml returns a new CaptiveCoreToml instance based off the existing instance with some modifications which are suitable for running the catchup command on captive core.

func (*CaptiveCoreToml) HistoryIsConfigured 

func (c *CaptiveCoreToml) HistoryIsConfigured() bool

HistoryIsConfigured returns true if the history archive locations are configured.

func (*CaptiveCoreToml) Marshal 

func (c *CaptiveCoreToml) Marshal() ([]byte, error)

Marshal serializes the CaptiveCoreToml into a toml document.

func (*CaptiveCoreToml) QuorumSetIsConfigured 

func (c *CaptiveCoreToml) QuorumSetIsConfigured() bool

QuorumSetIsConfigured returns true if there is a quorum set defined in the configuration.

type CaptiveCoreTomlParams 

type CaptiveCoreTomlParams struct {
	// NetworkPassphrase is the Stellar network passphrase used by captive core when connecting to the Stellar network.
	NetworkPassphrase string
	// HistoryArchiveURLs are a list of history archive urls.
	HistoryArchiveURLs []string
	// HTTPPort is the TCP port to listen for requests (defaults to 0, which disables the HTTP server).
	HTTPPort *uint
	// PeerPort is the TCP port to bind to for connecting to the Stellar network
	// (defaults to 11625). It may be useful for example when there's >1 Stellar-Core
	// instance running on a machine.
	PeerPort *uint
	// LogPath is the (optional) path in which to store Core logs, passed along
	// to Stellar Core's LOG_FILE_PATH.
	LogPath *string
	// Strict is a flag which, if enabled, rejects Stellar Core toml fields which are not supported by captive core.
	Strict bool
	// the path to the core binary, used to introspect core at runtime, determine some toml capabilities
	CoreBinaryPath string
	// Enforce EnableSorobanDiagnosticEvents and EnableDiagnosticsForTxSubmission when not disabled explicitly
	EnforceSorobanDiagnosticEvents bool
	// Enforce EnableSorobanTransactionMetaExtV1 when not disabled explicitly
	EnforceSorobanTransactionMetaExtV1 bool
	// Emits unified events for all operations
	EmitUnifiedEvents bool
	// Enable backfilled events
	EmitUnifiedEventsBeforeProtocol22 bool
	// Fast HTTP Query Server parameters
	HTTPQueryServerParams *HTTPQueryServerParams
	// CoreBuildVersionFn is a function that returns the build version of the stellar-core binary.
	CoreBuildVersionFn CoreBuildVersionFunc
	// CoreProtocolVersionFn is a function that returns the protocol version of the stellar-core binary.
	CoreProtocolVersionFn CoreProtocolVersionFunc
	// BackfillRestoreMeta is the value assigned to BACKFILL_RESTORE_META in the captive core toml file.
	// If omitted we will default BACKFILL_RESTORE_META to true.
	BackfillRestoreMeta *bool
	// This will enable a series of core configurations to generate the most verbose meta possible.
	EmitVerboseMeta bool
}

CaptiveCoreTomlParams defines captive core configuration provided by Horizon flags.

type CaptiveStellarCore 

type CaptiveStellarCore struct {
	// contains filtered or unexported fields
}

CaptiveStellarCore is a ledger backend that starts internal Stellar-Core subprocess responsible for streaming ledger data. It provides better decoupling than DatabaseBackend but requires some extra init time.

It operates in two modes:

  • When a BoundedRange is prepared it starts Stellar-Core in catchup mode that replays ledgers in memory. This is very fast but requires Stellar-Core to keep ledger state in RAM. It requires around 3GB of RAM as of August 2020.
  • When a UnboundedRange is prepared it runs Stellar-Core catchup mode to sync with the first ledger and then runs it in a normal mode. This requires the configAppendPath to be provided because a quorum set needs to be selected.

When running CaptiveStellarCore will create a temporary folder to store bucket files and other temporary files. The folder is removed when Close is called.

The communication is performed via filesystem pipe which is created in a temporary folder.

Currently BoundedRange requires a full-trust on history archive. This issue is being fixed in Stellar-Core.

While using BoundedRanges is straightforward there are a few gotchas connected to UnboundedRanges:

  • PrepareRange takes more time because all ledger entries must be stored on disk instead of RAM.
  • If GetLedger is not called frequently (every 5 sec. on average) the Stellar-Core process can go out of sync with the network. This happens because there is no buffering of communication pipe and CaptiveStellarCore has a very small internal buffer and Stellar-Core will not close the new ledger if it's not read.

Except for the Close function, CaptiveStellarCore is not thread-safe and should not be accessed by multiple go routines. Close is thread-safe and can be called from another go routine. Once Close is called it will interrupt and cancel any pending operations.

Requires Stellar-Core v13.2.0+.

func NewCaptive 

func NewCaptive(config CaptiveCoreConfig) (*CaptiveStellarCore, error)

NewCaptive returns a new CaptiveStellarCore instance.

func (*CaptiveStellarCore) Close 

func (c *CaptiveStellarCore) Close() error

Close closes existing Stellar-Core process, streaming sessions and removes all temporary files. Note, once a CaptiveStellarCore instance is closed it can no longer be used and all subsequent calls to PrepareRange(), GetLedger(), etc will fail. Close is thread-safe and can be called from another go routine.

func (*CaptiveStellarCore) GetCoreVersion 

func (c *CaptiveStellarCore) GetCoreVersion() string

func (*CaptiveStellarCore) GetLatestLedgerSequence 

func (c *CaptiveStellarCore) GetLatestLedgerSequence(ctx context.Context) (uint32, error)

GetLatestLedgerSequence returns the sequence of the latest ledger available in the backend. This method returns an error if not in a session (start with PrepareRange).

Note that for UnboundedRange the returned sequence number is not necessarily the latest sequence closed by the network. It's always the last value available in the backend.

func (*CaptiveStellarCore) GetLedger 

func (c *CaptiveStellarCore) GetLedger(ctx context.Context, sequence uint32) (xdr.LedgerCloseMeta, error)

GetLedger will block until the ledger is available in the backend (even for UnboundedRange), then return it's LedgerCloseMeta.

Call PrepareRange first to instruct the backend which ledgers to fetch. CaptiveStellarCore requires PrepareRange call first to initialize Stellar-Core. Requesting a ledger on non-prepared backend will return an error.

Please note that requesting a ledger sequence far after current ledger will block the execution for a long time.

Because ledger data is streamed from Stellar-Core sequentially, users should request sequences in a non-decreasing order. If the requested sequence number is less than the last requested sequence number, an error will be returned.

This function behaves differently for bounded and unbounded ranges:

  • BoundedRange: After getting the last ledger in a range this method will also Close() the backend.

func (*CaptiveStellarCore) IsPrepared 

func (c *CaptiveStellarCore) IsPrepared(ctx context.Context, ledgerRange Range) (bool, error)

IsPrepared returns true if a given ledgerRange is prepared.

func (*CaptiveStellarCore) PrepareRange 

func (c *CaptiveStellarCore) PrepareRange(ctx context.Context, ledgerRange Range) error

PrepareRange prepares the given range (including from and to) to be loaded. Captive stellar-core backend needs to initialize Stellar-Core state to be able to stream ledgers. Stellar-Core mode depends on the provided ledgerRange:

  • For BoundedRange it will start Stellar-Core in catchup mode.
  • For UnboundedRange it will first catchup to starting ledger and then run it normally (including connecting to the Stellar network).

Please note that using a BoundedRange, currently, requires a full-trust on history archive. This issue is being fixed in Stellar-Core.

type CoreBuildVersionFunc 

type CoreBuildVersionFunc func(coreBinaryPath string) (string, error)

type CoreProtocolVersionFunc 

type CoreProtocolVersionFunc func(coreBinaryPath string) (uint, error)

type HTTPQueryServerParams 

type HTTPQueryServerParams struct {
	Port            uint16
	ThreadPoolSize  uint16
	SnapshotLedgers uint16
}

type History 

type History struct {
	Get string `toml:"get"`
	// should we allow put and mkdir for captive core?
	Put   string `toml:"put,omitempty"`
	Mkdir string `toml:"mkdir,omitempty"`
}

History represents a [HISTORY] table in the captive core toml file.

type HomeDomain 

type HomeDomain struct {
	HomeDomain string `toml:"HOME_DOMAIN"`
	Quality    string `toml:"QUALITY"`
}

HomeDomain represents a [[HOME_DOMAINS]] entry in the captive core toml file.

type LedgerBackend 

type LedgerBackend interface {
	// GetLatestLedgerSequence returns the sequence of the latest ledger available
	// in the backend.
	GetLatestLedgerSequence(ctx context.Context) (sequence uint32, err error)
	// GetLedger will block until the ledger is available.
	GetLedger(ctx context.Context, sequence uint32) (xdr.LedgerCloseMeta, error)
	// PrepareRange prepares the given range (including from and to) to be loaded.
	// Some backends (like captive stellar-core) need to initalize data to be
	// able to stream ledgers. Blocks until the first ledger is available.
	PrepareRange(ctx context.Context, ledgerRange Range) error
	// IsPrepared returns true if a given ledgerRange is prepared.
	IsPrepared(ctx context.Context, ledgerRange Range) (bool, error)
	Close() error
}

LedgerBackend represents the interface to a ledger data store.

func WithMetrics 

func WithMetrics(base LedgerBackend, registry *prometheus.Registry, namespace string) LedgerBackend

WithMetrics decorates the given LedgerBackend with metrics

type MockDatabaseBackend 

type MockDatabaseBackend struct {
	mock.Mock
}

func (*MockDatabaseBackend) Close 

func (m *MockDatabaseBackend) Close() error

func (*MockDatabaseBackend) GetLatestLedgerSequence 

func (m *MockDatabaseBackend) GetLatestLedgerSequence(ctx context.Context) (uint32, error)

func (*MockDatabaseBackend) GetLedger 

func (m *MockDatabaseBackend) GetLedger(ctx context.Context, sequence uint32) (xdr.LedgerCloseMeta, error)

func (*MockDatabaseBackend) IsPrepared 

func (m *MockDatabaseBackend) IsPrepared(ctx context.Context, ledgerRange Range) (bool, error)

func (*MockDatabaseBackend) PrepareRange 

func (m *MockDatabaseBackend) PrepareRange(ctx context.Context, ledgerRange Range) error

type QuorumSet 

type QuorumSet struct {
	ThresholdPercent int      `toml:"THRESHOLD_PERCENT"`
	Validators       []string `toml:"VALIDATORS"`
}

QuorumSet represents a [QUORUM_SET] table in the captive core toml file.

type RPCLedgerBackend 

type RPCLedgerBackend struct {
	// contains filtered or unexported fields
}

func NewRPCLedgerBackend 

func NewRPCLedgerBackend(options RPCLedgerBackendOptions) *RPCLedgerBackend

NewRPCLedgerBackend creates a new RPCLedgerBackend instance that fetches ledger data from a Stellar RPC server.

Parameters:

  • options: RPCLedgerBackendOptions

Returns:

  • *RPCLedgerBackend: A new backend instance ready for use

func (*RPCLedgerBackend) Close 

func (b *RPCLedgerBackend) Close() error

Close cleans up the RPCLedgerBackend resources and closes the backend. It will halt any ongoing GerLedger calls that may be in progress on other goroutines.

func (*RPCLedgerBackend) GetLatestLedgerSequence 

func (b *RPCLedgerBackend) GetLatestLedgerSequence(ctx context.Context) (sequence uint32, err error)

GetLatestLedgerSequence returns the latest ledger sequence currently loaded by internal buffer.

func (*RPCLedgerBackend) GetLedger 

func (b *RPCLedgerBackend) GetLedger(ctx context.Context, sequence uint32) (xdr.LedgerCloseMeta, error)

GetLedger queries the RPC server for a specific ledger sequence and returns the meta data. If the requested ledger is not immediately available but is beyond the latest ledger in the RPC server, it will block and retry until either:

  • The ledger becomes available
  • The context is cancelled or times out
  • The backend is closed
  • An error occurs while fetching the ledger

The caller can control the maximum wait time by setting a timeout or deadline on the provided context. Or by invoking the Close method from another goroutine.

Parameters:

  • ctx: Context for cancellation and timeout control
  • sequence: The ledger sequence number to retrieve meta data for

Returns:

  • xdr.LedgerCloseMeta: The ledger meta data if found
  • error: if ledger sequence is not avaialble from the RPC server, or context is cancelled or times out.

func (*RPCLedgerBackend) IsPrepared 

func (b *RPCLedgerBackend) IsPrepared(ctx context.Context, ledgerRange Range) (bool, error)

func (*RPCLedgerBackend) PrepareRange 

func (b *RPCLedgerBackend) PrepareRange(ctx context.Context, ledgerRange Range) error

PrepareRange initiates retrieval of requested ledger range. It does minimal validation of data on RPC up front. It wil check if starting point of range is withing current historical retention window of the RPC server. It cannot gaurantee ledgers within historical ranges will be available when requested later by GetLedger. See Also: GetLedger for more details on how the RPCLedgerBackend handles ledger availability.

type RPCLedgerBackendOptions 

type RPCLedgerBackendOptions struct {
	// Required, URL of the Stellar RPC server
	RPCServerURL string

	// Optional, size of the ledger retrieval buffer to use with RPC server requests.
	// if not set, defaults to 10
	BufferSize uint32

	// Optional, custom HTTP client to use for RPC requests.
	// If nil, the default http.Client will be used.
	HttpClient *http.Client
}

type RPCLedgerGetter 

type RPCLedgerGetter interface {
	GetLedgers(ctx context.Context, req protocol.GetLedgersRequest) (protocol.GetLedgersResponse, error)
	GetHealth(ctx context.Context) (protocol.GetHealthResponse, error) // <-- Added
}

The minimum required RPC client methods used by RPCLedgerBackend.

type RPCLedgerMissingError 

type RPCLedgerMissingError struct {
	Sequence uint32
}

func (*RPCLedgerMissingError) Error 

func (e *RPCLedgerMissingError) Error() string

type Range 

type Range struct {
	// contains filtered or unexported fields
}

Range represents a range of ledger sequence numbers.

func BoundedRange 

func BoundedRange(from uint32, to uint32) Range

BoundedRange constructs a bounded range of ledgers with a fixed starting ledger and ending ledger.

func SingleLedgerRange 

func SingleLedgerRange(ledger uint32) Range

SingleLedgerRange constructs a bounded range containing a single ledger.

func UnboundedRange 

func UnboundedRange(from uint32) Range

BoundedRange constructs a unbounded range of ledgers with a fixed starting ledger.

func (Range) Bounded 

func (r Range) Bounded() bool

func (Range) Contains 

func (r Range) Contains(other Range) bool

func (Range) From 

func (r Range) From() uint32

func (Range) MarshalJSON 

func (r Range) MarshalJSON() ([]byte, error)

func (Range) String 

func (r Range) String() string

func (Range) To 

func (r Range) To() uint32

func (*Range) UnmarshalJSON 

func (r *Range) UnmarshalJSON(b []byte) error

type Validator 

type Validator struct {
	Name       string `toml:"NAME"`
	Quality    string `toml:"QUALITY,omitempty"`
	HomeDomain string `toml:"HOME_DOMAIN"`
	PublicKey  string `toml:"PUBLIC_KEY"`
	Address    string `toml:"ADDRESS,omitempty"`
	History    string `toml:"HISTORY,omitempty"`
}

Validator represents a [[VALIDATORS]] entry in the captive core toml file.

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.