snapshotdl

Documentation

Index

• func DownloadIncrementalSnapshot(rpcEndpoints []string, path string, referenceSlot int, fullSnapshotSlot int) (string, int, int, error)
• func DownloadIncrementalSnapshotWithConfig(path string, referenceSlot int, fullSnapshotSlot int, snapCfg SnapshotConfig) (string, int, int, error)
• func DownloadSnapshot(ctx context.Context, rpcEndpoints []string, path string) (string, int, int, error)
• func DownloadSnapshotWithConfig(ctx context.Context, path string, snapCfg SnapshotConfig) (string, int, int, error)
• func ExtractIncrementalSnapshotSlots(path string) (int, int)
• func GetIncrementalSnapshotURL(fullSnapshotURL string, referenceSlot int, fullSnapshotSlot int, ...) (string, int, int, error)
• func GetSnapshotURL(ctx context.Context, snapCfg SnapshotConfig) (string, int, int, error)
• type SnapshotConfig
  • func DefaultSnapshotConfig() SnapshotConfig
• type SnapshotInfo
  • func GetSnapshotURLWithInfo(ctx context.Context, snapCfg SnapshotConfig) (*SnapshotInfo, error)
  • func (s *SnapshotInfo) Age() int

Functions

func DownloadIncrementalSnapshot

func DownloadIncrementalSnapshot(rpcEndpoints []string, path string, referenceSlot int, fullSnapshotSlot int) (string, int, int, error)

DownloadIncrementalSnapshot downloads an incremental snapshot that matches the given fullSnapshotSlot.

It returns: (downloadPath, baseSlot, endSlot, error)

This function uses the new FindMatchingIncremental API to search across ranked nodes for an incremental snapshot that matches the full snapshot.

func DownloadIncrementalSnapshotWithConfig

func DownloadIncrementalSnapshotWithConfig(path string, referenceSlot int, fullSnapshotSlot int, snapCfg SnapshotConfig) (string, int, int, error)

DownloadIncrementalSnapshotWithConfig is like DownloadIncrementalSnapshot but accepts custom config

func DownloadSnapshot

func DownloadSnapshot(ctx context.Context, rpcEndpoints []string, path string) (string, int, int, error)

DownloadSnapshot downloads a full snapshot from the best available RPC node. It returns: (downloadPath, referenceSlot, snapshotSlot, error)

This function: 1. Gets the reference slot from the network 2. Discovers available RPC nodes from the cluster 3. Evaluates nodes for snapshot availability and download speed 4. Downloads from the fastest node with a recent snapshot

func DownloadSnapshotWithConfig

func DownloadSnapshotWithConfig(ctx context.Context, path string, snapCfg SnapshotConfig) (string, int, int, error)

DownloadSnapshotWithConfig is like DownloadSnapshot but accepts custom config

func ExtractIncrementalSnapshotSlots

func ExtractIncrementalSnapshotSlots(path string) (int, int)

ExtractIncrementalSnapshotSlots extracts the base and end slots from an incremental snapshot filename.

Expected format: incremental-snapshot-{baseSlot}-{endSlot}-{hash}.tar.zst Returns: (baseSlot, endSlot)

func GetIncrementalSnapshotURL

func GetIncrementalSnapshotURL(fullSnapshotURL string, referenceSlot int, fullSnapshotSlot int, snapCfg SnapshotConfig) (string, int, int, error)

GetIncrementalSnapshotURL finds an incremental snapshot URL that matches the full snapshot. It first tries the same source as the full snapshot, then searches other nodes if needed. Returns: (httpURL, baseSlot, endSlot, error)

Fallback strategy (different from full snapshot):

1. Try the same node that provided the full snapshot (fastest, most likely match)
2. If that fails, find ALL nodes with matching base slot (more flexible than full snapshot)
3. Among matching nodes, prioritize by: a. Freshness (highest end slot = most recent incremental) b. Speed (faster downloads preferred when end slots are equal)
4. Try multiple candidates for resilience (uses MaxSnapshotURLAttempts)

func GetSnapshotURL

func GetSnapshotURL(ctx context.Context, snapCfg SnapshotConfig) (string, int, int, error)

GetSnapshotURL discovers the best RPC node and returns the HTTP URL for streaming. Returns: (httpURL, referenceSlot, snapshotSlot, error)

This function: 1. Gets the reference slot from the network 2. Discovers available RPC nodes from the cluster 3. Evaluates nodes for snapshot availability and download speed 4. Returns HTTP URL from the fastest node (for streaming)

The returned URL can be passed directly to snapshot processing functions which will stream the data from HTTP (no disk download required).

Types

type SnapshotConfig

type SnapshotConfig struct {
	// RPC endpoints for reference slot and cluster discovery
	RPCAddresses []string

	// Stage 1: Fast parallel triage
	Stage1WarmKiB     int64
	Stage1WindowKiB   int64
	Stage1Windows     int
	Stage1TimeoutMS   int64
	Stage1Concurrency int

	// Stage 2: Sustained speed test for top candidates
	Stage2TopK       int
	Stage2WarmSec    int
	Stage2MeasureSec int
	Stage2MinRatio   float64
	Stage2MinAbsMBs  float64

	// Node filtering
	MaxRTTMs            int
	TCPTimeoutMs        int
	MinNodeVersion      string
	AllowedNodeVersions []string

	// Snapshot age thresholds (slots)
	FullThreshold        int
	IncrementalThreshold int

	// Snapshot storage (controls both saving and retention)
	// 0 = Stream-only mode (don't save snapshots to disk)
	// 1+ = Save snapshots and keep up to N on disk
	MaxFullSnapshots int

	// Safety
	SafetyMarginSlots int

	// Worker settings
	WorkerCount int

	// Output verbosity
	Verbose bool

	// Download path (only used when MaxFullSnapshots > 0)
	DownloadPath string

	// Fallback resilience
	MaxSnapshotURLAttempts int // Number of ranked nodes to try when getting snapshot URLs (0 = try all)

	// Incremental snapshot selection
	MinIncrementalSpeedMBs float64 // Minimum speed for incremental sources (MB/s, 0 = no minimum)

	// Slot constraints
	MaxSlot int64 // Maximum slot for snapshots (0 = no limit). Filters out full snapshots and incrementals above this slot.

	// Incremental snapshot targeting
	TargetIncrementalBase int64 // Only consider incrementals with this base slot (0 = any base)
	MaxIncrementalSlot    int64 // Maximum slot for incremental snapshots (0 = no limit)

	// Logging
	LogDir string // Directory for snapshot finder logs (default: /mnt/mithril-logs/snapshot-finder)
}

SnapshotConfig holds configuration for snapshot downloading. This can be populated from CLI flags or TOML config.

func DefaultSnapshotConfig

func DefaultSnapshotConfig() SnapshotConfig

DefaultSnapshotConfig returns production-ready defaults matching solana-snapshot-finder-go

type SnapshotInfo

type SnapshotInfo struct {
	URL           string  // HTTP URL for streaming
	Slot          int     // Full snapshot slot
	ReferenceSlot int     // Current network slot (for calculating age)
	NodeIP        string  // IP:port of the selected node
	NodeVersion   string  // Solana version of the node
	SpeedMBs      float64 // Download speed in MB/s from Stage 2 testing
	RTTMs         int     // Round-trip time in milliseconds
}

SnapshotInfo contains details about a selected snapshot source. This is returned by GetSnapshotURLWithInfo for display purposes.

func GetSnapshotURLWithInfo

func GetSnapshotURLWithInfo(ctx context.Context, snapCfg SnapshotConfig) (*SnapshotInfo, error)

GetSnapshotURLWithInfo discovers the best RPC node and returns detailed info about the source. Returns: (*SnapshotInfo, error)

This is like GetSnapshotURL but returns a SnapshotInfo struct with additional details useful for display (node IP, version, speed, age).

func (*SnapshotInfo) Age

func (s *SnapshotInfo) Age() int

Age returns how many slots behind the snapshot is from the current tip