WAL

package
v0.0.0-...-1d04b4a Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: May 12, 2026 License: Apache-2.0 Imports: 15 Imported by: 0

Documentation

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

func EventFactory 

func EventFactory(walType wal_types.WALType) (wal_types.EventAdapter, error)

EventFactory creates the appropriate event adapter based on WAL type

Types

type AccountSyncEvent 

type AccountSyncEvent struct {
	wal_types.BaseEvent
	Response  *accounts.AccountSyncResponse `json:"-"`
	ProtoData []byte                        `json:"proto_data"`
}

AccountSyncEvent records one page of missing accounts delivered to the client during AccountSync (Phase 5). The WAL entry is written before the accounts are persisted to the database, enabling crash recovery via idempotent CreateAccount.

Implements wal_types.EventAdapter.

func (*AccountSyncEvent) Deserialize 

func (e *AccountSyncEvent) Deserialize(data []byte) error

Deserialize reconstructs the event from WAL bytes produced by Serialize. Returns an error if either the JSON envelope or the embedded proto is malformed.

Time: O(n) where n = total bytes in the serialised form. Space: O(n) — allocates the decoded AccountSyncResponse and its accounts slice.

func (*AccountSyncEvent) GetOperation 

func (e *AccountSyncEvent) GetOperation() wal_types.EventOperation

GetOperation returns the event operation type.

Time: O(1). Space: O(1).

func (*AccountSyncEvent) GetType 

func (e *AccountSyncEvent) GetType() wal_types.WALType

GetType returns the WAL type identifier for AccountSync events.

Time: O(1). Space: O(1).

func (*AccountSyncEvent) Serialize 

func (e *AccountSyncEvent) Serialize() ([]byte, error)

Serialize encodes the event to bytes for WAL storage. The AccountSyncResponse proto is marshalled to ProtoData; the outer struct is then JSON-encoded so the BaseEvent LSN and type fields are preserved.

Time: O(n) where n = total bytes across all accounts in the page. Space: O(n) — allocates proto bytes + JSON bytes proportional to account count.

type DataSyncEvent 

type DataSyncEvent struct {
	wal_types.BaseEvent
	Response  *datasync.DataSyncResponse `json:"-"` // Proto message
	ProtoData []byte                     `json:"proto_data"`
}

DataSyncEvent represents a data synchronization event (non-header block data)

func (*DataSyncEvent) Deserialize 

func (e *DataSyncEvent) Deserialize(data []byte) error

func (*DataSyncEvent) GetOperation 

func (e *DataSyncEvent) GetOperation() wal_types.EventOperation

func (*DataSyncEvent) GetType 

func (e *DataSyncEvent) GetType() wal_types.WALType

func (*DataSyncEvent) Serialize 

func (e *DataSyncEvent) Serialize() ([]byte, error)

type HeaderSyncEvent 

type HeaderSyncEvent struct {
	wal_types.BaseEvent
	Response *headersync.HeaderSyncResponse `json:"-"` // Proto message, not JSON serialized
	// We store the proto as bytes in ProtoData for JSON serialization
	ProtoData []byte `json:"proto_data"`
}

HeaderSyncEvent represents a header synchronization event

func (*HeaderSyncEvent) Deserialize 

func (e *HeaderSyncEvent) Deserialize(data []byte) error

func (*HeaderSyncEvent) GetOperation 

func (e *HeaderSyncEvent) GetOperation() wal_types.EventOperation

func (*HeaderSyncEvent) GetType 

func (e *HeaderSyncEvent) GetType() wal_types.WALType

func (*HeaderSyncEvent) Serialize 

func (e *HeaderSyncEvent) Serialize() ([]byte, error)

type LSNMetadata 

type LSNMetadata struct {
	CurrentLSN     uint64 `json:"current_lsn"`
	LastFlushedLSN uint64 `json:"last_flushed_lsn"`
	LastUpdated    int64  `json:"last_updated"`
}

LSNMetadata stores the current LSN state for recovery

type MerkleSyncEvent 

type MerkleSyncEvent struct {
	wal_types.BaseEvent
	Message   *merkle.MerkleMessage `json:"-"` // Proto message
	ProtoData []byte                `json:"proto_data"`
}

MerkleSyncEvent represents a Merkle tree synchronization event

func (*MerkleSyncEvent) Deserialize 

func (e *MerkleSyncEvent) Deserialize(data []byte) error

func (*MerkleSyncEvent) GetOperation 

func (e *MerkleSyncEvent) GetOperation() wal_types.EventOperation

func (*MerkleSyncEvent) GetType 

func (e *MerkleSyncEvent) GetType() wal_types.WALType

func (*MerkleSyncEvent) Serialize 

func (e *MerkleSyncEvent) Serialize() ([]byte, error)

type PoTSBlockEvent 

type PoTSBlockEvent struct {
	wal_types.BaseEvent
	BlockNumber uint64 `json:"block_number"` // denormalised for fast index rebuilds
	BlockData   []byte `json:"block_data"`   // JSON-encoded types.ZKBlock
	Timestamp   int64  `json:"timestamp"`
}

PoTSBlockEvent stores a single finalised block received during the PoTS collection window. BlockData holds the JSON-encoded ZKBlock so this package stays free of a dependency on common/types (which already imports common/WAL).

func (*PoTSBlockEvent) Deserialize 

func (e *PoTSBlockEvent) Deserialize(data []byte) error

func (*PoTSBlockEvent) GetOperation 

func (e *PoTSBlockEvent) GetOperation() wal_types.EventOperation

func (*PoTSBlockEvent) GetType 

func (e *PoTSBlockEvent) GetType() wal_types.WALType

func (*PoTSBlockEvent) Serialize 

func (e *PoTSBlockEvent) Serialize() ([]byte, error)

type PriorSyncEvent 

type PriorSyncEvent struct {
	wal_types.BaseEvent
	Message   *priorsync.PriorSyncMessage `json:"-"` // Proto message
	ProtoData []byte                      `json:"proto_data"`
}

PriorSyncEvent represents a prior synchronization event

func (*PriorSyncEvent) Deserialize 

func (e *PriorSyncEvent) Deserialize(data []byte) error

func (*PriorSyncEvent) GetOperation 

func (e *PriorSyncEvent) GetOperation() wal_types.EventOperation

func (*PriorSyncEvent) GetType 

func (e *PriorSyncEvent) GetType() wal_types.WALType

func (*PriorSyncEvent) Serialize 

func (e *PriorSyncEvent) Serialize() ([]byte, error)

type ReconciliationBatchEntry 

type ReconciliationBatchEntry struct {
	AccountAddress string `json:"a"`
	NewBalance     string `json:"b"`
	Nonce          uint64 `json:"n"`
}

ReconciliationBatchEntry is a single account record inside a batch WAL event.

type ReconciliationBatchEvent 

type ReconciliationBatchEvent struct {
	wal_types.BaseEvent
	Accounts  []ReconciliationBatchEntry `json:"accounts"`
	Timestamp int64                      `json:"timestamp"`
}

ReconciliationBatchEvent writes all reconciled accounts as a single WAL entry. This is far more efficient than writing one ReconciliationEvent per account when the account count is in the hundreds of thousands.

func (*ReconciliationBatchEvent) Deserialize 

func (e *ReconciliationBatchEvent) Deserialize(data []byte) error

func (*ReconciliationBatchEvent) GetOperation 

func (e *ReconciliationBatchEvent) GetOperation() wal_types.EventOperation

func (*ReconciliationBatchEvent) GetType 

func (e *ReconciliationBatchEvent) GetType() wal_types.WALType

func (*ReconciliationBatchEvent) Serialize 

func (e *ReconciliationBatchEvent) Serialize() ([]byte, error)

type ReconciliationEvent 

type ReconciliationEvent struct {
	wal_types.BaseEvent
	AccountAddress string `json:"account_address"`
	OldBalance     string `json:"old_balance"`
	NewBalance     string `json:"new_balance"`
	Nonce          uint64 `json:"nonce"`
	Timestamp      int64  `json:"timestamp"`
}

ReconciliationEvent represents an account reconciliation event

func (*ReconciliationEvent) Deserialize 

func (e *ReconciliationEvent) Deserialize(data []byte) error

Deserialize reconstructs the event from bytes

func (*ReconciliationEvent) GetOperation 

func (e *ReconciliationEvent) GetOperation() wal_types.EventOperation

GetOperation returns the event operation type

func (*ReconciliationEvent) GetType 

func (e *ReconciliationEvent) GetType() wal_types.WALType

GetType returns the WAL type for reconciliation events

func (*ReconciliationEvent) Serialize 

func (e *ReconciliationEvent) Serialize() ([]byte, error)

Serialize converts the event to bytes for storage

type WAL 

type WAL struct {
	// Dir is the directory path where WAL files are stored
	Dir string

	// Log is the WAL handle from tidwall/wal
	Log *wal.Log

	// Buffer holds events in memory before flushing
	Buffer []WALEntry

	// BatchSize is the threshold to trigger a flush to disk
	BatchSize int

	// Mu protects the WAL state for concurrent access
	Mu sync.RWMutex
	// contains filtered or unexported fields
}

WAL represents the Write-Ahead Log for event sourcing

func NewWAL 

func NewWAL(dir string, batchSize int) (*WAL, error)

NewWAL creates a new WAL (Write-Ahead Log) instance. Parameters:

  • dir: The directory where WAL files and metadata will be stored.
  • batchSize: The number of events to buffer before auto-flushing to disk.

Returns a pointer to the WAL instance or an error if initialization fails.

func (*WAL) Close 

func (w *WAL) Close() error

Close ensures all buffered data is flushed before closing the underlying log.

func (*WAL) CreateCheckpoint 

func (w *WAL) CreateCheckpoint() (uint64, error)

CreateCheckpoint creates a new LSN marker and manages checkpoint rotation. It ensures that only MaxSnapshotLimit checkpoints exist and truncates the WAL up to the oldest checkpoint's LSN when the limit is reached. We are locking in the CreateCheckpoint() function so we are not locking again in the truncate functon.

func (*WAL) Flush 

func (w *WAL) Flush() error

Flush manually triggers a write of all buffered events to the disk. Use this when you need to ensure data persistence before a specific operation.

func (*WAL) GetEvent 

func (w *WAL) GetEvent(lsn uint64) (*WALEntry, error)

GetEvent retrieves a single WALEntry by its LSN. It first checks the in-memory buffer (for unflushed entries), then falls back to reading from the on-disk WAL.

func (*WAL) GetLastFlushedLSN 

func (w *WAL) GetLastFlushedLSN() uint64

GetLastFlushedLSN returns the highest LSN that is guaranteed to be on disk.

func (*WAL) GetLastLSN 

func (w *WAL) GetLastLSN() uint64

GetLastLSN returns the most recently assigned LSN (may not be on disk yet).

func (*WAL) ReplayEvents 

func (w *WAL) ReplayEvents(handler func(entry WALEntry) error) error

ReplayEvents iterates through all events stored in the WAL from the beginning. For each event, it calls the provided handler function. This is the core mechanism for state hydration/recovery.

func (*WAL) ReplayEventsByType 

func (w *WAL) ReplayEventsByType(walType wal_types.WALType, handler func(entry WALEntry) error) error

ReplayEventsByType is a filtered version of ReplayEvents that only processes events matching the specified WALType.

func (*WAL) TruncateBefore 

func (w *WAL) TruncateBefore(lsn uint64) error

TruncateBefore eliminates all log entries with an LSN lower than the specified value. CAUTION: This is a destructive operation. Use only after state is safely checkpointed.

func (*WAL) WriteEvent 

func (w *WAL) WriteEvent(event wal_types.EventAdapter) (uint64, error)

WriteEvent accepts an EventAdapter, assigns it an LSN, and adds it to the buffer. If the buffer reaches BatchSize, it triggers an automatic flush to disk. Returns the assigned LSN or an error if serialization/flushing fails.

type WALEntry 

type WALEntry struct {
	// Type is the event type (HeaderSync, MerkleSync, etc.)
	Type wal_types.WALType `json:"type"`

	// Data is the serialized event data
	Data []byte `json:"data"`

	// LSN is the Log Sequence Number - a monotonically increasing counter
	// that uniquely identifies this event across all WAL files
	LSN uint64 `json:"lsn"`

	// Timestamp when the event was written
	Timestamp int64 `json:"timestamp,omitempty"`
}

WALEntry represents a single event in the WAL

Source Files

View all Source files

Directories

Path Synopsis

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.