litefs

README

LiteFS Go Library

This Go library is for interacting with LiteFS features that cannot be accessed through the typical SQLite API.

Halting

LiteFS provides the ability to halt writes on the primary node in order that replicas may execute writes remotely and forward them back to the primary. This isn't necessary in most usage, however, it can make running migrations simpler.

Write forwarding from the replica is much slower than executing the write transaction directly on the primary so only use this for migrations or low-write scenarios.

// Initialize the database.
db, err := sql.Open("sqlite3", "/litefs/my.db")
if err != nil {
	return err
}
defer db.Close()

// Execute a write transaction from any node.
// If this is a replica, it will run the inner function with the HALT lock.
if err := litefs.WithHalt("/litefs/my.db", func() error {
	_, err := db.Exec(`CREATE TABLE users (id INTEGER PRIMARY KEY, name string)`)
	return err
}); err != nil {
	return err
}

Documentation

Index

• Constants
• Variables
• func Halt(f *os.File) error
• func Lag(databasePath string) (time.Duration, error)
• func Unhalt(f *os.File) error
• func WithHalt(databasePath string, fn func() error) error
• type Client
  • func (c *Client) MonitorPrimary() PrimaryMonitor
  • func (c *Client) SubscribeEvents() EventSubscription
• type Event
  • func (e *Event) UnmarshalJSON(data []byte) error
• type EventSubscription
• type InitEventData
• type PrimaryChangeEventData
• type PrimaryMonitor
• type TxEventData

Constants

const (
	EventTypeInit          = "init"
	EventTypeTx            = "tx"
	EventTypePrimaryChange = "primaryChange"
)

const (
	F_OFD_GETLK  = 36
	F_OFD_SETLK  = 37
	F_OFD_SETLKW = 38
)

Open file description lock constants.

const (
	HaltByte = 72
)

LiteFS lock offsets

Variables

var DefaultClient = &Client{
	URL:  "http://localhost:20202",
	HTTP: http.DefaultClient,
}

DefaultClient is a client for communicating with the default (localhost:20202) LiteFS node.

var (
	ErrClosed = errors.New("closed EventSubscription")
)

var (
	ErrNotReady = errors.New("awaiting first event")
)

Functions

func Halt

func Halt(f *os.File) error

Halt locks the HALT lock on the file handle to the LiteFS database lock file. This causes writes to be halted on the primary node so that this replica can perform writes until Unhalt() is invoked.

The HALT lock will automatically be released when the file descriptor is closed.

This function is a no-op if the current node is the primary.

func Lag

func Lag(databasePath string) (time.Duration, error)

Lag reports the how far this node is lagging behind the primary. The lag is 0 on the primary node. In the absence of new transactions from the primary, heartbeat messages are sent at one second intervals.

func Unhalt

func Unhalt(f *os.File) error

Unhalt releases the HALT lock on the file handle to the LiteFS database lock file. This allows writes to resume on the primary node. This replica will not be able to perform any more writes until Halt() is called again.

This function is a no-op if the current node is the primary or if the lock expired.

func WithHalt

func WithHalt(databasePath string, fn func() error) error

WithHalt executes fn with a HALT lock. This allows any node to perform writes on the database file. If this is a replica node, it will forward those writes back to the primary node. If this is the primary node, it will simply execute fn since it does not need the HALT lock.

Please note that this will stop all writes on the primary. It is slower than processing a write directly on the primary since it requires two round trips to acquire & release the lock.

This function should only be used for periodic migrations or low-write scenarios.

Types

type Client

type Client struct {
	// Base URL of the LiteFS cluster node.
	URL string

	// HTTP client to use for requests to cluster node.
	HTTP *http.Client
}

Client is an HTTP client for communicating with a LiteFS node.

func (*Client) MonitorPrimary

func (c *Client) MonitorPrimary() PrimaryMonitor

MonitorPrimary monitors the primary status of the LiteFS cluster via the LiteFS node's event stream.

func (*Client) SubscribeEvents

func (c *Client) SubscribeEvents() EventSubscription

SubscribeEvents subscribes to events from the LiteFS node.

type Event

type Event struct {
	Type string `json:"type"`
	DB   string `json:"db,omitempty"`
	Data any    `json:"data,omitempty"`
}

Event represents a generic event.

func (*Event) UnmarshalJSON

func (e *Event) UnmarshalJSON(data []byte) error

type EventSubscription

type EventSubscription interface {
	// Next attempts to read the next event from the LiteFS node. An error is
	// returned if the request fails. Calling `Next()` again after an error will
	// initiate a new HTTP request. ErrClosed is returned if the EventSubscription
	// is closed while this method is blocking.
	Next() (*Event, error)

	// Close aborts any in-progress requests to the LiteFS node.
	Close()
}

EventSubscription monitors a LiteFS node for published events.

type InitEventData

type InitEventData struct {
	IsPrimary bool   `json:"isPrimary"`
	Hostname  string `json:"hostname,omitempty"`
}

type PrimaryChangeEventData

type PrimaryChangeEventData struct {
	IsPrimary bool   `json:"isPrimary"`
	Hostname  string `json:"hostname,omitempty"`
}

type PrimaryMonitor

type PrimaryMonitor interface {
	// WaitReady blocks until ctx expires or a response or error has been received
	// from the local LiteFS node. IsPrimary and Hostname will return errors until
	// this method returns nil.
	WaitReady(ctx context.Context) error

	// IsPrimary reports whether the local node is primary node in the cluster. An
	// error is returned if this method is called before data has been received
	// from the local LiteFS node (see WaitReady). If an error is encountered while
	// communicating with the node, that error, along with the most recent
	// IsPrimary value will be returned.
	IsPrimary() (bool, error)

	// Hostname reports the name of the current primary node in the cluster. An
	// error is returned if this method is called before data has been received
	// from the local LiteFS node (see WaitReady). If an error is encountered while
	// communicating with the node, that error, along with the most recent
	// Hostname value will be returned.
	Hostname() (string, error)

	// Close unsubscribes to the LiteFS node's event stream.
	Close()
}

PrimaryMonitor monitors the current primary status of the LiteFS cluster.

type TxEventData

type TxEventData struct {
	TXID              string    `json:"txID"`              // ltx.TXID
	PostApplyChecksum string    `json:"postApplyChecksum"` // ltx.Checksum
	PageSize          uint32    `json:"pageSize"`
	Commit            uint32    `json:"commit"`
	Timestamp         time.Time `json:"timestamp"`
}