sqlitestream

README

sqlitestream

Open a SQLite database with WAL PRAGMAs suitable for Litestream. When a replica URL is set, the file is restored from the replica if it is missing locally, and continuous replication runs in the background until Close.

Pure-Go: uses the modernc.org/sqlite driver, no cgo.

Install

go get github.com/luzilla/sqlitestream

API

func Open(ctx context.Context, path string, opts ...Option) (*DB, error)
func Restore(ctx context.Context, outputPath string, force bool, opts ...Option) error

func (*DB) DB() *sql.DB                    // underlying handle
func (*DB) Close(ctx context.Context) error // flushes final WAL, stops replication

func WithReplicaURL(url string) Option
func WithAWSCredentials(accessKeyID, secretAccessKey string) Option
func WithLogger(logger *slog.Logger) Option

Without WithReplicaURL, Open just opens the file in WAL mode and does no replication.

Replica URL

An S3 replica URL looks like:

s3://bucket/path?region=eu-central-1

For an S3-compatible endpoint add endpoint and forcePathStyle:

s3://bucket/path?endpoint=http://127.0.0.1:9000&region=eu-central-1&forcePathStyle=true

[!IMPORTANT] Pass credentials with WithAWSCredentials. Without them, Litestream falls back to the AWS default chain (env vars, shared config, instance role). Credentials in the URL userinfo (s3://key:secret@…) are not read.

Examples

• examples/plain — database/sql
• examples/entgo — Ent

Development

make test   # go test -race
make lint   # go vet + go fmt

The suite includes an S3 round-trip test that starts an S3 mock server (adobe/s3mock); it needs a running Docker daemon.

Documentation

Overview

Package sqlitestream opens a SQLite database with PRAGMAs suitable for Litestream and, when a replica URL is configured via WithReplicaURL, restores the file from the replica if it's missing locally and runs continuous replication in the background.

Index

• Variables
• func Restore(ctx context.Context, outputPath string, force bool, opts ...Option) error
• type DB
  • func Open(ctx context.Context, path string, opts ...Option) (*DB, error)
  • func (d *DB) Close(ctx context.Context) error
  • func (d *DB) DB() *sql.DB
  • func (d *DB) LastSyncedAt() time.Time
  • func (d *DB) Replicating() bool
  • func (d *DB) Retention() bool
  • func (d *DB) SyncStatus(ctx context.Context) (litestream.SyncStatus, error)
• type Option
  • func WithAWSCredentials(accessKeyID, secretAccessKey string) Option
  • func WithLogger(logger *slog.Logger) Option
  • func WithReplicaURL(url string) Option

Variables

var ErrExists = errors.New("sqlitestream: file exists")

ErrExists is returned by Restore when the output path already exists and force is false.

var ErrNotReplicating = errors.New("sqlitestream: replication not configured")

ErrNotReplicating is returned when an operation requires a configured replica but replication is off.

var ErrReplicaEmpty = errors.New("sqlitestream: replica is empty — nothing to restore")

ErrReplicaEmpty is returned by Restore when the replica holds no snapshots. Callers can treat this as a fresh install.

Functions

func Restore

func Restore(ctx context.Context, outputPath string, force bool, opts ...Option) error

Restore is a one-shot restore from the replica (configured via WithReplicaURL) into outputPath. It refuses to overwrite an existing file unless force is true; with force the .db, .db-wal and .db-shm files are removed first to give SQLite a clean slate. Used by ops scripts and the `firehose restore` subcommand.

Types

type DB

type DB struct {
	// contains filtered or unexported fields
}

DB is an open SQLite database, optionally with background Litestream replication.

Use SQL to access the underlying *sql.DB; call Close when done — Close flushes the final WAL segment to S3 if replication is on.

func Open

func Open(ctx context.Context, path string, opts ...Option) (*DB, error)

Open opens path with PRAGMAs suitable for Litestream. If WithReplicaURL is supplied, the database is restored from the replica when missing locally and background replication is started before the SQL handle is opened.

func (*DB) Close

func (d *DB) Close(ctx context.Context) error

Close flushes any pending WAL to S3 (when replication is on), stops the replicator, and closes the SQL handle.

Pass a fresh context — the daemon's main context is typically already cancelled by shutdown time, and a cancelled context aborts the flush.

func (*DB) DB

func (d *DB) DB() *sql.DB

DB returns the underlying *sql.DB for use with Ent or database/sql.

func (*DB) LastSyncedAt

func (d *DB) LastSyncedAt() time.Time

LastSyncedAt returns the time of the last successful sync to the replica. The zero time means replication is off or nothing has been synced yet.

func (*DB) Replicating

func (d *DB) Replicating() bool

Replicating reports whether background replication is configured.

func (*DB) Retention

func (d *DB) Retention() bool

Retention reports whether litestream is handling retention.

func (*DB) SyncStatus

func (d *DB) SyncStatus(ctx context.Context) (litestream.SyncStatus, error)

SyncStatus reports the status between the local database and the configured (cloud) storage by comparing transaction records. This may entail I/O. Returns ErrNotReplicating when no replica is configured.

type Option

type Option func(*config)

Option configures Open and Restore.

func WithAWSCredentials

func WithAWSCredentials(accessKeyID, secretAccessKey string) Option

WithCredentials sets credentials separately from the URL. Preferred over embedding "key:secret@" in the URL because URLs are easy to log or expose via process listings.

func WithLogger

func WithLogger(logger *slog.Logger) Option

WithLogger sets the slog.Logger used for replication and restore events.

func WithReplicaURL

func WithReplicaURL(url string) Option

WithReplicaURL configures the Litestream replica destination as a URL, e.g. "s3://bucket/path?region=us-east-1" or "file:///var/lib/replica". Empty disables replication entirely.