pglite

README

pglite-go

Embedded PostgreSQL for Go — no external server, no libpq, no separate process.

It runs PGlite (real PostgreSQL 18 compiled to WebAssembly) on the wasmtime runtime, with a hand-written Emscripten/WASI host layer and an in-memory virtual filesystem. Queries run over the actual PostgreSQL v3 wire protocol against one persistent in-process backend, so you get typed results, RowsAffected, server-side prepared statements, and transactions — through the standard database/sql API.

import (
    "database/sql"
    _ "github.com/moriyoshi/pglite-go/pgdriver"
)

db, _ := sql.Open("pglite", "dir=./wasm")
db.SetMaxOpenConns(1) // one embedded backend = one connection
defer db.Close()

db.Exec(`CREATE TABLE users (id int primary key, name text)`)
db.Exec(`INSERT INTO users VALUES ($1, $2)`, 1, "alice")

var name string
db.QueryRow(`SELECT name FROM users WHERE id = $1`, 1).Scan(&name) // "alice"

Status

A working embedded database, not a toy: the database/sql driver passes CRUD, parameterized queries, NULLs, RowsAffected, and Begin/Commit/Rollback under -race, and jmoiron/sqlx works unchanged. It began as a proof of concept; see JOURNAL.md for the full development history (runtime benchmarks, the wire-protocol reverse-engineering, perf work).

Requirements

• Go 1.25+ with CGo enabled (the default). wasmtime-go vendors a prebuilt libwasmtime, so there is nothing else to install.

• The PGlite wasm assets (pglite.wasm, initdb.wasm, pglite.data, pglite.manifest.json). They are not committed; obtain them in whichever way suits your build:

  go generate ./...              # fetch the pinned release from jsDelivr -> ./wasm
  

  go generate runs a small Go fetcher (internal/fetchwasm, no shell/python deps) that downloads the version this library targets (pglite.PgliteVersion); set PGLITE_VERSION for a deliberate bump.

  Then choose how the binary gets the assets:

  Approach	How	Trade-off
  Directory (default)	ship wasm/ beside the binary, or WasmDir/dir=	assets on disk at runtime
  Embed	go generate ./... && go build -tags embed	self-contained binary (+~16 MB)
  On-demand	Config{Download: true} / download=true	no assets shipped; fetches from jsDelivr into the user cache on first run (needs network)
  Your own fs.FS	Config{WasmFS: myEmbedFS}	embed assets you fetched in your own app

Usage

database/sql (+ sqlx)

import (
    "database/sql"
    _ "github.com/moriyoshi/pglite-go/pgdriver"

    "github.com/jmoiron/sqlx"
)

db, _ := sql.Open("pglite", "dir=./wasm database=template1")
db.SetMaxOpenConns(1)

// sqlx wraps any database/sql driver:
dbx := sqlx.NewDb(db, "pglite")
var users []User
dbx.Select(&users, `SELECT id, name FROM users WHERE active = $1`, true)

DSN is a space-separated list of key=value pairs:

key	meaning	default
dir	directory holding the wasm assets	wasm
database	database to connect to	template1
persist	host dir to save/restore the cluster	user cache dir
ephemeral	keep the cluster only in memory (true)	false
download	fetch wasm assets from jsDelivr if no local source (true)	false

Lower-level library

import pglite "github.com/moriyoshi/pglite-go"

db, _ := pglite.Open(pglite.Config{WasmDir: "wasm"})
defer db.Close()

rows, _ := db.Query("SELECT 1 + 1 AS n, 'hi'::text AS s")
// rows.Columns, rows.TypeOIDs, rows.Rows ([][]any of typed Go values), rows.AffectedRows

// Server-side prepared statement (extended protocol):
rows, _ = db.QueryParams("SELECT $1::int + $2::int", []string{"2", "3"}, []bool{false, false})

How it works

• Backend: Open initializes a data directory (initdb) or loads a persisted one, keeps it in an in-memory VFS, and starts a single PostgreSQL backend driven over PGlite's pgl_set_rw_cbs + PostgresMainLoopOnce wire-protocol hooks.
• Queries feed v3 protocol messages to that live backend and decode the responses (RowDescription, DataRow, CommandComplete, …). Session state — the open transaction, temp tables, SET, prepared statements — persists across calls.
• Persistence: on close (or Sync) the cluster is saved to the host under the cache dir; the next Open loads it and skips initdb. Compiled native code is cached to a .cwasm on disk. So a warm start is a fraction of a second.

Performance

On an idle x86-64 Mac (see JOURNAL.md for methodology):

First run (compile + initdb + persist)	~2.7 s
Warm start (.cwasm cache + skip initdb)	~0.2 s
Query latency (warm, simple SELECT)	~150 µs

Limitations

• Single connection. The embedded backend is one connection (as in PGlite); use db.SetMaxOpenConns(1). The library serializes internally.
• Result types. Common numeric/bool/bytea columns are received in binary format and decoded to typed Go values (int64, float64, bool, []byte); other types arrive as text strings, which Scan converts as usual. Binary applies to parameterized (extended-protocol) queries; plain Query uses the text protocol.
• No COPY / LISTEN/NOTIFY yet.
• CGo by default; pure-Go optional. The default build runs on wasmtime (CGo). A pure-Go build (-tags wazero, CGO_ENABLED=0) runs the same library and wire protocol on the wazero runtime — no driver/feature differences — but is considerably slower (notably a ~5 min cold compile of pglite.wasm; warm starts are cached). database/sql support still needs CGo, since the driver depends on it only through the default backend.

Layout

.                     pglite package — Open/Query/Exec/QueryParams (the library)
                      backend_wasmtime.go / backend_wazero.go select the runtime
pgdriver/             database/sql driver, registered as "pglite"
emscripten/           Emscripten + WASI host layer + wire hooks
                      wasmtime_port.go (default) / wazero_port.go (-tags wazero)
vfs/                  in-memory virtual filesystem
cmd/pglite/           demo using the library (builds on both backends)
internal/fetchwasm/   `go generate` fetcher: pulls the PGlite wasm assets from jsDelivr

Credits

Built on PGlite by ElectricSQL and wasmtime by the Bytecode Alliance.

License

Licensed under the Apache License, Version 2.0. See NOTICE for third-party attributions (PGlite, wasmtime, wazero) redistributed with this work.

Documentation

Overview

Package pglite embeds PostgreSQL in Go by running PGlite (PostgreSQL compiled to WebAssembly) on a WebAssembly runtime — wasmtime by default, or pure-Go wazero with -tags wazero — with a hand-written Emscripten/WASI host layer and an in-memory VFS.

A DB is a cluster: Open initializes (or loads a persisted) data directory, keeps it in an in-memory filesystem, and starts one persistent PostgreSQL backend driven over the v3 wire protocol (PGlite's pgl_set_rw_cbs + PostgresMainLoopOnce). Query/Exec feed protocol messages to that live backend, so results carry column type OIDs and command tags (RowsAffected), and session state — the current transaction, temp tables, SET, prepared statements — persists across calls: BEGIN in one Query and COMMIT in a later one form a single transaction. QueryParams runs a real server-side prepared statement (extended protocol). For the standard Go database API, use the registered "pglite" database/sql driver (package pgdriver).

The backend is a single connection (as in PGlite itself), so concurrent use must be serialized — the DB does this internally, and database/sql callers should set db.SetMaxOpenConns(1).

Index

• Constants
• type Config
• type DB
  • func Open(cfg Config) (*DB, error)
  • func (db *DB) Close() error
  • func (db *DB) Exec(sql string) (string, error)
  • func (db *DB) Query(sql string) (*Rows, error)
  • func (db *DB) QueryParams(sql string, params []string, isNull []bool) (*Rows, error)
  • func (db *DB) ServeConn(conn net.Conn) error
  • func (db *DB) Sync() error
• type Rows

Constants

const PgliteVersion = pgassets.Version

PgliteVersion is the pinned PGlite release the artifacts must match.

Types

type Config

type Config struct {
	// WasmFS, if set, supplies the wasm artifacts (pglite.wasm, initdb.wasm,
	// pglite.data, pglite.manifest.json) from any io/fs — e.g. a caller-provided
	// embed.FS. Takes precedence over WasmDir. See resolveWasmFS for the full
	// source-selection order.
	WasmFS fs.FS
	// WasmDir holds the wasm artifacts on the host filesystem. Used when WasmFS
	// is nil. Empty falls back to embedded assets (-tags embed), then "./wasm",
	// then an on-demand download when Download is set.
	WasmDir string
	// Download allows fetching the pinned artifacts from the jsDelivr CDN into the
	// user cache dir when no local/embedded source is available. Off by default so
	// Open never performs surprise network I/O.
	Download bool
	// PersistDir is the host directory the initialized cluster is saved to and
	// loaded from. Empty uses the user cache dir; set Ephemeral to disable.
	PersistDir string
	// Ephemeral, if true, keeps the cluster only in memory (no host persistence).
	Ephemeral bool
	// Database to connect to. Defaults to "template1".
	Database string
}

Config configures a DB. The zero value is usable: it looks for wasm assets in "./wasm", persists the cluster under the user cache dir, and uses template1.

type DB

type DB struct {
	// contains filtered or unexported fields
}

DB is an embedded PostgreSQL cluster backed by an in-memory filesystem.

A DB is safe for concurrent use: query execution is serialized (the WASM backend is single-threaded and the in-memory cluster is shared), so callers (including database/sql's pool) block rather than race.

func Open

func Open(cfg Config) (*DB, error)

Open initializes or loads a cluster and returns a ready DB.

func (*DB) Close

func (db *DB) Close() error

Close ends the backend session, syncs the cluster to the host, and releases resources.

func (*DB) Exec

func (db *DB) Exec(sql string) (string, error)

Exec runs sql for its side effects and returns the command tag (e.g. "INSERT 0 1"). Use the Rows.AffectedRows from Query for a numeric count.

func (*DB) Query

func (db *DB) Query(sql string) (*Rows, error)

Query runs sql via the simple query protocol on the persistent backend and returns its rows. Session state (open transaction, temp tables, SET) persists across calls, so BEGIN in one Query and COMMIT in a later one form a single transaction.

func (*DB) QueryParams

func (db *DB) QueryParams(sql string, params []string, isNull []bool) (*Rows, error)

QueryParams runs sql with bind parameters via the extended query protocol (a real server-side prepared statement). Values are given as their text encodings; a true entry in isNull sends SQL NULL for that position.

func (*DB) ServeConn

func (db *DB) ServeConn(conn net.Conn) error

ServeConn speaks the PostgreSQL v3 frontend/backend protocol to a single client over conn, relaying the client's messages to the embedded backend and the backend's responses back to the client. It lets ordinary PostgreSQL clients (psql, pgx, lib/pq, JDBC, …) talk to the in-process cluster over a real socket — e.g. in an Accept loop:

ln, _ := net.Listen("tcp", "127.0.0.1:5432")
for {
	c, err := ln.Accept()
	if err != nil {
		return err
	}
	go db.ServeConn(c)
}

The embedded cluster has exactly one persistent backend session, so ServeConn does not open a new one per client: it answers the startup handshake itself (AuthenticationOk, a synthetic set of ParameterStatus values, BackendKeyData, ReadyForQuery — authentication is always trust) and then shuttles protocol messages to and from that single shared session. The SSL/GSS negotiation and StartupMessage are handled before the backend lock is taken, so a concurrently connecting client can finish its handshake; from the startup reply onward ServeConn holds the DB's backend exclusively for the rest of conn's lifetime, so other ServeConn calls and Query/Exec/QueryParams block there until the served client disconnects.

A client Terminate ('X') ends only that client; it is never forwarded to the shared backend. ServeConn closes conn before returning.

func (*DB) Sync

func (db *DB) Sync() error

Sync persists the in-memory cluster to the host (no-op when Ephemeral).

type Rows

type Rows struct {
	Columns  []string // column names, in order
	TypeOIDs []uint32 // PostgreSQL type OID per column
	// Rows is row-major, one []any per row. Each value is a decoded Go value —
	// int64, float64, bool, []byte, string, or nil (SQL NULL) — chosen by the
	// column's type OID. Common numeric/bool/bytea types are received in binary
	// format (extended-protocol queries); everything else arrives as text and is
	// returned as a string.
	Rows         [][]any
	Command      string // CommandComplete tag, e.g. "INSERT 0 1", "SELECT 3"
	AffectedRows int64  // rows affected/returned, parsed from Command
}

Rows is the structured result of a query, decoded from the PostgreSQL wire protocol.