Documentation
¶
Overview ¶
Package gostgrator provides database-agnostic schema migration utilities for Go applications. It loads *.sql* migration files, tracks execution state in a table you choose, and moves the database forward or backward to any version you specify.
A thin driver layer (currently PostgreSQL and SQLite) supplies SQL dialect differences. Companion CLI tools live under sub-packages *pg* and *sqlite*; the core logic is here.
Install ¶
go get github.com/bcomnes/gostgrator/v2@latest
Quick start ¶
import (
"context"
"database/sql"
"os"
_ "github.com/jackc/pgx/v5/stdlib" // or modernc.org/sqlite
"github.com/bcomnes/gostgrator/v2"
)
func main() {
db, _ := sql.Open("pgx", os.Getenv("DATABASE_URL"))
client := gostgrator.NewPostgresClient(gostgrator.Config{}, db)
cfg := gostgrator.Config{
Migrations: gostgrator.DiskMigrations{Pattern: "migrations/*.sql"},
}
g, _ := gostgrator.NewGostgrator(cfg, client)
g.Migrate(context.Background(), "max")
}
Configuration ¶
Use Config to tweak behaviour:
- SchemaTable — pass to the client constructor (default "schemaversion")
- Migrations — DiskMigrations or FSMigrations for locating migration files
- Newline — line-ending normalization for migration checksums
- ValidateChecksums — compare MD5 hashes before running *up* migrations
NewGostgrator accepts a Client. NewPostgresClient and NewSqlite3Client select their dialect and apply client defaults. NewClient offers configuration-driven selection using Config.Driver.
Migration files ¶
A migration *pair* is two files with the same version and name:
001.do.create_users.sql // apply 001.undo.create_users.sql // roll back
Versions may be plain integers (*001*, *002*, …) or timestamps if you prefer. The CLI’s *new* command scaffolds these files for you.
Programmatic API ¶
NewGostgrator(cfg, client) → *Gostgrator, error (*Gostgrator).Migrate(ctx, v) → []Migration, error (*Gostgrator).Down(ctx, n) → []Migration, error (*Gostgrator).GetMigrations() → []Migration, error (*Gostgrator).GetDatabaseVersion(ctx) → int, error
All operations are context-aware; cancel the context to abort long runs.
CLI helpers ¶
If you prefer shell commands, install driver-specific binaries:
go get -tool github.com/bcomnes/gostgrator/v2/pg@latest # PostgreSQL go get -tool github.com/bcomnes/gostgrator/v2/sqlite@latest # SQLite
See each sub-package’s doc for flags and usage.
Exit codes ¶
The library returns errors; the CLIs exit with non-zero status on any failure. ValidateErrors include version and MD5 info for easy triage.
Versioning ¶
A semantic version string is exposed as:
var Version = "vX.Y.Z"
Embed it in your own commands to surface gostgrator’s build version.
Generated documentation; update whenever public API or CLI flags change.
Index ¶
- Variables
- func CreateMigration(cfg Config, description string, mode string) error
- type Client
- type Config
- type DiskMigrations
- type FSMigrations
- type Gostgrator
- func (g *Gostgrator) CreateMigration(description, mode string) error
- func (g *Gostgrator) Down(ctx context.Context, steps int) ([]Migration, error)
- func (g *Gostgrator) GetDatabaseVersion(ctx context.Context) (int, error)
- func (g *Gostgrator) GetMaxVersion() (int, error)
- func (g *Gostgrator) GetMigrations() ([]Migration, error)
- func (g *Gostgrator) GetRunnableMigrations(databaseVersion, targetVersion int) ([]Migration, error)
- func (g *Gostgrator) Migrate(ctx context.Context, target string) ([]Migration, error)
- func (g *Gostgrator) QueryContext(ctx context.Context, query string, args ...any) (*sql.Rows, error)
- func (g *Gostgrator) RunMigrations(ctx context.Context, migrations []Migration) ([]Migration, error)
- func (g *Gostgrator) ValidateMigrations(ctx context.Context, databaseVersion int) error
- type Migration
- type MigrationSource
- type PostgresClient
- func (c *PostgresClient) EnsureTable(ctx context.Context) error
- func (c *PostgresClient) ExecContext(ctx context.Context, script string, args ...any) (sql.Result, error)
- func (c *PostgresClient) GetDatabaseVersionSql() Statement
- func (c *PostgresClient) GetMd5Sql(m Migration) Statement
- func (c *PostgresClient) HasVersionTable(ctx context.Context) (bool, error)
- func (c *PostgresClient) PersistActionSql(m Migration) Statement
- func (c *PostgresClient) QueryContext(ctx context.Context, query string, args ...any) (*sql.Rows, error)
- type Sqlite3Client
- func (c *Sqlite3Client) EnsureTable(ctx context.Context) error
- func (c *Sqlite3Client) ExecContext(ctx context.Context, script string, args ...any) (sql.Result, error)
- func (c *Sqlite3Client) GetDatabaseVersionSql() Statement
- func (c *Sqlite3Client) GetMd5Sql(m Migration) Statement
- func (c *Sqlite3Client) HasVersionTable(ctx context.Context) (bool, error)
- func (c *Sqlite3Client) PersistActionSql(m Migration) Statement
- func (c *Sqlite3Client) QueryContext(ctx context.Context, query string, args ...any) (*sql.Rows, error)
- type Statement
Examples ¶
Constants ¶
This section is empty.
Variables ¶
var DefaultConfig = Config{ SchemaTable: "schemaversion", ValidateChecksums: true, }
DefaultConfig provides default values for configuration.
var (
Version = "2.0.2"
)
Functions ¶
func CreateMigration ¶
CreateMigration creates a new pair of migration files (do/undo) on disk. It accepts DiskMigrations or legacy MigrationPattern and rejects FSMigrations. description: a human-readable description that will be kebab-cased for the filename. mode: "int" for integer increment (default) or "timestamp" to use the Unix timestamp.
Types ¶
type Client ¶
type Client interface {
QueryContext(ctx context.Context, query string, args ...any) (*sql.Rows, error)
ExecContext(ctx context.Context, script string, args ...any) (sql.Result, error)
GetDatabaseVersionSql() Statement
HasVersionTable(ctx context.Context) (bool, error)
EnsureTable(ctx context.Context) error
GetMd5Sql(m Migration) Statement
PersistActionSql(m Migration) Statement
}
Client defines SQL execution and database-specific migration bookkeeping. Implementations can be supplied to NewGostgrator to support custom dialects or wrap an existing client. The client is responsible for its tracking table configuration; Gostgrator does not configure or close an injected client. SQL builders return statements with bound bookkeeping values. Execution methods must forward args to the driver; migration scripts are executed without args.
Example ¶
This example implements every Client method directly against database/sql. A new dialect supplies its own tracking-table DDL and bookkeeping queries.
package main
import (
"context"
"database/sql"
"fmt"
"testing/fstest"
"github.com/bcomnes/gostgrator/v2"
_ "modernc.org/sqlite"
)
// sqliteClient implements its own bookkeeping rather than wrapping a built-in client.
// Identifiers are fixed, quoted SQL; all variable values are bound parameters.
// It owns a fixed tracking table; it does not upgrade legacy tracking schemas.
type sqliteClient struct {
db *sql.DB
}
var _ gostgrator.Client = (*sqliteClient)(nil)
func (c *sqliteClient) QueryContext(ctx context.Context, query string, args ...any) (*sql.Rows, error) {
return c.db.QueryContext(ctx, query, args...)
}
func (c *sqliteClient) ExecContext(ctx context.Context, script string, args ...any) (sql.Result, error) {
return c.db.ExecContext(ctx, script, args...)
}
func (c *sqliteClient) GetDatabaseVersionSql() gostgrator.Statement {
return gostgrator.Statement{SQL: `SELECT COALESCE(MAX(version), 0) FROM "example_migrations"`}
}
func (c *sqliteClient) HasVersionTable(ctx context.Context) (bool, error) {
var exists bool
err := c.db.QueryRowContext(ctx, `
SELECT EXISTS (
SELECT 1 FROM sqlite_master
WHERE type = ? AND name = ?
)
`, "table", "example_migrations").Scan(&exists)
return exists, err
}
func (c *sqliteClient) EnsureTable(ctx context.Context) error {
_, err := c.db.ExecContext(ctx, `
CREATE TABLE IF NOT EXISTS "example_migrations" (
version INTEGER PRIMARY KEY,
name TEXT NOT NULL,
md5 TEXT NOT NULL,
run_at TEXT NOT NULL DEFAULT CURRENT_TIMESTAMP
)
`)
return err
}
func (c *sqliteClient) GetMd5Sql(m gostgrator.Migration) gostgrator.Statement {
return gostgrator.Statement{SQL: `SELECT md5 FROM "example_migrations" WHERE version = ?`, Args: []any{m.Version}}
}
func (c *sqliteClient) PersistActionSql(m gostgrator.Migration) gostgrator.Statement {
if m.Action == "undo" {
return gostgrator.Statement{SQL: `DELETE FROM "example_migrations" WHERE version = ?`, Args: []any{m.Version}}
}
return gostgrator.Statement{
SQL: `INSERT INTO "example_migrations" (version, name, md5) VALUES (?, ?, ?)`,
Args: []any{m.Version, m.Name, m.Md5},
}
}
// This example implements every Client method directly against database/sql.
// A new dialect supplies its own tracking-table DDL and bookkeeping queries.
func main() {
db, err := sql.Open("sqlite", ":memory:")
if err != nil {
panic(err)
}
defer db.Close() // The caller owns the connection, not Gostgrator.
db.SetMaxOpenConns(1)
client := &sqliteClient{db: db}
g, err := gostgrator.NewGostgrator(gostgrator.Config{
Migrations: gostgrator.FSMigrations{
FS: fstest.MapFS{
"001.do.create-items.sql": {Data: []byte("CREATE TABLE items (id INTEGER PRIMARY KEY);")},
"001.undo.create-items.sql": {Data: []byte("DROP TABLE items;")},
},
Pattern: "*.sql",
},
ValidateChecksums: true,
}, client)
if err != nil {
panic(err)
}
ctx := context.Background()
applied, err := g.Migrate(ctx, "max")
if err != nil {
panic(err)
}
fmt.Printf("Applied %d migration\n", len(applied))
// Re-running validates the stored checksum without applying anything again.
applied, err = g.Migrate(ctx, "max")
if err != nil {
panic(err)
}
fmt.Printf("Applied %d migrations on rerun\n", len(applied))
undone, err := g.Migrate(ctx, "0")
if err != nil {
panic(err)
}
fmt.Printf("Undid %d migration\n", len(undone))
version, err := g.GetDatabaseVersion(ctx)
if err != nil {
panic(err)
}
fmt.Printf("Database version: %d\n", version)
}
Output: Applied 1 migration Applied 0 migrations on rerun Undid 1 migration Database version: 0
func NewClient ¶
NewClient creates a new Client based on the provided configuration and database connection.
func NewPostgresClient ¶
NewPostgresClient creates a PostgreSQL client using the supplied database. SchemaTable defaults to DefaultConfig.SchemaTable. The constructor selects the PostgreSQL dialect regardless of Config.Driver; the caller owns db. SchemaTable is an unquoted table or schema.table name. Each dot-separated component is quoted independently; embedded double quotes are escaped.
func NewSqlite3Client ¶
NewSqlite3Client creates a SQLite client using the supplied database. SchemaTable defaults to DefaultConfig.SchemaTable. The constructor selects the SQLite dialect regardless of Config.Driver; the caller owns db. SchemaTable is quoted as one literal table name, including any dots; embedded double quotes are escaped. It is not an attached-database qualifier.
type Config ¶
type Config struct {
// Driver selects the SQL dialect: "pg", "sqlite", or the legacy alias "sqlite3".
// It is used by NewClient, not by NewGostgrator or dialect-specific constructors.
Driver string `json:"driver,omitempty"`
// SchemaTable is the name of the migration table for built-in clients.
// Pass it to the client constructor; NewGostgrator does not configure the client.
SchemaTable string `json:"schemaTable,omitempty"`
// Migrations selects an explicit disk or filesystem migration source.
// It must not be combined with MigrationPattern.
Migrations MigrationSource `json:"-"`
// MigrationPattern is the legacy disk glob pattern (e.g. "./migrations/*.sql").
// It is used when Migrations is nil by the legacy APIs.
// NewGostgrator rejects this field; use Migrations instead.
MigrationPattern string `json:"migrationPattern,omitempty"`
// Newline is the desired newline style ("LF", "CR", or "CRLF").
Newline string `json:"newline,omitempty"`
// ValidateChecksums indicates if the tool should validate migration checksums.
ValidateChecksums bool `json:"validateChecksums,omitempty"`
// Conn is the connection string used by the CLI.
// Library constructors do not open a connection or use this field.
Conn string `json:"conn,omitempty"`
}
Config holds settings for migrations.
type DiskMigrations ¶
type DiskMigrations struct {
// Pattern is a filepath.Glob pattern, which may be absolute or relative.
Pattern string
}
DiskMigrations selects migration files using OS filesystem paths.
type FSMigrations ¶
type FSMigrations struct {
// FS contains the migration files and must not be nil.
FS fs.FS
// Pattern is an fs.Glob pattern relative to FS, using forward slashes.
// It must not contain empty, dot, or dot-dot path components.
Pattern string
}
FSMigrations selects migration files from a filesystem, including embed.FS. The filesystem is only read; migration creation is not supported.
type Gostgrator ¶
type Gostgrator struct {
// contains filtered or unexported fields
}
Gostgrator is the main orchestrator for running database migrations.
It loads migration files, determines the current database version, validates checksums (if enabled), and runs the necessary migrations to reach a target version.
func NewGostgrator ¶
func NewGostgrator(cfg Config, client Client) (*Gostgrator, error)
NewGostgrator creates a Gostgrator using a caller-provided Client. Use NewPostgresClient or NewSqlite3Client for built-in database support, or supply a custom implementation. The client owns SQL execution and migration tracking-table behavior; the caller is responsible for configuring it and managing its database resources. Config.Driver, Config.Conn, and Config.SchemaTable do not configure or override the supplied client. Use Config.Migrations to select migration files; a nonempty Config.MigrationPattern is rejected. A nil client, including a typed nil implementation, is rejected.
Example (CustomClient) ¶
package main
import (
"context"
"database/sql"
"fmt"
"log"
"os"
"testing/fstest"
"github.com/bcomnes/gostgrator/v2"
_ "modernc.org/sqlite"
)
type loggingClient struct {
gostgrator.Client
logger *log.Logger
}
var _ gostgrator.Client = (*loggingClient)(nil)
func (c *loggingClient) ExecContext(ctx context.Context, script string, args ...any) (sql.Result, error) {
result, err := c.Client.ExecContext(ctx, script, args...)
c.logger.Printf("SQL execution succeeded: %t", err == nil)
return result, err
}
func main() {
db, err := sql.Open("sqlite", ":memory:")
if err != nil {
panic(err)
}
defer db.Close()
db.SetMaxOpenConns(1)
client := &loggingClient{
Client: gostgrator.NewSqlite3Client(gostgrator.Config{
SchemaTable: "app_migrations",
}, db),
logger: log.New(os.Stdout, "", 0),
}
// Use an in-memory filesystem to make the example self-contained.
// DiskMigrations or an embed.FS can be used with the same client.
migrations := fstest.MapFS{
"001.do.create-items.sql": {Data: []byte("CREATE TABLE items (id INTEGER PRIMARY KEY);")},
}
g, err := gostgrator.NewGostgrator(gostgrator.Config{
Migrations: gostgrator.FSMigrations{FS: migrations, Pattern: "*.sql"},
}, client)
if err != nil {
panic(err)
}
applied, err := g.Migrate(context.Background(), "max")
if err != nil {
panic(err)
}
fmt.Printf("Applied %d migration\n", len(applied))
}
Output: SQL execution succeeded: true SQL execution succeeded: true Applied 1 migration
func (*Gostgrator) CreateMigration ¶
func (g *Gostgrator) CreateMigration(description, mode string) error
(Optional) If you prefer to expose this functionality as a method on Gostgrator, you can add the following method.
func (*Gostgrator) Down ¶
Down rolls back the migrations by the given number of steps. It computes the target version as the current version minus steps (not going below zero), and then calls Migrate to perform the undo operations.
func (*Gostgrator) GetDatabaseVersion ¶
func (g *Gostgrator) GetDatabaseVersion(ctx context.Context) (int, error)
GetDatabaseVersion returns the current database version. If the migration table is not initialized, it returns 0.
func (*Gostgrator) GetMaxVersion ¶
func (g *Gostgrator) GetMaxVersion() (int, error)
GetMaxVersion returns the highest migration version available.
func (*Gostgrator) GetMigrations ¶
func (g *Gostgrator) GetMigrations() ([]Migration, error)
func (*Gostgrator) GetRunnableMigrations ¶
func (g *Gostgrator) GetRunnableMigrations(databaseVersion, targetVersion int) ([]Migration, error)
func (*Gostgrator) Migrate ¶
Migrate moves the schema to the target version. If target is "max" or empty, it migrates to the highest available version.
func (*Gostgrator) QueryContext ¶
func (g *Gostgrator) QueryContext(ctx context.Context, query string, args ...any) (*sql.Rows, error)
QueryContext executes a query with bound args using the underlying client.
func (*Gostgrator) RunMigrations ¶
func (g *Gostgrator) RunMigrations(ctx context.Context, migrations []Migration) ([]Migration, error)
RunMigrations applies the provided migrations in sequence.
func (*Gostgrator) ValidateMigrations ¶
func (g *Gostgrator) ValidateMigrations(ctx context.Context, databaseVersion int) error
ValidateMigrations verifies that applied migrations have not changed by comparing MD5 checksums.
type Migration ¶
type Migration struct {
// Version of the migration.
Version int
// Action, e.g., "do" or "undo".
Action string
// Filename is the OS path, or the path relative to the source filesystem.
Filename string
// Name is an optional descriptive name of the migration.
Name string
// Md5 is the MD5 checksum of the migration file.
Md5 string
// contains filtered or unexported fields
}
Migration represents a single migration file. Values are comparable, but equality includes the identity of the retained filesystem source. Independently loaded filesystem migrations may compare unequal despite identical metadata; compare the relevant public fields instead when checking migration identity or content.
type MigrationSource ¶
type MigrationSource interface {
// contains filtered or unexported methods
}
MigrationSource selects a disk or filesystem migration source. Use DiskMigrations or FSMigrations, as a value or a non-nil pointer.
type PostgresClient ¶
type PostgresClient struct {
// contains filtered or unexported fields
}
PostgresClient implements the Client interface for PostgreSQL.
func (*PostgresClient) EnsureTable ¶
EnsureTable creates the migration table if it does not exist and adds missing columns.
func (*PostgresClient) ExecContext ¶
func (c *PostgresClient) ExecContext(ctx context.Context, script string, args ...any) (sql.Result, error)
Exposing ExecContext from the configured db connection.
func (*PostgresClient) GetDatabaseVersionSql ¶
func (c *PostgresClient) GetDatabaseVersionSql() Statement
GetDatabaseVersionSql returns a statement to fetch the highest applied version.
func (*PostgresClient) GetMd5Sql ¶
GetMd5Sql returns a statement to fetch the checksum for a bound migration version.
func (*PostgresClient) HasVersionTable ¶
HasVersionTable checks for the existence of the migration table.
func (*PostgresClient) PersistActionSql ¶
PersistActionSql generates a statement to record a migration action.
type Sqlite3Client ¶
type Sqlite3Client struct {
// contains filtered or unexported fields
}
Sqlite3Client implements the Client interface for SQLite.
func (*Sqlite3Client) EnsureTable ¶
EnsureTable creates the migration table if it does not exist and adds missing columns.
func (*Sqlite3Client) ExecContext ¶
func (c *Sqlite3Client) ExecContext(ctx context.Context, script string, args ...any) (sql.Result, error)
Exposing ExecContext from the configured db connection.
func (*Sqlite3Client) GetDatabaseVersionSql ¶
func (c *Sqlite3Client) GetDatabaseVersionSql() Statement
GetDatabaseVersionSql returns a statement to fetch the highest applied version.
func (*Sqlite3Client) GetMd5Sql ¶
GetMd5Sql returns a statement to fetch the checksum for a bound migration version.
func (*Sqlite3Client) HasVersionTable ¶
HasVersionTable checks for the existence of the migration table.
func (*Sqlite3Client) PersistActionSql ¶
PersistActionSql generates a statement to record a migration action.
type Statement ¶ added in v2.0.2
type Statement struct {
// SQL is the query text, with placeholders matching the client's dialect.
SQL string
// Args contains values forwarded to QueryContext or ExecContext.
// It may be nil when SQL requires no bound arguments.
Args []any
}
Statement contains SQL text and the arguments to bind when executing it. It is not a prepared statement (sql.Stmt) and holds no database resources. Placeholders are dialect-specific; identifiers must be quoted by the client, not passed in Args.
Source Files
¶
Directories
¶
| Path | Synopsis |
|---|---|
|
Package main provides gostgrator‑pg, a PostgreSQL‑specific command‑line interface for the gostgrator migration library.
|
Package main provides gostgrator‑pg, a PostgreSQL‑specific command‑line interface for the gostgrator migration library. |
|
Package main provides gostgrator‑sqlite, a SQLite‑specific command‑line interface for the gostgrator migration library.
|
Package main provides gostgrator‑sqlite, a SQLite‑specific command‑line interface for the gostgrator migration library. |

