gobase

README

gobase

A lightweight database migration library for Go, inspired by Liquibase. Uses only the Go standard library.

Features

• XML Changelog Format — Define migrations in a simple XML structure
• Checksum Validation — Detects if applied changesets were modified (prevents silent schema drift)
• Preconditions — Conditionally execute changesets based on SQL queries
• Rollback Support — Revert migrations to a specific changeset
• Transaction Safety — Each changeset executes within a transaction
• Idempotent — Safe to run multiple times; already-applied changes are skipped
• Migration Runners — Write complex migrations in Go when SQL isn't enough

Philosophy

gobase is designed with simplicity and zero dependencies in mind:

• Standard Library Only — No external dependencies beyond Go's standard library
• Basic Functionality — Features are added only as needed, keeping the codebase minimal and maintainable
• Database Agnostic — Works with any database that supports Go's database/sql interface
• Progressive Enhancement — Start simple with SQL migrations, add Go runners when needed

This project is open source and welcomes contributions that align with these principles.

Installation

go get github.com/mainvec/gobase

Usage

Changelog XML Format

Create a changelog file (e.g., changelog.xml):

<gobase changelogtable="gobase_dbchangelog">
    <change author="dev" id="001">
        <comment>Create users table</comment>
        <sql>
            CREATE TABLE users (
                id INTEGER PRIMARY KEY,
                email VARCHAR(255) NOT NULL,
                created_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP
            )
        </sql>
        <rollback>DROP TABLE users</rollback>
    </change>
    
    <change author="dev" id="002">
        <comment>Create posts table (only if users table has data)</comment>
        <cond>
            <sql>SELECT 1 FROM users LIMIT 1</sql>
        </cond>
        <sql>
            CREATE TABLE posts (
                id INTEGER PRIMARY KEY,
                user_id INTEGER NOT NULL,
                title VARCHAR(255),
                FOREIGN KEY (user_id) REFERENCES users(id)
            )
        </sql>
        <rollback>DROP TABLE posts</rollback>
    </change>
</gobase>

Running Migrations

package main

import (
    "context"
    "database/sql"
    "log"
    "os"

    "github.com/mainvec/gobase"
    _ "your/database/driver" // e.g., github.com/mattn/go-sqlite3
)

func main() {
    db, err := sql.Open("sqlite3", "app.db")
    if err != nil {
        log.Fatal(err)
    }
    defer db.Close()

    // Open changelog file
    changelog, err := os.Open("changelog.xml")
    if err != nil {
        log.Fatal(err)
    }
    defer changelog.Close()

    // Run migrations
    ctx := context.Background()
    if err := gobase.RunMigration(ctx, db, changelog); err != nil {
        log.Fatal(err)
    }

    log.Println("Migrations completed successfully")
}

Rolling Back Migrations

// Rollback to a specific changeset (exclusive — keeps that changeset, removes later ones)
changelog, _ := os.Open("changelog.xml")
err := gobase.Rollback(ctx, db, changelog, "001")

// Rollback all changesets
changelog, _ := os.Open("changelog.xml")
err := gobase.Rollback(ctx, db, changelog, "")

Changelog Elements

Element	Description
<gobase changelogtable="...">	Root element. changelogtable specifies the tracking table name (default: gobase_dbchangelog)
<change id="..." author="...">	A single migration. id + author must be unique
<sql>	Inline SQL to execute
<sqlFile path="..." rollback="...">	Execute SQL from an embedded file (requires RunMigrationWithFS)
<runMigration runner="...">	Execute a registered Go migration runner
<rollback>	Inline SQL to undo the change (for <sql> changesets)
<cond><sql>...</sql></cond>	Precondition — changeset runs only if query returns rows
<comment>	Human-readable description (stored in tracking table)
failOnError="false"	Attribute on <change> to continue on error (default: halt on error)

Note: Each <change> must have exactly one of: <sql>, <sqlFile>, or <runMigration>.

XML Schema (XSD)

An XSD schema file (changelog.xsd) is provided for IDE autocomplete and validation support. The library performs comprehensive runtime validation without external dependencies:

• ✓ Required attributes (id, author)
• ✓ Unique changeset combinations (no duplicate id+author)
• ✓ Exactly one execution type per changeset
• ✓ Required attributes for <sqlFile> and <runMigration>
• ✓ Valid precondition structure
• ✓ At least one changeset in changelog

Validation errors are reported with clear messages indicating the problematic changeset.

SQL Files (go:embed)

For larger or reusable SQL scripts, use <sqlFile> to reference external files via Go's embed package.

Directory Structure

myapp/
├── main.go
├── migrations/
│   ├── changelog.xml
│   ├── 001_create_users.sql
│   ├── 001_rollback.sql
│   ├── 002_create_posts.sql
│   └── 002_rollback.sql

Changelog with sqlFile

<gobase changelogtable="gobase_dbchangelog">
    <change author="dev" id="001">
        <comment>Create users table</comment>
        <sqlFile path="001_create_users.sql" rollback="001_rollback.sql"/>
    </change>
    
    <change author="dev" id="002">
        <comment>Create posts table</comment>
        <sqlFile path="002_create_posts.sql" rollback="002_rollback.sql"/>
    </change>
</gobase>

Using embed.FS

package main

import (
    "context"
    "database/sql"
    "embed"
    "log"

    "github.com/mainvec/gobase"
    _ "your/database/driver"
)

//go:embed migrations/*.sql migrations/*.xml
var migrationsFS embed.FS

func main() {
    db, _ := sql.Open("sqlite3", "app.db")
    defer db.Close()

    // Open changelog from embedded filesystem
    changelog, err := migrationsFS.Open("migrations/changelog.xml")
    if err != nil {
        log.Fatal(err)
    }
    defer changelog.Close()

    ctx := context.Background()

    // Run migrations with filesystem for sqlFile support
    if err := gobase.RunMigrationWithFS(ctx, db, changelog, migrationsFS); err != nil {
        log.Fatal(err)
    }

    log.Println("Migrations completed")
}

Rollback with sqlFile

// Rollback also needs filesystem access for sqlFile rollback paths
changelog, _ := migrationsFS.Open("migrations/changelog.xml")
err := gobase.RollbackWithFS(ctx, db, changelog, "001", migrationsFS)

Migration Runners

When SQL alone isn't sufficient (data transformations, external API calls, complex business logic), you can write migrations in Go.

Defining a Migration Runner

Implement the MigrationRunner interface:

package migrations

import (
    "context"
    "database/sql"

    "github.com/mainvec/gobase"
)

// MigrateUserData transforms user data during migration
type MigrateUserData struct{}

func (m *MigrateUserData) Run(ctx context.Context, tx *sql.Tx) error {
    // Complex data transformation logic
    rows, err := tx.QueryContext(ctx, "SELECT id, legacy_name FROM users")
    if err != nil {
        return err
    }
    defer rows.Close()

    for rows.Next() {
        var id int
        var legacyName string
        if err := rows.Scan(&id, &legacyName); err != nil {
            return err
        }

        // Split legacy_name into first_name and last_name
        firstName, lastName := splitName(legacyName)
        _, err = tx.ExecContext(ctx,
            "UPDATE users SET first_name = ?, last_name = ? WHERE id = ?",
            firstName, lastName, id)
        if err != nil {
            return err
        }
    }
    return rows.Err()
}

func (m *MigrateUserData) Rollback(ctx context.Context, tx *sql.Tx) error {
    // Reverse the transformation
    _, err := tx.ExecContext(ctx,
        "UPDATE users SET legacy_name = first_name || ' ' || last_name")
    return err
}

// Register the runner during package initialization
func init() {
    gobase.RegisterMigrationRunner("migrate-user-data", &MigrateUserData{})
}

Using Runners in Changelog

Reference the runner in your XML changelog:

<gobase changelogtable="gobase_dbchangelog">
    <!-- First, add the new columns with SQL -->
    <change author="dev" id="001">
        <sql>
            ALTER TABLE users ADD COLUMN first_name VARCHAR(100);
            ALTER TABLE users ADD COLUMN last_name VARCHAR(100);
        </sql>
        <rollback>
            ALTER TABLE users DROP COLUMN first_name;
            ALTER TABLE users DROP COLUMN last_name;
        </rollback>
    </change>

    <!-- Then, run the Go migration to transform data -->
    <change author="dev" id="002">
        <comment>Split legacy_name into first_name and last_name</comment>
        <runMigration runner="migrate-user-data"/>
    </change>

    <!-- Finally, drop the legacy column -->
    <change author="dev" id="003">
        <sql>ALTER TABLE users DROP COLUMN legacy_name</sql>
        <rollback>ALTER TABLE users ADD COLUMN legacy_name VARCHAR(200)</rollback>
    </change>
</gobase>

Running Migrations with Runners

Import the package containing your runners to ensure they're registered:

package main

import (
    "context"
    "database/sql"
    "log"
    "os"

    "github.com/mainvec/gobase"
    _ "your/database/driver"
    _ "yourapp/migrations" // Import to trigger init() and register runners
)

func main() {
    db, _ := sql.Open("sqlite3", "app.db")
    defer db.Close()

    changelog, _ := os.Open("changelog.xml")
    defer changelog.Close()

    ctx := context.Background()
    if err := gobase.RunMigration(ctx, db, changelog); err != nil {
        log.Fatal(err)
    }
}

Runner API Reference

// MigrationRunner interface
type MigrationRunner interface {
    Run(ctx context.Context, tx *sql.Tx) error
    Rollback(ctx context.Context, tx *sql.Tx) error
}

// Register a runner (typically in init())
gobase.RegisterMigrationRunner("runner-id", runner)

// Get a registered runner
runner, err := gobase.GetMigrationRunner("runner-id")

// List all registered runners
ids := gobase.ListMigrationRunners()

// Unregister a runner (useful for testing)
gobase.UnregisterMigrationRunner("runner-id")

// Clear all runners (useful for testing)
gobase.ClearMigrationRunners()

Note: A changeset must have either <sql> or <runMigration>, but not both.

Tracking Table Schema

gobase automatically creates a tracking table with the following columns:

Column	Type	Description
id	VARCHAR(255)	Changeset ID
author	VARCHAR(255)	Changeset author
checksum	VARCHAR(64)	MD5 hash to detect modifications
dateexecuted	TIMESTAMP	When the changeset was applied
orderexecuted	INTEGER	Execution order
description	TEXT	Comment from the changeset

License

See LICENSE for details.

Documentation

Index

• func ClearMigrationRunners()
• func ListMigrationRunners() []string
• func RegisterMigrationRunner(id string, runner MigrationRunner)
• func Rollback(ctx context.Context, db *sql.DB, gobaseChangelog io.Reader, targetID string) error
• func RollbackWithFS(ctx context.Context, db *sql.DB, gobaseChangelog io.Reader, targetID string, ...) error
• func RunMigration(ctx context.Context, db *sql.DB, gobaseChangelog io.Reader) error
• func RunMigrationWithFS(ctx context.Context, db *sql.DB, gobaseChangelog io.Reader, fsys fs.FS) error
• func UnregisterMigrationRunner(id string) bool
• type AppliedChange
• type Change
• type Cond
• type Gobase
• type MigrationError
  • func (e *MigrationError) Error() string
  • func (e *MigrationError) Unwrap() error
• type MigrationRunner
  • func GetMigrationRunner(id string) (MigrationRunner, error)
• type RollbackSQL
• type RunnerRef
• type SqlFile

Functions

func ClearMigrationRunners

func ClearMigrationRunners()

ClearMigrationRunners removes all registered migration runners. Useful for testing.

func ListMigrationRunners

func ListMigrationRunners() []string

ListMigrationRunners returns a list of all registered runner IDs.

func RegisterMigrationRunner

func RegisterMigrationRunner(id string, runner MigrationRunner)

RegisterMigrationRunner registers a migration runner with the given ID. This should typically be called during package initialization (init functions). Panics if a runner with the same ID is already registered.

func Rollback

func Rollback(ctx context.Context, db *sql.DB, gobaseChangelog io.Reader, targetID string) error

Rollback reverts applied changesets up to (but not including) the target changeset ID If targetID is empty, rolls back all changesets Use RollbackWithFS if your changelog contains <sqlFile> elements with rollback paths

func RollbackWithFS

func RollbackWithFS(ctx context.Context, db *sql.DB, gobaseChangelog io.Reader, targetID string, fsys fs.FS) error

RollbackWithFS reverts applied changesets up to (but not including) the target changeset ID If targetID is empty, rolls back all changesets The fsys parameter provides access to SQL files referenced by <sqlFile rollback="..."> attributes

func RunMigration

func RunMigration(ctx context.Context, db *sql.DB, gobaseChangelog io.Reader) error

RunMigration executes pending database migrations from the changelog Use RunMigrationWithFS if your changelog contains <sqlFile> elements

func RunMigrationWithFS

func RunMigrationWithFS(ctx context.Context, db *sql.DB, gobaseChangelog io.Reader, fsys fs.FS) error

RunMigrationWithFS executes pending database migrations from the changelog The fsys parameter provides access to SQL files referenced by <sqlFile> elements Pass an embed.FS or os.DirFS for the directory containing your SQL files

func UnregisterMigrationRunner

func UnregisterMigrationRunner(id string) bool

UnregisterMigrationRunner removes a migration runner from the registry. Useful for testing. Returns true if the runner was found and removed.

Types

type AppliedChange

type AppliedChange struct {
	ID            string
	Author        string
	Checksum      string
	DateExecuted  time.Time
	OrderExecuted int
	Description   string
}

AppliedChange represents a record in the changelog tracking table

type Change

type Change struct {
	ID          string       `xml:"id,attr"`
	Author      string       `xml:"author,attr"`
	FailOnError *bool        `xml:"failOnError,attr"`
	Condition   *Cond        `xml:"cond"`
	SQL         string       `xml:"sql"`
	SqlFile     *SqlFile     `xml:"sqlFile"`
	RunnerRef   *RunnerRef   `xml:"runMigration"`
	Rollback    *RollbackSQL `xml:"rollback"`
	Comment     string       `xml:"comment"`
}

Change represents a single changeset

type Cond

type Cond struct {
	SQL string `xml:"sql"`
}

Cond represents a precondition for a changeset

type Gobase

type Gobase struct {
	XMLName        xml.Name `xml:"gobase"`
	ChangelogTable string   `xml:"changelogtable,attr"`
	Changes        []Change `xml:"change"`
}

Gobase represents the root element of a changelog file

type MigrationError

type MigrationError struct {
	ChangeID string
	Author   string
	Err      error
}

MigrationError represents an error during migration with changeset context

func (*MigrationError) Error

func (e *MigrationError) Error() string

func (*MigrationError) Unwrap

func (e *MigrationError) Unwrap() error

type MigrationRunner

type MigrationRunner interface {
	// Run executes the migration within the given transaction.
	// The transaction is managed by gobase - do not commit or rollback.
	Run(ctx context.Context, tx *sql.Tx) error

	// Rollback reverts the migration within the given transaction.
	// The transaction is managed by gobase - do not commit or rollback.
	Rollback(ctx context.Context, tx *sql.Tx) error
}

MigrationRunner defines the interface for programmatic migrations. Implement this interface when SQL alone is insufficient (e.g., data transformations, external API calls, complex business logic).

func GetMigrationRunner

func GetMigrationRunner(id string) (MigrationRunner, error)

GetMigrationRunner retrieves a registered migration runner by ID. Returns an error if the runner is not found.

type RollbackSQL

type RollbackSQL struct {
	SQL string `xml:",chardata"`
}

RollbackSQL represents rollback SQL for a changeset

type RunnerRef

type RunnerRef struct {
	Runner string `xml:"runner,attr"`
}

RunnerRef represents a reference to a programmatic migration runner

type SqlFile

type SqlFile struct {
	Path     string `xml:"path,attr"`
	Rollback string `xml:"rollback,attr"`
}

SqlFile represents a reference to an external SQL file