godfish

README

godfish

godfish is a database migration manager, similar to the very good dogfish, but written in golang.

goals

• use the native query language in the migration files, no other high-level DSLs
• interface with many DBs
• light on dependencies
• not terrible error messages

build

Make a CLI binary for the DB you want to use. This tool comes with some driver implementations. Build one like so:

make build-cassandra
make build-mysql
make build-postgres
make build-sqlite3
make build-sqlserver

From there you could move it to $GOPATH/bin, move it to your project or whatever else you need to do.

usage

godfish help
godfish -h
godfish <command> -h

Configuration options are read from command line flags first. If those are not set, then it checks the configuration file.

connecting to the db

Database connection parameters are always read from environment variables. Set:

DB_DSN=

configure file paths

Manually set path to db migration files.

godfish -files db/migrations <command>

Make your life easier by creating a configuration file by invoking godfish init. This creates a file at .godfish.json, where you can configure things.

Change the path to the configuration file.

mv .godfish.json foo.json
godfish -conf foo.json

everything else

cat .godfish.json
# { "path_to_files": "db/migrations" }

godfish create-migration -name alpha
# outputs:
# db/migrations/forward-20200128070010-alpha.sql
# db/migrations/reverse-20200128070010-alpha.sql

godfish create-migration -name bravo -reversible=false
# outputs:
# db/migrations/forward-20200128070106-bravo.sql

#
# ... write the sql in those files ...
#

# apply migrations
godfish migrate
# apply migrations to up a specific version
godfish migrate -version 20060102150405

# show status
godfish info

# apply a reverse migration
godfish rollback

# rollback and re-apply the last migration
godfish remigrate

# show build metadata
godfish version
godfish version -json

other minutiae

Here are some notable differences between dogfish and godfish:

Filenames:

• dogfish: migrate-${date}-${name}.sql, or rollback-${date}-${name}.sql
• godfish: forward-${date}-${name}.sql, or reverse-${date}-${name}.sql

Note, dogfish uses the words, "migrate" and "rollback" to describe the migration's direction whereas godfish uses "forward" and "reverse". They are the same in that they are two complementaries. This change has one trivial benefit, the pieces of metadata encoded into the filename naturally align:

cd /path/to/db/migrations && ls -1

forward-20191112050547-init_foos.sql
forward-20191127051242-add_bars.sql
forward-20191205031405-update_more_stuff.sql
reverse-20191112050547-init_foos.sql
reverse-20191127051242-add_bars.sql
reverse-20191205031405-update_more_stuff.sql

contributing

These are welcome. To get you started, the code has some documentation, a godoc page, at least one implementation of each interface and tests.

Comments line lengths should be limited to 80 characters wide. Try not to make source code lines too long. More lines is fine with the exception of declarations of exported identifiers; they should be on one line, otherwise the generated godoc looks weird. There are also tests, those should pass.

The GitHub Actions run a security scanner on all of the source code using gosec. There should be no rule violations here. The Makefile provides a convenience target if you want to run gosec on your development machine.

tests

Docker and docker-compose are used to create environments and run the tests against a live database. Each database has a separate configuration. All of this lives in ci.Makefile and the .ci/ directory.

Build environments and run tests

make -f ci.Makefile ci-cassandra3-up
make -f ci.Makefile ci-cassandra4-up

make -f ci.Makefile ci-sqlserver-up

make -f ci.Makefile ci-mariadb-up
make -f ci.Makefile ci-mysql57-up
make -f ci.Makefile ci-mysql8-up

make -f ci.Makefile ci-postgres14-up
make -f ci.Makefile ci-postgres15-up

make -f ci.Makefile ci-sqlite3-up

Teardown

make -f ci.Makefile ci-cassandra3-down
make -f ci.Makefile ci-cassandra4-down

make -f ci.Makefile ci-sqlserver-down

make -f ci.Makefile ci-mariadb-down
make -f ci.Makefile ci-mysql57-down
make -f ci.Makefile ci-mysql8-down

make -f ci.Makefile ci-postgres14-down
make -f ci.Makefile ci-postgres15-down

make -f ci.Makefile ci-sqlite3-down

Documentation

Overview

Package godfish is a database migration library built to support the command line tool.

Index

• Variables
• func ApplyMigration(driver Driver, directoryPath string, forward bool, version string) (err error)
• func CreateMigrationFiles(migrationName string, reversible bool, dirpath, fwdlabel, revlabel string) (err error)
• func Info(driver Driver, directoryPath string, forward bool, finishAtVersion string, ...) (err error)
• func Init(pathToFile string) (err error)
• func Migrate(driver Driver, directoryPath string, forward bool, finishAtVersion string) (err error)
• type AppliedVersions
• type Driver

Variables

var ErrSchemaMigrationsDoesNotExist = errors.New("schema migrations table does not exist")

ErrSchemaMigrationsDoesNotExist means there is no database table to record migration status.

Functions

func ApplyMigration

func ApplyMigration(driver Driver, directoryPath string, forward bool, version string) (err error)

ApplyMigration runs a migration at directoryPath with the specified version and direction.

func CreateMigrationFiles

func CreateMigrationFiles(migrationName string, reversible bool, dirpath, fwdlabel, revlabel string) (err error)

CreateMigrationFiles takes care of setting up a new DB migration by generating empty migration files in a directory at dirpath. Passing in true for reversible means that a complementary file will be made for rollbacks. Names for directions in the filename could be overridden from their default values (forward and reverse) with the input vars fwdlabel, revlabel when non-empty.

func Info

func Info(driver Driver, directoryPath string, forward bool, finishAtVersion string, w io.Writer, format string) (err error)

Info writes status of migrations to w in formats json or tsv.

func Init

func Init(pathToFile string) (err error)

Init creates a configuration file at pathToFile unless it already exists.

func Migrate

func Migrate(driver Driver, directoryPath string, forward bool, finishAtVersion string) (err error)

Migrate executes all migrations at directoryPath in the specified direction.

Types

type AppliedVersions

type AppliedVersions interface {
	Close() error
	Next() bool
	Scan(dest ...interface{}) error
}

AppliedVersions represents an iterative list of migrations that have been run against the database and have been recorded in the schema migrations table. It's enough to convert a *sql.Rows struct when implementing the Driver interface since a *sql.Rows already satisfies this interface. See existing Driver implementations in this package for examples.

type Driver

type Driver interface {
	// Name should return the name of the driver: ie: postgres, mysql, etc
	Name() string

	// Connect should open a connection to the database.
	Connect(dsn string) error
	// Close should close the database connection.
	Close() error

	// AppliedVersions queries the schema migrations table for migration
	// versions that have been executed against the database. If the schema
	// migrations table does not exist, the returned error should be
	// ErrSchemaMigrationsDoesNotExist.
	AppliedVersions() (AppliedVersions, error)
	// CreateSchemaMigrationsTable should create a table to record migration
	// versions once they've been applied. The version should be a timestamp as
	// a string, formatted as the TimeFormat variable in this package.
	CreateSchemaMigrationsTable() error
	// Execute runs the schema change and commits it to the database. The query
	// parameter is a SQL string and may contain placeholders for the values in
	// args. Input should be passed to conn so it could be sanitized, escaped.
	Execute(query string, args ...interface{}) error
	// UpdateSchemaMigrations records a timestamped version of a migration that
	// has been successfully applied by adding a new row to the schema
	// migrations table.
	UpdateSchemaMigrations(forward bool, version string) error
}

Driver adapts a database implementation to use godfish.