README
¶
dh - A simple Database Deployment Handler in Go
dh is a reaction to
DBIx::Class::DeploymentHandler,
a database migration system I built in in 2010. While the original DBICDH was built on
top of an existing ORM and schema transformation tooling, this version is intentionally
simpler, and allows users to drop in ORM support if needed.
Getting Started
The primary users of this program are expected to be Go programmers, using dh
as a library. First, create your deployment directory:
mkdir -p dh/001
cat <<EOF > dh/001/dh.sql
CREATE TABLE users (
"id",
"name",
"email"
);
EOF
cat <<EOF > dh/001/shortlinks.sql
CREATE TABLE shortlinks (
"from",
"to",
"deleted"
PRIMARY KEY ("from")
);
EOF
cat <<EOF > dh/001/history.sql
CREATE TABLE IF NOT EXISTS history (
"from",
"to",
"when",
"who"
);
EOF
Add that directory to your migration plan:
echo 000-sqlite > dh/plan.txt
echo 001 >> dh/plan.txt
Now let's create a migration to add another column:
mkdir dh/002
cat <<SQL >dh/002/shortlinks.sql
ALTER TABLE shortlinks ADD COLUMN "description" NOT NULL DEFAULT ''
SQL
cat <<SQL > dh/002/history.sql
ALTER TABLE history ADD COLUMN "description" NOT NULL DEFAULT ''
SQL
And add it to your plan:
echo 002 >> dh/plan.txt
And here's how you'd use it:
dbh := sql.Connect(...)
m := dh.NewMigrator()
if err := m.MigrateOne(dbh, dh.DHMigrations, "000-sqlite"); err != nil {
panic(err)
}
if err := m.MigrateAll(dbh, os.DirFS("dh")); err != nil {
return err
}
Out of the box dh can apply SQL or lists of SQL from JSON files, but the
migration interface is extensible so you could wire up
gopher-lua to do more advanced
migrations.
Documentation
¶
Overview ¶
package dh is a tool for maintaining RDBMS versions.
Synopsis ¶
db := sqlx.NewDb(dbh, "sqlite3")
e := dh.NewMigrator(dh.ExtensionMigrator{})
if err := e.MigrateOne(db, dh.DHMigrations, "000-sqlite"); err != nil {
panic(err)
}
if err := e.MigrateAll(db, migrationDir); err != nil {
panic(err)
}
Description ¶
dh is inspired by my own DBIx::Class::DeploymentHandler, which worked well but required the use of an ORM, and additionally had some other limitations, like needing to reliably split the SQL in the migration files into statements, which ended up being pretty frustrating.
dh simplifies the situation dramatically. You create directories of migrations; out of the box these migrations can contain SQL files (defined by having the `.sql` extension) or JSON files (defined by having the `.json` extension, containing simply an array of strings to be run as SQL statements.)
In addition to directories of files, at the same level as the directories you define a plan (in `plan.txt`) that simply lists the migrations to be applied in order. Comments (starting with `#`) are ignored, as are blank lines.
$ ls -F migrations 001/ 002/ plan.txt $ cat plan.txt 000-sqlite # built into dh 001 002
## The First Migration
The first migration is special, as it must create the table that dh uses to store which migrations have been applied. To work with the built in MigrationStorage it should be named `dh_migrations` and only needs two columns, `version`, which is the directory name applied, and `id`, which must increase with each applied version, so that queries for the last `id` will return the most recently applied version.
dh includes a migration fit for the first migration that works for SQLite, which you can apply by doing something like this:
e := dh.NewMigrator(dh.ExtensionMigrator{})
err := e.MigrateOne(db, dh.DHMigrations, "000-sqlite")
If people submit MRs for other databases I'll happily have them.
Index ¶
Constants ¶
This section is empty.
Variables ¶
var DHMigrations fs.FS
DHMigrations is an fs.FS containing pre-made migrations for the migration storage. Currently contains:
- 000-sqlite
Functions ¶
This section is empty.
Types ¶
type DoesMigrate ¶
type DoesMigrate interface {
// Migrate applies a file to the database within the current transaction.
Migrate(*sql.Tx, fs.File) error
}
DoesMigrate are things that can apply migrations to the database.
Built in examples include the JSONMigrator, SQLMigrator, and the related dispatcher, ExtensionMigrator. There's an example of a slightly more complicated migrator in the tests.
type DoesMigrationStorage ¶
type DoesMigrationStorage interface {
// StoreVersion inserts v in db.
StoreVersion(db sqlx.Execer, version string) error
// CurrentVersion returns the version stored in db.
CurrentVersion(db sqlx.Ext) (string, error)
}
DoesMigrationStorage is the interface a type must conform to to be able to be used by dh.
type ExtensionMigrator ¶
type ExtensionMigrator struct {
// contains filtered or unexported fields
}
ExtensionMigrator applies migrations based on their suffix. It is intentionally simple; rather than submitting Pull Requests to add support for (eg) YAML, you are encouraged to maintain your own.
type JSONMigrator ¶
type JSONMigrator struct{}
JSONMigrator applies statements in a json file read as an array of strings. For example you could have a file that looks like this:
[ "INSERT INTO foo (a, b, c) VALUES (1, 2, 3)", "INSERT INTO foo (a, b, c) VALUES (3, 2, 1)", "INSERT INTO foo (a, b, c) VALUES (9, 9, 9)" ]
type MigrationStorage ¶
type MigrationStorage struct{}
func (MigrationStorage) CurrentVersion ¶
func (s MigrationStorage) CurrentVersion(dbh sqlx.Ext) (string, error)
func (MigrationStorage) StoreVersion ¶
func (s MigrationStorage) StoreVersion(dbh sqlx.Execer, v string) error
type Migrator ¶
type Migrator struct {
DoesMigrationStorage
DoesMigrate
// contains filtered or unexported fields
}
Migrator is the main type to be used within dh. It orchestrates the migrations based on the provided plan.
func NewMigrator ¶
func NewMigrator() Migrator
NewMigrator returns a Migrator instance with MigrationStorage and the ExtensionMigrator. Create your own Migrator instace or replace those values if you need to.
func (Migrator) MigrateAll ¶
MigrateAll applies the migrations listed in `plan.txt`, starting with the migration after the one most recently applied to the database.
type Plan ¶
type Plan struct{}
Plan parses the `plan.txt` file. It ignores comments (prefixed with `#`) and whitespace.
type SQLMigrator ¶
type SQLMigrator struct{}
SQLMigrator applies a file as a single Exec call. If that will cause issues with your database, you can either switch to JSONMigrator, or add a SQL Parser to your project to properly split statements.
Source Files