README
¶
migrate 
Database migrations for your Go project. Supports files, functions, and query migrations.
Version: 0.2.0
Project Links: Docs - Issues - Mailing List - Contributing
Author: Peter Sanchez (https://petersanchez.com)
Requirements
- Go 1.16+ (probably works with older, I just didn't test it)
What?
This is a simple tool to manage database migrations from within your Go project. It's not a CLI tool, but a module that you plugin to your own project as needed.
It's based on github.com/joncalhoun/migrate but I've done many changes to make it fit my needs. That is why I am just maintaining my own fork versus submitting a PR. I feel mine has diverged too far from what Jon originally wanted.
Here's some of the changes:
- Removed sqlx as dependency.
- Added support for a global
context.Contextwithin migrations. - Added support for specific timeouts (via
context.Context) per migration. - Added ability to specify a up (migrate) or down (rollback) specific version.
- Added ability to "fake" a migration (up only). Essentially just adding the entry to migration history without actually running it. Useful for edge cases.
- Added checks for migration ID duplication.
- Added check to ensure migration ID exists when a specific ID is given. This is to avoid running migrations when an entry error has occurred.
My fork is a copy of the original repo as of this commit. I probably
should have done a repo migration to Mercurial but here we are. All code up to
the commit above is licensed under the MIT license, see the PREVIOUS_LICENSES
document.
Example usage
This example will assume you're using PostgreSQL as well (note the changing of
binding type to migrate.DOLLAR).
Let's say you have a simple directory structure like so:
.
├── accounts
│ ├── input.go
│ ├── models.go
│ ├── routes.go
├── cmd
│ └── webapp
│ └── main.go
├── go.mod
├── go.sum
├── webapp_binary
├── Makefile
├── migrations
│ ├── 0001_initial_rollback.sql
│ └── 0001_initial.sql
├── templates
├── login.html
└── register.html
You can see there is a migrations directory. You can store your migration
files anywhere you want, but in this example we'll store them there.
Now let's add migrations to the main binary. We'll edit cmd/webapp/main.go to
always call migrations when starting. I'm not recommending this, just a simple
example.
import (
...
"petersanchez.com/migrate",
...
)
func main() {
// `db` is your database connection (*sql.DB)
// Custom functions should be watching for the context's Done() channel
migrations := []migrate.Migration{
migrate.FileMigration("0001_initial", "migrations/0001_initial.sql", "migrations/0001_initial_rollback.sql", 0),
migrate.Migration{
ID: "0002_function",
Migrate: func(ctx context.Context, tx *sql.Tx) error {
// Run your migrate code here.
return nil
},
Rollback: func(ctx context.Context, tx *sql.Tx) error {
// Run your rollback code here.
return nil
},
Timeout: 40, // Set a 40 second timeout on the functions
},
}
if len(os.Args) > 0 {
if os.Args[1] != "migrate" {
err := migrate.Command(db, migrations, migrate.DOLLAR)
// handle err
os.Exit(0)
}
}
// rest of your main() code
}
Why?
I found the larger database migration projects for Go to be overkill for my needs. Especially if you wanted to run custom function migrations.
During my research I stumbled on an article titled Database migrations in Go by Jon Calhoun. I had already been thinking of how to create my own migrations package as minimally as possible and this article was close enough to what I was thinking. John even created the original migrate package, but it lacked several things I wanted in my migration solution.
So I stripped out external dependencies, added a bunch of stuff I wanted, and a few development hours later, here we are.
Status
My needs are fairly simple and the current state of the module basically covers them. Please submit bugs and/or patches if you find any issues.
Contributing
We accept patches submitted via hg email which is the patchbomb extension
included with Mercurial.
The mailing list where you submit your patches is
~petersanchez/public-inbox@lists.code.netlandish.com. You can also view the
archives on the web here:
https://lists.code.netlandish.com/~petersanchez/public-inbox
To quickly setup your clone of migrate to submit to the mailing
list just edit your .hg/hgrc file and add the following:
[email]
to = ~petersanchez/public-inbox@lists.code.netlandish.com
[patchbomb]
flagtemplate = {separate(' ', 'migrate', flags)}
[diff]
git = 1
We have more information on the topic here:
Copyright & Warranty
All documentation, libraries, and sample code are Copyright 2022 Peter Sanchez <pjs@petersanchez.com>. The library and sample code are made available to you under the terms of the BSD license which is contained in the included file, LICENSE.
Documentation
¶
Overview ¶
Package migrate is a simple and flexible module to manage database migrations.
You can use migration files, queries, and custom functions to manage your migrations. Also has full Context support (globally and per migration) with has no external dependencies.
Index ¶
Constants ¶
const ( UNKNOWN = iota // Use ? for sql placeholder QUESTION // Use $<num> for sql placeholder DOLLAR // Use :<name> for sql placeholder NAMED // Use @ for sql placeholder AT )
Pulled from sqlx: https://github.com/jmoiron/sqlx
Variables ¶
This section is empty.
Functions ¶
func Command ¶ added in v0.2.0
Command will run a very simple command line processor using `os.Args[2:]`. bindType should be set to one of the constants exported in this module.
flag options are:
-fake - this will "fake" migrations and just add entries to the migration table. -verbose - this will enable verbose output.
positional arguments are
up | down - Migrate up (forward) or down (rollback). If not given "up" is assumed. <migration id> - If given, migrations will stop after this migration is processed.
Types ¶
type Migration ¶
type Migration struct {
// Unique ID for the migration. You must self enforce this.
ID string
// Optional timeout for this particular migration.
Timeout int
// Function to migrate forward. Functions should respect context's Done
// channel if they want the timeout to work.
Migrate func(ctx context.Context, tx *sql.Tx) error
// Function to migrate backward. Functions should respect context's Done
// channel if they want the timeout to work.
Rollback func(ctx context.Context, tx *sql.Tx) error
}
Migration is a unique ID plus a function that uses a sql transaction to perform a database migration step.
func FSFileMigration ¶ added in v0.3.1
FSFileMigration is the same as FileMigration except that it uses the provided `fs.FS` instance to read the file from. Useful for embedding migrations, etc.
func FileMigration ¶
FileMigration will create a Migration using the provided migration ID, up/down files, and migration timeout. If timeout is set, the queries will be cancelled if the timeout period passes.
type MigrationEngine ¶
type MigrationEngine struct {
Migrations []Migration
// Printf is used to print out additional information during a migration, such
// as which step the migration is currently on. It can be replaced with any
// custom printf function, including one that just ignores inputs. If nil it
// will default to fmt.Printf.
Printf func(format string, a ...interface{}) (n int, err error)
// contains filtered or unexported fields
}
MigrationEngine is the base migration engine
func NewEngine ¶
NewEngine returns a new MigrationEngine object bindType should be one of the constants: QUESTION, DOLLAR, NAMED or AT
func (*MigrationEngine) Migrate ¶
Migrate will run the migrations using the provided db connection. `stopID` is the migration ID to stop at (after running it) `fakeIt` is a flag that says whether or not to "fake" the migration. Faking the migration means it will add the migration entry into the `migrations` db table as if it ran but without actually running it.
Source Files
¶
- command.go
- doc.go
- migrate.go
- sql.go