migrate

README

migrate

Database migrations written in Go. Use as CLI or import as library.

go get -u -tags 'postgres' -o migrate github.com/mattes/migrate/cli

import (
  "github.com/mattes/migrate"
  _ "github.com/mattes/migrate/database/postgres"
)

Databases

Database drivers are responsible for applying migrations to databases. Implementing a new database driver is easy. Just implement database/driver interface

• PostgreSQL
• Cassandra
• SQLite
• MySQL/ MariaDB
• Neo4j
• Ql
• MongoDB
• CrateDB
• Shell

Migration Sources

Source Drivers read migrations from various locations. Implementing a new source driver is easy. Just implement the source/driver interface.

• Filesystem - read from fileystem (always included)
• Go-Bindata - read from embedded binary data (jteeuwen/go-bindata)
• Github - read from remote Github repositories
• AWS S3 - read from Amazon Web Services S3
• Google Cloud Storage - read from Google Cloud Platform Storage

CLI usage

# dowload, build and install the CLI tool
# -tags takes database and source drivers and will only build those
$ go get -u -tags 'postgres' -o migrate github.com/mattes/migrate/cli

$ migrate -help
Usage: migrate OPTIONS COMMAND [arg...]
       migrate [ -version | -help ]

Options:
  -source      Location of the migrations (driver://url)
  -path        Shorthand for -source=file://path
  -database    Run migrations against this database (driver://url)
  -prefetch N  Number of migrations to load in advance before executing (default 10)
  -verbose     Print verbose logging
  -version     Print version
  -help        Print usage

Commands:
  goto V       Migrate to version V
  up [N]       Apply all or N up migrations
  down [N]     Apply all or N down migrations
  drop         Drop everyting inside database
  version      Print current migration version


# so let's say you want to run the first two migrations
migrate -database postgres://localhost:5432/database up 2

# if your migrations are hosted on github
migrate -source github://mattes:personal-access-token@mattes/migrate_test \
  -database postgres://localhost:5432/database down 2

Use in your Go project

import (
  "github.com/mattes/migrate/migrate"
  _ "github.com/mattes/migrate/database/postgres"
  _ "github.com/mattes/migrate/source/github"
)

func main() {
  m, err := migrate.New("github://mattes:personal-access-token@mattes/migrate_test",
    "postgres://localhost:5432/database?sslmode=enable")
  m.Steps(2)
}

Migration files

Each migration version has an up and down migration.

1481574547_create_users_table.up.sql
1481574547_create_users_table.down.sql

Development, Testing and Contributing

1. Make sure you have a running Docker daemon (Install for MacOS)
2. Fork this repo and git clone somewhere to $GOPATH/src/github.com/%you%/migrate
3. make rewrite-import-paths to update imports to your local fork
4. Confirm tests are working: make test-short
5. Write awesome code ...
6. make test to run all tests against all database versions
7. make restore-import-paths to restore import paths
8. Push code and open Pull Request

Some more notes:

• You can specify which database/ source tests to run:
  make test-short SOURCE='file go-bindata' DATABASE='postgres cassandra'
• After make test, run make html-coverage which opens a shiny test coverage overview.
• Missing imports? make deps
• make build-cli builds the CLI in directory cli/build/.
• make list-external-deps lists all external dependencies for each package

Alternatives

• https://bitbucket.org/liamstask/goose
• https://github.com/tanel/dbmigrate
• https://github.com/BurntSushi/migration
• https://github.com/DavidHuie/gomigrate
• https://github.com/rubenv/sql-migrate

Documentation

Index

• Variables
• type ErrShortLimit
  • func (e ErrShortLimit) Error() string
• type Logger
• type Migrate
  • func New(sourceUrl, databaseUrl string) (*Migrate, error)
  • func NewWithDatabaseInstance(sourceUrl string, databaseName string, databaseInstance database.Driver) (*Migrate, error)
  • func NewWithInstance(sourceName string, sourceInstance source.Driver, databaseName string, ...) (*Migrate, error)
  • func NewWithSourceInstance(sourceName string, sourceInstance source.Driver, databaseUrl string) (*Migrate, error)
  • func (m *Migrate) Close() (sourceErr error, databaseErr error)
  • func (m *Migrate) Down() error
  • func (m *Migrate) Drop() error
  • func (m *Migrate) Migrate(version uint) error
  • func (m *Migrate) Steps(n int) error
  • func (m *Migrate) Up() error
  • func (m *Migrate) Version() (uint, error)
• type Migration
  • func NewMigration(body io.ReadCloser, identifier string, version uint, targetVersion int) (*Migration, error)
  • func (m *Migration) Buffer() error
  • func (m *Migration) String() string
  • func (m *Migration) StringLong() string
• type MultiError
  • func NewMultiError(errs ...error) MultiError
  • func (m MultiError) Error() string

Variables

var (
	ErrNoChange   = fmt.Errorf("no change")
	ErrNilVersion = fmt.Errorf("no migration")
	ErrLocked     = fmt.Errorf("database locked")
)

var DefaultBufferSize = uint(100000)

var DefaultPrefetchMigrations = uint(10)

var ErrNoName = fmt.Errorf("no name")

Types

type ErrShortLimit

type ErrShortLimit struct {
	Short uint
}

func (ErrShortLimit) Error

func (e ErrShortLimit) Error() string

type Logger

type Logger interface {
	Printf(format string, v ...interface{})
	Verbose() bool
}

type Migrate

type Migrate struct {
	Log Logger

	GracefulStop chan bool

	PrefetchMigrations uint
	// contains filtered or unexported fields
}

func New

func New(sourceUrl, databaseUrl string) (*Migrate, error)

func NewWithDatabaseInstance

func NewWithDatabaseInstance(sourceUrl string, databaseName string, databaseInstance database.Driver) (*Migrate, error)

func NewWithInstance

func NewWithInstance(sourceName string, sourceInstance source.Driver, databaseName string, databaseInstance database.Driver) (*Migrate, error)

func NewWithSourceInstance

func NewWithSourceInstance(sourceName string, sourceInstance source.Driver, databaseUrl string) (*Migrate, error)

func (*Migrate) Close

func (m *Migrate) Close() (sourceErr error, databaseErr error)

func (*Migrate) Down

func (m *Migrate) Down() error

func (*Migrate) Drop

func (m *Migrate) Drop() error

func (*Migrate) Migrate

func (m *Migrate) Migrate(version uint) error

func (*Migrate) Steps

func (m *Migrate) Steps(n int) error

func (*Migrate) Up

func (m *Migrate) Up() error

func (*Migrate) Version

func (m *Migrate) Version() (uint, error)

type Migration

type Migration struct {
	Identifier    string
	Version       uint
	TargetVersion int

	Body         io.ReadCloser
	BufferedBody io.Reader
	BufferSize   uint

	Scheduled         time.Time
	StartedBuffering  time.Time
	FinishedBuffering time.Time
	FinishedReading   time.Time
	BytesRead         int64
	// contains filtered or unexported fields
}

func NewMigration

func NewMigration(body io.ReadCloser, identifier string, version uint, targetVersion int) (*Migration, error)

func (*Migration) Buffer

func (m *Migration) Buffer() error

Buffer buffers up to BufferSize (blocking, call with goroutine)

func (*Migration) String

func (m *Migration) String() string

func (*Migration) StringLong

func (m *Migration) StringLong() string

type MultiError

type MultiError struct {
	Errs []error
}

func NewMultiError

func NewMultiError(errs ...error) MultiError

func (MultiError) Error

func (m MultiError) Error() string