dbump

README

dbump

Go database schema migrator library (See cristalhq/dbumper tool).

Rationale

Most of the tools related to database migrations are bloated with the questionable features and dependecies. However, in most of the cases migrations are just files with queries to run. This library does this.

Features

• Simple.
• Clean and tested code.
• Supports fs.FS and go:embed.
• ZigZag mode to better test migrations.
• Dependency-free (database connectors are optional).
• Supported databases:
  • Postgres
  • MySQL
  • ClickHouse
  • or your own!

See GUIDE.md for more details.

Install

Go version 1.16+

go get github.com/cristalhq/dbump

Example

ctx := context.Background()

cfg := dbump.Config{
	Migrator: dbump_pg.NewMigrator(db),
	Loader:   dbump.NewFileSysLoader(embed, "/"),
	Mode:     dbump.ModeApplyAll,
}

err := dbump.Run(ctx, cfg)
if err != nil {
	panic(err)
}

Also see examples: example_test.go.

Documentation

See these docs.

License

MIT License.

Documentation

Index

• Constants
• Variables
• func Run(ctx context.Context, config Config) error
• type Config
• type DiskLoader
  • func NewDiskLoader(path string) *DiskLoader
  • func (dl *DiskLoader) Load() ([]*Migration, error)
• type FS
• type FileSysLoader
  • func NewFileSysLoader(fsys FS, path string) *FileSysLoader
  • func (el *FileSysLoader) Load() ([]*Migration, error)
• type Loader
• type Migration
• type Migrator
• type MigratorMode
• type SliceLoader
  • func NewSliceLoader(migrations []*Migration) *SliceLoader
  • func (sl *SliceLoader) AddMigration(m *Migration)
  • func (sl *SliceLoader) Load() ([]*Migration, error)
• type Step

Examples

• Migrator

Constants

const MigrationDelimiter = `--- apply above / revert below ---`

MigrationDelimiter separates apply and revert queries inside a migration step/file. Is exported just to be used by https://github.com/cristalhq/dbumper

Variables

var ErrMigrationAlreadyLocked = errors.New("migration is locked already")

ErrMigrationAlreadyLocked is returned only when migration lock is already hold. This might be in a situation when previous dbump migration has not finished properly or just someone already holds this lock. See Config.UseForce to force lock acquire.

Functions

func Run

func Run(ctx context.Context, config Config) error

Run the Migrator with migration queries provided by the Loader.

Types

type Config

type Config struct {
	// Migrator represents a database.
	Migrator Migrator

	// Loader of migrations.
	Loader Loader

	// Mode of the migration.
	// Default is zero ModeNotSet (zero value) which is an incorrect value.
	// Set mode explicitly to show how migration should be done.
	Mode MigratorMode

	// Num is a value for ModeApplyN or ModeRevertN modes.
	// Must be greater than 0 for this two modes.
	Num int

	// Timeout per migration step. Default is 0 which means no timeout.
	// Only Migrator.DoStep method will be bounded with this timeout.
	Timeout time.Duration

	// NoDatabaseLock set to true will run migrations without taking a database lock.
	// Default is false.
	NoDatabaseLock bool

	// DisableTx will run every migration not in a transaction.
	// This completely depends on a specific Migrator implementation
	// because not every database supports transaction, so this option can be no-op all the time.
	DisableTx bool

	// UseForce to get a lock on a database. MUST be used with the caution.
	// Should be used when previous migration run didn't unlock the database,
	// and this blocks subsequent runs.
	UseForce bool

	// ZigZag migration. Useful in tests.
	// Going up does apply-revert-apply of each migration.
	// Going down does revert-apply-revert of each migration.
	ZigZag bool

	// BeforeStep function will be invoked right before the DoStep for each step.
	// Default is nil and means no-op.
	BeforeStep func(ctx context.Context, step Step)
	// AfterStep function will be invoked right after the DoStep for each step.
	// Default is nil and means no-op.
	AfterStep func(ctx context.Context, step Step)
}

Config for the migration process. Is used only by Run function.

type DiskLoader

type DiskLoader struct {
	// contains filtered or unexported fields
}

DiskLoader can load migrations from disk/OS.

func NewDiskLoader

func NewDiskLoader(path string) *DiskLoader

NewDiskLoader instantiates a new DiskLoader.

func (*DiskLoader) Load

func (dl *DiskLoader) Load() ([]*Migration, error)

Load is a method for Loader interface.

type FS

type FS interface {
	fs.ReadDirFS
	fs.ReadFileFS
}

type FileSysLoader

type FileSysLoader struct {
	// contains filtered or unexported fields
}

FileSysLoader can load migrations from fs.FS.

func NewFileSysLoader

func NewFileSysLoader(fsys FS, path string) *FileSysLoader

NewFileSysLoader instantiates a new FileSysLoader.

func (*FileSysLoader) Load

func (el *FileSysLoader) Load() ([]*Migration, error)

Load is a method for Loader interface.

type Loader

type Loader interface {
	Load() ([]*Migration, error)
}

Loader returns migrations to be applied on a database.

type Migration

type Migration struct {
	ID     int    // ID of the migration, unique, positive, starts from 1.
	Name   string // Name of the migration
	Apply  string // Apply query
	Revert string // Revert query
}

Migration represents migration step that will be runned on a database.

type Migrator

type Migrator interface {
	// LockDB to prevent running other migrators at the same time.
	LockDB(ctx context.Context) error
	// UnlockDB to allow running other migrators later.
	UnlockDB(ctx context.Context) error

	// Init the dbump database where database state is saved.
	// What is created by this method completely depends on migrator implementation
	// and might be different between databases.
	Init(ctx context.Context) error

	// Drop is used only in ModeDrop to remove dbump database.
	// Before drop all the migrations will be reverted (as for ModeRevertAll).
	Drop(ctx context.Context) error

	// Version of the migration. Used only once in the beginning.
	Version(ctx context.Context) (version int, err error)

	// DoStep runs the given query and sets a new version on success.
	DoStep(ctx context.Context, step Step) error
}

Migrator represents database over which we will run migrations.

Example

package main

import (
	"context"

	"github.com/cristalhq/dbump"
)

func main() {
	ctx := context.Background()
	var m dbump.Migrator
	var l dbump.Loader

	err := dbump.Run(ctx, dbump.Config{
		Migrator: m,
		Loader:   l,
		Mode:     dbump.ModeApplyAll,
	})
	if err != nil {
		panic(err)
	}
}

type MigratorMode

type MigratorMode int

MigratorMode to change migration flow.

const (
	ModeNotSet MigratorMode = iota
	ModeApplyAll
	ModeApplyN
	ModeRevertN
	ModeRevertAll
	ModeRedo
	ModeDrop
)

type SliceLoader

type SliceLoader struct {
	// contains filtered or unexported fields
}

SliceLoader loads given migrations.

func NewSliceLoader

func NewSliceLoader(migrations []*Migration) *SliceLoader

NewSliceLoader instantiates a new SliceLoader.

func (*SliceLoader) AddMigration

func (sl *SliceLoader) AddMigration(m *Migration)

AddMigration to loader.

func (*SliceLoader) Load

func (sl *SliceLoader) Load() ([]*Migration, error)

Load is a method for Loader interface.

type Step

type Step struct {
	Version   int
	Query     string
	DisableTx bool
}

Step represents exact thing that is going to run.