simplemigrate

README

simplemigrate

A database migration library for Go. Designed for ease of use and safety in handling schema changes, simplemigrate currently supports SQLite, with plans to extend support to PostgreSQL and MySQL soon.

Introduction

simplemigrate addresses common challenges encountered in team environments when managing database migrations. It ensures seamless integration of schema changes, preventing issues like duplicate version numbers or partially applied migrations.

Additionally, it offers validation for the SQL queries involved by utilizing SQLFluff[

Key Features

• Migrate Up Only: Designed to only migrate up. Rolling back changes must be done manually, enhancing safety. Usually, when in production it's better to do another migration to rollback the previous. This way, this will be also recorded.
• Sequential Versioning: Migration versions must be sequential integers starting from 1, ensuring order and clarity. This is enforced and there is no other option to name migrations.
• No Duplicate Versions: Duplicate version numbers in the migration folder are not allowed. The tool complains if that happens.
• Migration Logging: All applied migrations are logged with timestamps and the hash of the SQL executed.
• Transaction Support: Capability to run all migrations within a single transaction.
• Transactional SQL Statements: Each SQL statement in a migration file is executed in a transaction. Multiple statements can be separated with -- migrate:next.
• Library Usage: Easily usable as a library in Go projects.
• Query Validation: Supports the ability to validate SQL statements before execution using a SQL linter.
• Work-In-Progress: This project is WIP, and currently only SQLite is supported. PostgreSQL and MySQL support are on the roadmap.

Getting Started

Prerequisites

To enable query validation you need to install SQLFluff.

pip install sqlfluff

(if you do not want query validation this is not required)

Installation

To install simplemigrate, use the following go get command:

go get github.com/gosom/simplemigrate

Usage

Command-Line Interface

Set the DATABASE_URL environment variable to your SQLite database connection string. Run migrations using the command-line interface:

simplemigrate -migrations-folder="path/to/migrations" --enable-query-validation

other command line options:

  -enable-query-validation
        enables query validation
  -migrations-folder string
        migrations folder (default "migrations")
  -migrations-table-name string
        migrations table name (default "schema_migrations")
  -transaction
        run all migrations in a transaction

As a Library

Import simplemigrate into your Go project and use it to manage migrations:

import "github.com/gosom/simplemigrate"

migrator := simplemigrate.New(driver, opts...)
err := migrator.Migrate(ctx)

I recommend to check usage in cmd/main.go

Configuration

simplemigrate can be configured with various options:

• WithInTransaction: Runs all migrations within a single transaction.
• WithQueryValidation: Enables SQL query validation in migration files.
• WithSystemFS: Uses the system filesystem for migration files.
• WithEmbedFS: Uses a embed file system (if you want to embed your migrations in the binary)
• WithMigrationTable: Change the default (schema_migrations) table name

Contributing

Contributions to simplemigrate are welcome. Feel free to open issues or submit pull requests.

License

simplemigrate is released under the MIT License. See the LICENSE file for more details.

Contact

For questions and support, please open an issue in the GitHub repository.

Logo

The logo is generated using OpenAI's DALL-E.

Documentation

Index

• Variables
• type DBDriver
• type Migration
• type Migrator
  • func New(driver DBDriver, opts ...Option) *Migrator
  • func (m *Migrator) Migrate(ctx context.Context) error
• type Option
  • func WithEmbedFS(f fs.FS) Option
  • func WithInTransaction() Option
  • func WithMigrationTable(migrationsTable string) Option
  • func WithQueryValidator(validator QueryValidator) Option
  • func WithSystemFS(root string) Option
• type QueryValidator

Variables

var (
	// ErrMigrationTableNameMissing is returned when the migrations table name is empty
	ErrMigrationTableNameMissing = errors.New("migrations table name cannot be empty")
	// ErrUnknownDriver is returned when the driver is unknown
	ErrUnknownDriver = errors.New("unknown driver")
	// ErrInvalidMigrationFile is returned when the migration file is invalid
	ErrInvalidMigrationFile = errors.New("invalid migration file")
	// ErrMigrationFolder is returned when the migration folder is invalid
	ErrMigrationFolder = errors.New("invalid migration folder")
	// ErrInvalidQuery is returned when the query is invalid
	ErrInvalidQuery = errors.New("invalid query")
)

Types

type DBDriver

type DBDriver interface {
	// Dialect returns the database dialect
	Dialect() string
	// Close closes the connection to the database
	Close(ctx context.Context) error
	// CreateMigrationsTable creates the migrations table
	// migrationsTable is the name of the migrations table
	// If the table already exists, it does nothing
	// It returns an error if something goes wrong
	CreateMigrationsTable(ctx context.Context, migrationsTable string) error
	// SelectMigrations selects all migrations from the migrations table
	// migrationsTable is the name of the migrations table
	// It returns a sorted slice (by Version ascending) of migrations or an error
	SelectMigrations(ctx context.Context, migrationsTable string) ([]Migration, error)
	// ApplyMigrations applies migrations to the database
	// migrationsTable is the name of the migrations table
	// inTx is a flag that indicates if the migrations should be applied in a transaction
	// migrations is the slice of migrations to apply
	// It returns an error if something goes wrong
	ApplyMigrations(ctx context.Context, migrationsTable string, inTx bool, migrations []Migration) error
}

DBDriver represents a database driver

type Migration

type Migration struct {
	Version    int
	Fname      string
	AppliedAt  *time.Time
	Statements []string
	Hash       string
}

Migration represents a single migration

type Migrator

type Migrator struct {
	// contains filtered or unexported fields
}

Migrator is a struct that represents a migrator It is used to migrate a database

func New

func New(driver DBDriver, opts ...Option) *Migrator

New is a constructor for Migrator Use this to create a new migrator and apply migrations to a database The driver is the database driver The options are used to configure the migrator

func (*Migrator) Migrate

func (m *Migrator) Migrate(ctx context.Context) error

Migrate is used to apply migrations to a database It returns an error if something goes wrong

type Option

type Option func(*Migrator) error

Option represents a migrator option

func WithEmbedFS

func WithEmbedFS(f fs.FS) Option

WithEmbedFS is an option to use the embed filesystem The fs is the embed filesystem It is nil by default

func WithInTransaction

func WithInTransaction() Option

WithInTransaction is an option to apply all migrations in a transaction If an error occurs, the transaction is rolled back It is disabled by default

func WithMigrationTable

func WithMigrationTable(migrationsTable string) Option

WithMigrationTable is an option to set the migrations table name It is "schema_migrations" by default

func WithQueryValidator

func WithQueryValidator(validator QueryValidator) Option

WithQueryValidator is an option to enable query validation It is disabled by default Its purpose is to validate queries before applying them

func WithSystemFS

func WithSystemFS(root string) Option

WithSystemFS is an option to use the system filesystem The root is the root folder of the migrations It is "migrations" by default

type QueryValidator

type QueryValidator interface {
	ValidateQuery(ctx context.Context, dialect, query string) error
}

QueryValidator represents a query validator