testfixtures

README

Go Test Fixtures

Warning: this package will wipe the database data before loading the fixtures! It is supposed to be used on a test database. Please, double check if you are running it against the correct database.

Writing tests is hard, even more when you have to deal with an SQL database. This package aims to make writing functional tests for web apps written in Go easier.

Basically this package mimics the "Rails' way" of writing tests for database applications, where sample data is kept in fixtures files. Before the execution of every test, the test database is cleaned and the fixture data is loaded into the database.

The idea is running tests against a real database, instead of relying in mocks, which is boring to setup and may lead to production bugs not being caught in the tests.

Installation

First, get it:

go get -u -v gopkg.in/testfixtures.v2

Usage

Create a folder for the fixture files. Each file should contain data for a single table and have the name <table_name>.yml:

myapp/
  myapp.go
  myapp_test.go
  ...
  fixtures/
    posts.yml
    comments.yml
    tags.yml
    posts_tags.yml
    ...

The file would look like this (it can have as many record you want):

# comments.yml
- id: 1
  post_id: 1
  content: A comment...
  author_name: John Doe
  author_email: john@doe.com
  created_at: 2016-01-01 12:30:12
  updated_at: 2016-01-01 12:30:12

- id: 2
  post_id: 2
  content: Another comment...
  author_name: John Doe
  author_email: john@doe.com
  created_at: 2016-01-01 12:30:12
  updated_at: 2016-01-01 12:30:12

# ...

An YAML object or array will be converted to JSON. It can be stored on a native JSON type like JSONB on PostgreSQL or as a TEXT or VARCHAR column on other databases.

- id: 1
  post_attributes:
    author: John Due
    author_email: john@due.com
    title: "..."
    tags:
      - programming
      - go
      - testing
    post: "..."

If you need to write raw SQL, probably to call a function, prefix the value of the column with RAW=:

- id: 1
  uuid_column: RAW=uuid_generate_v4()
  postgis_type_column: RAW=ST_GeomFromText('params...')
  created_at: RAW=NOW()
  updated_at: RAW=NOW()

Your tests would look like this:

package myapp

import (
    "database/sql"
    "log"

    _ "github.com/lib/pq"
    "gopkg.in/testfixtures.v2"
)

var (
    db *sql.DB
    fixtures *testfixtures.Context
)

func TestMain(m *testing.M) {
    var err error

    // Open connection with the test database.
    // Do NOT import fixtures in a production database!
    // Existing data would be deleted
    db, err = sql.Open("postgres", "dbname=myapp_test")
    if err != nil {
        log.Fatal(err)
    }

    // creating the context that hold the fixtures
    // see about all compatible databases in this page below
    fixtures, err = testfixtures.NewFolder(db, &testfixtures.PostgreSQL{}, "testdata/fixtures")
    if err != nil {
        log.Fatal(err)
    }

    os.Exit(m.Run())
}

func prepareTestDatabase() {
    if err := fixtures.Load(); err != nil {
        log.Fatal(err)
    }
}

func TestX(t *testing.T) {
    prepareTestDatabase()
    // your test here ...
}

func TestY(t *testing.T) {
    prepareTestDatabase()
    // your test here ...
}

func TestZ(t *testing.T) {
    prepareTestDatabase()
    // your test here ...
}

Alternatively, you can use the NewFiles function, to specify which files you want to load into the database:

fixtures, err := testfixtures.NewFiles(db, &testfixtures.PostgreSQL{},
    "fixtures/orders.yml",
    "fixtures/customers.yml",
    // add as many files you want
)
if err != nil {
    log.Fatal(err)
}

Security check

In order to prevent you from accidentally wiping the wrong database, this package will refuse to load fixtures if the database name (or database filename for SQLite) doesn't contains "test". If you want to disable this check, use:

testfixtures.SkipDatabaseNameCheck(true)

Sequences

For PostgreSQL or Oracle, this package also resets all sequences to a high number to prevent duplicated primary keys while running the tests. The default is 10000, but you can change that with:

testfixtures.ResetSequencesTo(10000)

Compatible databases

PostgreSQL

This package has two approaches to disable foreign keys while importing fixtures in PostgreSQL databases:

With DISABLE TRIGGER

This is the default approach. For that use:

&testfixtures.PostgreSQL{}

With the above snippet this package will use DISABLE TRIGGER to temporarily disabling foreign key constraints while loading fixtures. This work with any version of PostgreSQL, but it is required to be connected in the database as a SUPERUSER. You can make a PostgreSQL user a SUPERUSER with:

ALTER USER your_user SUPERUSER;

With ALTER CONSTRAINT

This approach don't require to be connected as a SUPERUSER, but only work with PostgreSQL versions >= 9.4. Try this if you are getting foreign key violation errors with the previous approach. It is as simple as using:

&testfixtures.PostgreSQL{UseAlterConstraint: true}

Skipping reset of sequences

You can skip the reset of PostgreSQL sequences if you're debugging a problem with it, but most of the time you shouldn't do it:

&testfixtures.PostgreSQL{SkipResetSequences: true}

MySQL / MariaDB

Just make sure the connection string have the multistatement parameter set to true, and use:

&testfixtures.MySQL{}

SQLite

SQLite is also supported. It is recommended to create foreign keys as DEFERRABLE (the default) to prevent problems. See more on the SQLite documentation. (Foreign key constraints are no-op by default on SQLite, but enabling it is recommended).

&testfixtures.SQLite{}

Microsoft SQL Server

SQL Server support requires SQL Server >= 2008. Inserting on IDENTITY columns are handled as well. Just make sure you are logged in with a user with ALTER TABLE permission.

&testfixtures.SQLServer{}

Oracle

Oracle is supported as well. Use:

&testfixtures.Oracle{}

Generating fixtures for a existing database (experimental)

The following code will generate a YAML file for each table of the database in the given folder. It may be useful to boostrap a test scenario from a sample database of your app.

err := testfixtures.GenerateFixtures(db, &testfixtures.PostgreSQL{}, "testdata/fixtures")
if err != nil {
    log.Fatalf("Error generating fixtures: %v", err)
}

Or

err := testfixtures.GenerateFixturesForTables(
    db,
    []*TableInfo{
        &TableInfo{Name: "table_name", Where: "foo = 'bar'"},
        // ...
    },
    &testfixtures.PostgreSQL{},
    "testdata/fixtures",
)
if err != nil {
    log.Fatalf("Error generating fixtures: %v", err)
}

This was thought to run in small sample databases. It will likely break if run in a production/big database.

Contributing

Tests were written to ensure everything work as expected. You can run the tests with:

# running tests for PostgreSQL
go test -tags postgresql

# running test for MySQL
go test -tags mysql

# running tests for SQLite
go test -tags sqlite

# running tests for SQL Server
go test -tags sqlserver

# running tests for Oracle
go test -tags oracle

# running test for multiple databases at once
go test -tags 'sqlite postgresql mysql'

# running tests + benchmark
go test -v -bench=. -tags postgresql

Travis runs tests for PostgreSQL, MySQL and SQLite. AppVeyor run for all these and also Microsoft SQL Server.

To set the connection string of tests for each database, copy the .sample.env file as .env and edit it according to your environment.

Alternatives

If you don't think using fixtures is a good idea, you can try one of these packages instead:

• factory-go: Factory for Go. Inspired by Python's Factory Boy and Ruby's Factory Girl
• go-txdb (Single transaction SQL driver for Go): Use a single database transaction for each functional test, so you can rollback to previous state between tests to have the same database state in all tests
• go-sqlmock: A mock for the sql.DB interface. This allow you to unit test database code without having to connect to a real database
• dbcleaner - Clean database for testing, inspired by database_cleaner for Ruby

Documentation

Index

• Variables
• func GenerateFixtures(db *sql.DB, helper Helper, dir string) error
• func GenerateFixturesForTables(db *sql.DB, tables []*TableInfo, helper Helper, dir string) error
• func LoadFixtureFiles(db *sql.DB, helper Helper, files ...string) error
• func LoadFixtures(folderName string, db *sql.DB, helper Helper) error
• func ResetSequencesTo(value int64)
• func SkipDatabaseNameCheck(value bool)
• type Context
  • func NewFiles(db *sql.DB, helper Helper, fileNames ...string) (*Context, error)
  • func NewFolder(db *sql.DB, helper Helper, folderName string) (*Context, error)
  • func (c *Context) DetectTestDatabase() error
  • func (c *Context) Load() error
• type DataBaseHelper
• type Helper
• type InsertError
  • func (e *InsertError) Error() string
• type MySQL
• type MySQLHelper
• type Oracle
• type OracleHelper
• type PostgreSQL
• type PostgreSQLHelper
• type SQLServer
• type SQLServerHelper
• type SQLite
• type SQLiteHelper
• type TableInfo

Variables

var (
	// ErrWrongCastNotAMap is returned when a map is not a map[interface{}]interface{}
	ErrWrongCastNotAMap = errors.New("Could not cast record: not a map[interface{}]interface{}")

	// ErrFileIsNotSliceOrMap is returned the the fixture file is not a slice or map.
	ErrFileIsNotSliceOrMap = errors.New("The fixture file is not a slice or map")

	// ErrKeyIsNotString is returned when a record is not of type string
	ErrKeyIsNotString = errors.New("Record map key is not string")

	// ErrNotTestDatabase is returned when the database name doesn't contains "test"
	ErrNotTestDatabase = errors.New(`Loading aborted because the database name does not contains "test"`)
)

var ErrCouldNotConvertToTime = errors.New("Could not convert string to time")

ErrCouldNotConvertToTime is returns when a string is not a reconizable time format

Functions

func GenerateFixtures

func GenerateFixtures(db *sql.DB, helper Helper, dir string) error

GenerateFixtures generates fixtures for the current contents of a database, and saves them to the specified directory

func GenerateFixturesForTables

func GenerateFixturesForTables(db *sql.DB, tables []*TableInfo, helper Helper, dir string) error

GenerateFixturesForTables generates fixtures for the current contents of specified tables in a database, and saves them to the specified directory

func LoadFixtureFiles

func LoadFixtureFiles(db *sql.DB, helper Helper, files ...string) error

LoadFixtureFiles load all specified fixtures files into database:

LoadFixtureFiles(db, &PostgreSQL{},
	"fixtures/customers.yml", "fixtures/orders.yml")
	// add as many files you want

Deprecated: Use NewFiles() and Load() instead.

func LoadFixtures

func LoadFixtures(folderName string, db *sql.DB, helper Helper) error

LoadFixtures loads all fixtures in a given folder into the database:

LoadFixtures("myfixturesfolder", db, &PostgreSQL{})

Deprecated: Use NewFolder() and Load() instead.

func ResetSequencesTo

func ResetSequencesTo(value int64)

ResetSequencesTo sets the value the sequences will be reset to. This is used by PostgreSQL and Oracle. Defaults to 10000.

func SkipDatabaseNameCheck

func SkipDatabaseNameCheck(value bool)

SkipDatabaseNameCheck If true, loading fixtures will not check if the database name constaint "test". Use with caution!

Types

type Context

type Context struct {
	// contains filtered or unexported fields
}

Context holds the fixtures to be loaded in the database.

func NewFiles

func NewFiles(db *sql.DB, helper Helper, fileNames ...string) (*Context, error)

NewFiles creates a context for all specified fixtures files into database:

NewFiles(db, &PostgreSQL{},
    "fixtures/customers.yml",
    "fixtures/orders.yml"
    // add as many files you want
)

func NewFolder

func NewFolder(db *sql.DB, helper Helper, folderName string) (*Context, error)

NewFolder creates a context for all fixtures in a given folder into the database:

NewFolder(db, &PostgreSQL{}, "my/fixtures/folder")

func (*Context) DetectTestDatabase

func (c *Context) DetectTestDatabase() error

DetectTestDatabase returns nil if databaseName matches regexp

if err := fixtures.DetectTestDatabase(); err != nil {
    log.Fatal(err)
}

func (*Context) Load

func (c *Context) Load() error

Load wipes and after load all fixtures in the database.

if err := fixtures.Load(); err != nil {
    log.Fatal(err)
}

type DataBaseHelper

type DataBaseHelper Helper

DataBaseHelper is the helper interface Deprecated: Use Helper instead

type Helper

type Helper interface {
	// contains filtered or unexported methods
}

Helper is the generic interface for the database helper

type InsertError

type InsertError struct {
	Err    error
	File   string
	Index  int
	SQL    string
	Params []interface{}
}

InsertError will be returned if any error happens on database while inserting the record

func (*InsertError) Error

func (e *InsertError) Error() string

type MySQL

type MySQL struct {
	// contains filtered or unexported fields
}

MySQL is the MySQL helper for this package

type MySQLHelper

type MySQLHelper struct {
	MySQL
}

MySQLHelper is the MySQL helper Deprecated: Use MySQL{} instead

type Oracle

type Oracle struct {
	// contains filtered or unexported fields
}

Oracle is the Oracle database helper for this package

type OracleHelper

type OracleHelper struct {
	Oracle
}

OracleHelper is the Oracle helper Deprecated: Use Oracle{} instead

type PostgreSQL

type PostgreSQL struct {

	// UseAlterConstraint If true, the contraint disabling will do
	// using ALTER CONTRAINT sintax, only allowed in PG >= 9.4.
	// If false, the constraint disabling will use DISABLE TRIGGER ALL,
	// which requires SUPERUSER privileges.
	UseAlterConstraint bool

	// SkipResetSequences prevents the reset of the databases
	// sequences after load fixtures time
	SkipResetSequences bool
	// contains filtered or unexported fields
}

PostgreSQL is the PG helper for this package

type PostgreSQLHelper

type PostgreSQLHelper struct {
	PostgreSQL
	UseAlterConstraint bool
}

PostgreSQLHelper is the PostgreSQL helper Deprecated: Use PostgreSQL{} instead

type SQLServer

type SQLServer struct {
	// contains filtered or unexported fields
}

SQLServer is the helper for SQL Server for this package. SQL Server >= 2008 is required.

type SQLServerHelper

type SQLServerHelper struct {
	SQLServer
}

SQLServerHelper is the SQLServer helper Deprecated: Use SQLServer{} instead

type SQLite

type SQLite struct {
	// contains filtered or unexported fields
}

SQLite is the SQLite Helper for this package

type SQLiteHelper

type SQLiteHelper struct {
	SQLite
}

SQLiteHelper is the SQLite helper Deprecated: Use SQLite{} instead

type TableInfo

type TableInfo struct {
	Name  string // Table name
	Where string // A condition for extracting records. If this value is empty, extracts all records.
}

TableInfo is settings for generating a fixture for table.