aceptadora

README

aceptadora

Aceptadora provides the boilerplate to orchestrate the containers for an acceptance test.

Aceptadora is a replacement for docker-compose in acceptance tests, and it also allows running and debugging tests from your IDE.

Example

The acceptance tests of this package are an example of usage for this package. There are also CI builds running on both Github Actions and Travis CI that serve as an example.

Long story short:

• Define your service in a YAML file inspired by docker-compose format, with an idiomatic name of aceptadora.yml
• Define your environment variables in some config.env files
• Load them in your test:

   aceptadora.SetEnv(
   	s.T(),
   	aceptadora.OneOfEnvConfigs(
   		aceptadora.EnvConfigWhenEnvVarPresent("../config/gitlab.env", "GITLAB_CI"),
   		aceptadora.EnvConfigAlways("../config/default.env"),
   	),
   	aceptadora.EnvConfigAlways("acceptance.env"),
   ) 

• Fill the aceptadora.Config values, you can use github.com/colega/envconfig for that

   envconfig.MustProcess("ACCEPTANCE", &s.cfg)

• Instantiate the aceptadora:

   aceptadora := aceptadora.New(t, cfg)

• Start your service:

   aceptadora.Run(ctx, "redis")

• Test stuff
• When you're done, stop it using aceptadora.StopAll(ctx) or aceptadora.Stop(ctx, "redis")

Motivations

We've been using docker-compose for acceptance tests for a long time, and while this approach did have issues (since test subjects and dependencies were not restarted between tests, we had to have some suites depending on other ones, which made the tests flaky and slow) we lived with it until the introduction of gRPC. Once we started playing with gRPC we found an issue: the gRPC golang client tries to connect to the service when the service is started, and if it fails, it will keep returning that error for a while. Since we started our test subjects before the acceptance-tester in the former approach, our test subjects failed to call the mocked gRPC servers on the acceptance-tester.

So, we created a library that restarted our test subject interacting with docker from the acceptance test itself.

Then, we found the need to restart multiple test subjects plus the complexity of having to handle two kinds of configurations (docker-compose for dependencies and aceptadora for test subjects) so we decided to extend the functionality of aceptadora to completely replace the docker-compose and allow managing the lifecycle of all dependencies from the test.

You can read the full story on Medium.

Decisions

Everything in aceptadora accepts t *testing.T and everything does require.NoError(t, err) because in tests nobody's going to handle the errors anyway, so we apply a fail-fast strategy, removing the retured errors and keeping the API clean for clearer acceptance tests.

Running

In order to handle multiple environments there are some stages in config loading. Notice that all the configs loaded expand the env vars set by ${VAR} to their values from what's already loaded.

First, we load some very basic env-dependant config, deciding on env vars to load a local or gitlab config. This configuration mostly provides details about networking setup:

• Where can acceptance-tester reach the services? Usually this would be localhost, but on Gitlab it's docker as we're running dind.
• Where can services reach the acceptance-tester? This will be set to the first local non-loopback IP address in the environment variable called TESTER_ADDRESS. You can set this variable to something more specific before running the test too, in which case it won't be overwritten.

One may wonder: why don't we just decide all of that in some kind of test-loading shellscript? The answer is that deciding that from the test itself allows us running the tests from any IDE as a normal test, instead of having a proxy script. Of course loading some env vars would work, but since aceptadora can do this for you, why should we care? This way we can make sure that test running is portable and doesn't require any external dependencies.

Then we load more env configs for the test itself, usually acceptance.env, which tells the acceptance test where the aceptadora.yml file is located, and how images from different docker registries are pulled. Notice that acceptance.env can be specific to each suite you may have if their paths are different. You can load as many common configs as you want, loading an acceptance.env and then ../config/acceptance.env for example.

Here aceptadora.New() would load the aceptadora.yml file from the provided dir and filename, and it will also set ${YAMLDIR} variable for the yamls itself to be able to reference its configs and binds from where the yaml is located instead of from where the test is located: this allows us having multiple test suites in different folders using the same aceptadora.yml file.

Finally, we run services by just running aceptadora.Run(ctx, "svc-name-in-the-yaml").

Aceptadora will also take care of stopping the services, you can call aceptadora.Stop(ctx, svcName) to stop one of them, or StopAll(ctx) to stop all the (still running) services.

Unit tests

This package doesn't have unit tests. All the testing is performed by the example itself in the acceptance tests folder. The unit tests would require either defining interfaces for the functionality that docker provides and mocking them, which would overcomplicate the code without offering enough value in exchange, or testing using the docker real docker API, which is already covered by the acceptance tests. However, this is opinionated. Feel free to disagree, open us an issue with your proposal, or even better, a pull request.

Documentation

Index

• Constants
• func SetEnv(t *testing.T, matchers ...ConfigPathMatcher)
• type Aceptadora
  • func New(t *testing.T, imagePuller ImagePuller, cfg Config) *Aceptadora
  • func (a *Aceptadora) PullImages(ctx context.Context)
  • func (a *Aceptadora) Run(ctx context.Context, name string)
  • func (a *Aceptadora) Stop(ctx context.Context, name string)
  • func (a *Aceptadora) StopAll(ctx context.Context)
• type Config
• type ConfigPathMatcher
  • func EnvConfigAlways(path string) ConfigPathMatcher
  • func EnvConfigWhenEnvVarPresent(path, envVarName string) ConfigPathMatcher
  • func OneOfEnvConfigs(matchers ...ConfigPathMatcher) ConfigPathMatcher
• type ImagePuller
• type ImagePullerConfig
• type ImagePullerImpl
  • func NewImagePuller(t *testing.T, cfg ImagePullerConfig) *ImagePullerImpl
  • func (i *ImagePullerImpl) Pull(ctx context.Context, imageName string)
• type RepositoryConfig
• type Runner
  • func NewRunner(t *testing.T, name string, svc Service, puller ImagePuller) *Runner
  • func (r *Runner) Start(ctx context.Context)
  • func (r *Runner) Stop(ctx context.Context) error
  • func (r *Runner) StopWithTimeout(ctx context.Context, timeout time.Duration) error
• type Service
• type YAML
  • func LoadYAML(filename string) (YAML, error)

Constants

const DefaultNetwork = "acceptance-testing"

Functions

func SetEnv

func SetEnv(t *testing.T, matchers ...ConfigPathMatcher)

SetEnv loads the configuration from a file, choosing the proper one depending on the provided matchers. Every matcher returning true as second value will be executed. Matchers are executed in their provided order

Types

type Aceptadora

type Aceptadora struct {
	// contains filtered or unexported fields
}

func New

func New(t *testing.T, imagePuller ImagePuller, cfg Config) *Aceptadora

New creates a new Aceptadora. It will try to load the YAML config from the path provided by Config If something goes wrong, it will use testing.T to fail.

func (*Aceptadora) PullImages

func (a *Aceptadora) PullImages(ctx context.Context)

PullImages pulls all the images mentioned in aceptadora.yml This allows doing this outside of the context of the test, and avoid unrelated flaky timeouts in the tests happening when most of the context has been consumed by pulling the image

func (*Aceptadora) Run

func (a *Aceptadora) Run(ctx context.Context, name string)

Run will start a given service (from aceptadora.yml) and register it for stopping later

func (*Aceptadora) Stop

func (a *Aceptadora) Stop(ctx context.Context, name string)

Stop will try to stop the service with the name provided It will fail fatally if such service isn't defined It will skip the service if it's already stopped, and set it to nil once stopped, making this call idempotent Stop is not thread safe.

func (*Aceptadora) StopAll

func (a *Aceptadora) StopAll(ctx context.Context)

StopAll will stop all the services in the reverse order If you need to explicitly stop some service in first place, use Stop() previously.

type Config

type Config struct {
	YAMLDir  string `default:"./"`
	YAMLName string `default:"aceptadora.yml"`

	// StopTimeout will be used to stop containers gracefully.
	// If zero (default), then containers will be forced to stop immediately saving some tear down time.
	StopTimeout time.Duration `default:"0s"`
}

Config is intended to be loaded by "github.com/colega/envconfig"

type ConfigPathMatcher

type ConfigPathMatcher func() (path string, use bool)

func EnvConfigAlways

func EnvConfigAlways(path string) ConfigPathMatcher

EnvConfigAlways is a matcher for SetEnv that is always true, useful to load the common acceptance.env config once the env-specifics are loaded.

func EnvConfigWhenEnvVarPresent

func EnvConfigWhenEnvVarPresent(path, envVarName string) ConfigPathMatcher

EnvConfigWhenEnvVarPresent is a matcher for SetEnv that matches when the provided env var name is present

func OneOfEnvConfigs

func OneOfEnvConfigs(matchers ...ConfigPathMatcher) ConfigPathMatcher

OneOfEnvConfigs provides a matcher that returns as the result the result of the first matching matcher. Useful when several matchers would match, like for instance a Github Actions environment is being a Linux at the same time.

type ImagePuller

type ImagePuller interface {
	Pull(ctx context.Context, imageName string)
}

type ImagePullerConfig

type ImagePullerConfig struct {
	Repo []RepositoryConfig
}

ImagePullerConfig configures the pulling options for different image repositories

type ImagePullerImpl

type ImagePullerImpl struct {
	// contains filtered or unexported fields
}

func NewImagePuller

func NewImagePuller(t *testing.T, cfg ImagePullerConfig) *ImagePullerImpl

func (*ImagePullerImpl) Pull

func (i *ImagePullerImpl) Pull(ctx context.Context, imageName string)

type RepositoryConfig

type RepositoryConfig struct {
	// Domain is used to specify which domain this config applies to, like `docker.io`
	Domain string
	// SkipPulling can be specified if images from this domain are not intended to be pulled
	// Useful for images previously built locally, or for local testing when repository credentials are not passed to the test
	SkipPulling bool

	// Auth provides the default docker library's field to authenticate and will be used for pulling.
	// Usually Username & Password fields should be filled.
	Auth types.AuthConfig
}

RepositoryConfig provides the details of access to a docker repository.

type Runner

type Runner struct {
	// contains filtered or unexported fields
}

func NewRunner

func NewRunner(t *testing.T, name string, svc Service, puller ImagePuller) *Runner

func (*Runner) Start

func (r *Runner) Start(ctx context.Context)

func (*Runner) Stop

func (r *Runner) Stop(ctx context.Context) error

Stop will try to stop the container within the context provided.

func (*Runner) StopWithTimeout

func (r *Runner) StopWithTimeout(ctx context.Context, timeout time.Duration) error

StopWithTimeout will stop the containers within the given timeout, if 0 it's just a force stop.

type Service

type Service struct {
	Image   string   `yaml:"image"`
	Network string   `yaml:"network"`
	Binds   []string `yaml:"binds"`
	Command []string `yaml:"command"`
	EnvFile []string `yaml:"env_file"`
	Ports   []string `yaml:"ports"`

	IgnoreLogs bool `yaml:"ignore_logs"`
}

Service describes a service aceptadora can run

type YAML

type YAML struct {
	Services map[string]Service `yaml:"services"`
}

YAML defines the aceptadora yaml config This enumerates the services aceptadora can run, their images, volumes to be mounted, ports to be mapped, and env configs

func LoadYAML

func LoadYAML(filename string) (YAML, error)

LoadYAML reads the aceptadora.yml config, expanding the env var references to their values.