swap

README

Swap

Dynamically instantiate and configure structs and/or parse config files in to them recursively, based on your build environment.
Keep your projects and their configuration files ordered and maintainable while boosting your productivity.

Swap package is based on three different tools:

• The Builder which recursively builds/configures structs with the help of some abstract Factory methods, and the two other tools below.
• An EnvironmentHandler which always determine the current environment, needed to determine which configurations to use. Can be used indipendently.
• An agnostic, layered, ConfigParser (YAML, JSON, TOML and Env) which can also be used in conjunction with the EnvironmentHandler and/or indipendently.

Installation

import "github.com/oblq/swap"

Quick start

Given that project structure:

├── config
│   ├── Tool1.yaml
│   ├── Tool1.production.yaml
│   ├── Tool2.yaml
│   ├── Tool3.yaml
│   └── Tool4.yaml
└── main.go

...we can load a struct configuring any of its fields with the right config file for the current build environment:

// main.go

import (
    "reflect"

    "github.com/oblq/swap"
)

// shared singleton
var ToolBox struct {
    // By default, swap will look for a file named like the
    // struct field name (Tool1.*, case-sensitive).
    // Optionally pass one or more config file name in the tag,
    // they will be added to the config files passed in factory methods.
    // The file extension can be omitted.
    Tool1 tools.ToolConfigurable
    Tool2 tools.ToolWFactory `swap:"Tool3|Tool4"`
    Tool3 tools.ToolRegistered
    Tool4 tools.ToolNotRecognized

    Nested1 struct {
        Tool1   tools.ToolConfigurable
        Nested2 struct {
            Tool2   tools.ToolConfigurable
            Nested3 struct {
                Tool3 tools.ToolConfigurable
            }
        }
    }

    // Add the '-' value to skip the field.
    OmittedTool        tools.ToolConfigurable `swap:"-"`
}

func init() {
    // Get a new Builder instance for the given config path.
    builder := swap.NewBuilder("./config")

    // Register a `FactoryFunc` for the `tools.ToolRegistered` type.
    builder.RegisterType(reflect.TypeOf(tools.ToolRegistered{}),
        func(configFiles ...string) (i interface{}, err error) {
            instance := &tools.Tool{}
            err = swap.Parse(&instance, configFiles...)
            return instance, err
        })

    // Force the current build environment manually to `production`.
    // This way the production config files, if exist, will be
    // passed to the constructor/configurator func along with the 
    // default one, but later, so that it will override previously passed parameters.
    builder.EnvHandler.SetCurrent("production")

    // Build the ToolBox
    if err := builder.Build(&ToolBox); err != nil {
        panic(err)
    }
}

And this is the result in console:

Builder

The builder is kind of Director pattern implementation which recursively builds and/or configures every field of the given struct.
It must be initialized with the path containing the configurations files, and can optionally use a custom EnvironmentHandler:

// Get a new Builder instance for the given config path.
builder := swap.NewBuilder("./config")

// Optionally set a custom EnvironmentHandler.
envHandler := swap.NewEnvironmentHandler(swap.DefaultEnvs.Slice())
builder = builder.WithCustomEnvHandler(envHandler)

A struct field can be 'made' or 'configured' automatically by the builder if:

• Implement the swap.Factory interface:

  // Factory is the abstract factory interface.
  type Factory interface {
      New(configFiles ...string) (interface{}, error)
  }
  
  // so:
  func (t *Tool) New(configFiles ...string) (i interface{}, err error) {
      instance := &Tool{}
      err = swap.Parse(&instance, configFiles...)
      return instance, err
  }
  

• Implement the swap.Configurable interface:

  // Configurable interface allow the configuration of fields
  // which are automatically initialized to their zero value.
  type Configurable interface {
      Configure(configFiles ...string) error
  }
  
  // so:
  func (t *Tool) Configure(configFiles ...string) (err error) {
      return swap.Parse(t, configFiles...)
  }
  

• A swap.FactoryFunc has been registerd for that specific field type.

  // Register a `FactoryFunc` for `Tool` type.
  builder.RegisterType(reflect.TypeOf(Tool{}),
      func(configFiles ...string) (i interface{}, err error) {
          instance := &Tool{}
          err = swap.Parse(&instance, configFiles...)
          return instance, err
      })
  }
  

In any of these cases the config files passed already contains environment specific ones (config.<environment>.*) if they exist.

The builder interpret its specific struct field tag:

• `swap:"<a_config_file_to_add>|<another_one>"` Provides additional config files, they will be parsed in the same order and after the generic file (the one with the name of the struct field) if found, and also after the environment specific files.

• `swap:"-"` Skip this field.

EnvironmentHandler

The EnvironmentHandler is initialized with a list of environments ([]*Environment) and the current one is determined matching a tag against its specific RegExp.

The five standard build environments are provided for convenience, and if you use git-flow they're ready:

// Default environment's configurations.
var DefaultEnvs = defaultEnvs{
    Production:  NewEnvironment("production", `(production)|(master)|(^v(0|[1-9]+)(\\.(0|[1-9]+)+)?(\\.(\\*|(0|[1-9]+)+))?$)`),
    Staging:     NewEnvironment("staging", `(staging)|(release/*)|(hotfix/*)|(bugfix/*)`),
    Testing:     NewEnvironment("testing", `(testing)|(test)`),
    Development: NewEnvironment("development", `(development)|(develop)|(dev)|(feature/*)`),
    Local:       NewEnvironment("local", `local`),
}

Provide your custom environments otherwise, the primary tag should be matched by the regexp itself:

myCustomEnv := swap.NewEnvironment("server1", `(server1)|(server1.*)`)

There are different ways to set the tag which will determine the current environment, EnvironmentHandler will try to grab that tag in three different ways, in that precise order, if one can't be determined it will fallback to the next one:

1. The manually set tag:

   envHandlerInstance.SetCurrent("development")
   envHandlerInstance.SetCurrent("server1")
   

2. The system environment variable (BUILD_ENV by default, can be changed):

   envHandlerInstance.Sources.SystemEnvironmentTagKey = "DATACENTER"
   

3. The Git branch name, by default the working dir is used, you can pass a different git repository path:

   envHandlerInstance.Sources.Git = swap.NewRepository("path/to/repo")
   

When running tests the environment will be set automatically to 'testing' if not set manually and git has not been initialized in the project root.

Finally you can check the current env in code:

myCustomEnv := swap.NewEnvironment("server1", `(server1)|(server1.*)`)
envs := append(swap.DefaultEnvs.Slice(), myCustomEnv)

envHandler := NewEnvironmentHandler(envs)
envHandler.SetCurrent(myCustomEnv.Tag())

if envHandler.Current() == myCustomEnv {
    println("YES")
}

ConfigParser (agnostic, layered, configs unmarshalling)

Swap implement two config parser funcs:

• swap.Parse()
• swap.ParseByEnv()

Both uses three specific struct field tags:

• `swapcp:"default=<default_value>"` Provides a default value that will be used if not provided by the parsed config file.

• `swapcp:"env=<system_environment_var_name>"` Will grab the value from the env var, if exist, overriding both config file provided values and/or default values.

• `swapcp:"required"` Will return error if no value is provided for this field.

Supposing we have these two yaml files in a path 'config':
pg.yaml

port: 2222

pg.production.yaml

port: 2345

...to unmarshal that config files to a struct you just need to call swap.Parse():

var PostgresConfig struct {
    // Environment vars overrides both default values and config file provided values.
    DB       string `swapcp:"env=POSTGRES_DB,default=postgres"`
    User     string `swapcp:"env=POSTGRES_USER,default=default_user"`
    // If no value is found that will return an error: 'required'.
    Password string `swapcp:"env=POSTGRES_PASSWORD,required"`
    Port     int    `swapcp:"default=5432"`
}

_ = os.Setenv("POSTGRES_PASSWORD", "my_secret_pass")

if err := swap.Parse(&PostgresConfig, "config/pg.yaml"); err != nil {
    fmt.Println(err)
}

fmt.Printf("%#v\n", PostgresConfig) 
// Config{
//      DB:         "postgres"
//      User:       "default_user"
//      Password:   "my_secret_pass"
//      Port:       2222
// }

if err := swap.ParseByEnv(&PostgresConfig, swap.DefaultEnvs.Production, "config/pg.yaml"); err != nil {
    fmt.Println(err)
}

fmt.Printf("%#v\n", PostgresConfig) 
// Config{
//      DB:         "postgres"
//      User:       "default_user"
//      Password:   "my_secret_pass"
//      Port:       2345
// }

Parse() strictly parse the passed files while ParseByEnv() look for environment specific files and will parse them to the interface pointer after the default config.

Depending on the passed environment, trying to load config/pg.yml will also load config/pg.<environment>.yml (eg.: cfg.production.yml).
If any environment-specific file will be found, for the current environment, that will override the generic one.

A file is found for a specific environment when it has that env Tag in the name before the extension (eg.: config.production.yml for the production environment).

It is possible to load multiple separated config files, also of different type, so components configs can be reused:

swap.Parse(&pusherConfig, "config/pusher.yml", "config/postgres.json")

Be aware that:

1. YAML files uses lowercased keys by default, unless you define a yaml field tag with a custom name the struct field Postgres will become "postgres", while in TOML or JSON it will remain "Postgres".
2. The default map interface is map[interface{}]interface{} in YAML, not map[string]interface{} as in JSON or TOML and this can produce some errors if the two types are parsed together to the same struct.

Also, both Parse() and ParseByEnv() will parse text/template placeholders in config files, the key used in placeholders must match the key of the config interface:

type Config struct {
    Base string
    URL string
}

base: "https://example.com"
url: "{{.Base}}/api/v1" # -> will be parsed to: "https://example.com/api/v1"

Examples

• example

To start it run:

make example

Vendored packages

• gopkg.in/yaml.v3
• github.com/BurntSushi/toml

License

Swap is available under the MIT license. See the LICENSE file for more information.

Documentation

Overview

Package swap is an agnostic config parser (supporting YAML, TOML, JSON and environment vars) and a toolbox factory with automatic configuration based on your build environment.

Index

• Variables
• func Parse(config interface{}, files ...string) (err error)
• func ParseByEnv(config interface{}, env *Environment, files ...string) (err error)
• func SetColoredLogs(enabled bool)
• type Builder
  • func NewBuilder(configsPath string) *Builder
  • func (s *Builder) Build(toolBox interface{}) (err error)
  • func (s *Builder) RegisterType(t reflect.Type, factory FactoryFunc) *Builder
  • func (s *Builder) WithCustomEnvHandler(eh *EnvironmentHandler) *Builder
• type Configurable
• type Environment
  • func NewEnvironment(tag, regexp string) *Environment
  • func (e *Environment) Info() string
  • func (e *Environment) MatchTag(tag string) bool
  • func (e *Environment) Tag() string
• type EnvironmentHandler
  • func NewEnvironmentHandler(environments []*Environment) *EnvironmentHandler
  • func (eh *EnvironmentHandler) Current() *Environment
  • func (eh *EnvironmentHandler) SetCurrent(tag string)
• type Factory
• type FactoryFunc
• type Repository
  • func NewGitRepository(path string) *Repository
  • func (g *Repository) Info() string
• type Sources

Variables

var DefaultEnvs = defaultEnvs{
	Production:  NewEnvironment("production", `(production)|(master)|(^v(0|[1-9]+)(\\.(0|[1-9]+)+)?(\\.(\\*|(0|[1-9]+)+))?$)`),
	Staging:     NewEnvironment("staging", `(staging)|(release/*)|(hotfix/*)|(bugfix/*)`),
	Testing:     NewEnvironment("testing", `(testing)|(test)`),
	Development: NewEnvironment("development", `(development)|(develop)|(dev)|(feature/*)`),
	Local:       NewEnvironment("local", `local`),
}

DefaultEnvs contains the default environment's configurations.

var FileSearchCaseSensitive bool

FileSearchCaseSensitive determine config files search mode, false by default.

Functions

func Parse

func Parse(config interface{}, files ...string) (err error)

Parse strictly parse only the specified config files in the exact order they are into the config interface, one by one. The latest files will override the former. Will also parse fmt template keys in configs and struct flags.

func ParseByEnv

func ParseByEnv(config interface{}, env *Environment, files ...string) (err error)

ParseByEnv parse all the passed files plus all the matched ones for the given Environment (if not nil) into the config interface. Environment specific files will override generic files. The latest files passed will override the former. Will also parse fmt template keys and struct flags.

func SetColoredLogs

func SetColoredLogs(enabled bool)

SetColoredLogs enable / disable colors in the stdOut.

Types

type Builder

type Builder struct {
	EnvHandler *EnvironmentHandler

	DebugOptions debugOptions
	// contains filtered or unexported fields
}

Builder recursively build/configure struct fields on the given struct, choosing the right configuration files based on the build environment.

func NewBuilder

func NewBuilder(configsPath string) *Builder

NewBuilder return a builder, a custom EnvHandler can be provided later.

func (*Builder) Build

func (s *Builder) Build(toolBox interface{}) (err error)

Build initialize and (eventually) configure the provided struct pointer looking for the config files in the provided configPath.

func (*Builder) RegisterType

func (s *Builder) RegisterType(t reflect.Type, factory FactoryFunc) *Builder

RegisterType register a configurator func for a specific type and return the builder itself.

func (*Builder) WithCustomEnvHandler

func (s *Builder) WithCustomEnvHandler(eh *EnvironmentHandler) *Builder

WithCustomEnvHandler return the same instance of the Builder but with the custom environmentHandler.

type Configurable

type Configurable interface {
	Configure(configFiles ...string) error
}

Configurable interface allow the configuration of fields which are automatically initialized to their zero value.

type Environment

type Environment struct {
	// contains filtered or unexported fields
}

Environment struct represent an arbitrary environment with its tag and regexp (to detect it based on custom criterions).

func NewEnvironment

func NewEnvironment(tag, regexp string) *Environment

NewEnvironment create a new instance of Environment. It will panic if an invalid regexp is provided or if the regexp does not match the primary tag.

`tag` is the primary environment tag and the part of the config files name that the config parser will look for while searching for environment specific files.

`regexp` must match all the tags we consider valid for the receiver.

func (*Environment) Info

func (e *Environment) Info() string

Info returns some environment info.

func (*Environment) MatchTag

func (e *Environment) MatchTag(tag string) bool

MatchTag return true if the environment regexp match the passed string.

func (*Environment) Tag

func (e *Environment) Tag() string

Tag return the primary tag of the receiver.

type EnvironmentHandler

type EnvironmentHandler struct {
	// Sources define the sources used to determine the current environment.
	Sources *Sources
	// contains filtered or unexported fields
}

EnvironmentHandler is the object that manges the environment.

func NewEnvironmentHandler

func NewEnvironmentHandler(environments []*Environment) *EnvironmentHandler

NewEnvironmentHandler return a new instance of environmentHandler with default Sources and the passed environments.

Sources define the sources used to determine the current environment. If DirectEnvironmentTag is empty then the system environment variable SystemEnvironmentTagKey will be checked, if also the system environment variable is empty the Git.BranchName will be used.

func (*EnvironmentHandler) Current

func (eh *EnvironmentHandler) Current() *Environment

Current returns the current active environment by matching the found tag against any environments regexp.

func (*EnvironmentHandler) SetCurrent

func (eh *EnvironmentHandler) SetCurrent(tag string)

SetCurrent set the current environment using a tag. It must be matched by one of the environments regexp.

type Factory

type Factory interface {
	New(configFiles ...string) (interface{}, error)
}

Factory is the abstract factory interface.

type FactoryFunc

type FactoryFunc func(configFiles ...string) (interface{}, error)

FactoryFunc is the factory method type.

type Repository

type Repository struct {
	BranchName, Commit, Build, Tag string

	Error error
	// contains filtered or unexported fields
}

Repository represent a git repository.

func NewGitRepository

func NewGitRepository(path string) *Repository

NewGitRepository return a new *Repository instance for the given path.

func (*Repository) Info

func (g *Repository) Info() string

Info return Git repository info.

type Sources

type Sources struct {

	// SystemEnvironmentTagKey is the system environment variable key
	// for the build environment tag, the default value is 'BUILD_ENV'.
	SystemEnvironmentTagKey string

	// Git is the project version control system.
	// The default path is './' (the working directory).
	Git *Repository
	// contains filtered or unexported fields
}

Sources define alternative methods to obtain the current environment.