sprbox

README

SpareBox

Dynamically create toolbox singletons with automatic configuration based on your build environment.
SpareBox is also an agnostic, layered, config parser (supporting YAML, TOML, JSON and Environment vars).
Keep your projects and their configuration files ordered and maintainable.

Installation

Using dep:

dep ensure -add github.com/oblq/sprbox@master

...or go get:

go get -u github.com/oblq/sprbox

ToolBox autoload (init and config)

1. Define your toolbox.

Fields can be of any type, sprbox will init pointers and pass config files where needed (configurable structs, struct pointers, slices or maps).
To load a configuration file a field must implement the configurable interface.

type ToolBox struct {
	// By default sprbox will look for a file named like the
	// struct field name (Services.*, case sensitive).
	// Services.<environment>.yml will override Services.yml
	// for the given env, if exist.
	Services services.ServicesMap

	// MediaProcessing does not implement the 'configurable' interface
	// so it will be traversed recursively.
	// Recursion only stop when no more embedded elements are found 
	// or when a 'configurable' element is found instead.
	// 'configurable' elements will not be traversed.
	MediaProcessing struct {
		// Optionally pass one or more specific config file name, 
		// separated by the pipe symbol: |.
		// The latest will overrides others, from right to left,
		// you can see that using sprbox.SetDebug(true).
		// sprbox will always try to find the file named
		// like the struct field first (Pictures.*).
		// File extension can be omitted.
		//
		// Use a sub-directory for embedded structs to keep things ordered:
		// mp/Pics -> "./config/mp/Pics.*"
		Pictures services.Service `sprbox:"mp/Pics|mp/PicsOverride"`
		Videos   services.Service
	}

	WP  Workerful
	// Workerful implement the 'configurableInCollections' interface,
	// so it can be loaded also directly inside 
	// slices or maps using a single config file.
	WPS []Workerful 

	// will print the error in console
	ToolMissingConfig *Tool

	// Optionally add the 'omit' value to skip a field.
	OmittedTool Tool `sprbox:"omit"`
}

var ToolBox MyToolBox

2. Init and configure the toolbox in one line.

In sprbox.LoadToolBox() environment-specific config files (cfg.<environment>.*) will override generic ones (cfg.*):

sprbox.PrintInfo()
// Optionally set debug mode.
// sprbox.SetDebug(true)
sprbox.LoadToolBox(&ToolBox, "./config")

NOTE: tool's exported pointer fields will be automatically initialized before to call the configurable interface.

The build environment

The build environment is determined matching a tag against some predefined environment specific RegEx, since any of the env's RegEx can be edited users have the maximum flexibility on the method to use.
For instance, the machine hostname (cat /etc/hostname) can be used.

sprbox will try to grab that tag in three different ways, in a precise order, if one can't be determined it will check for the next one:

1. The BUILDENV var in sprbox package:

   sprbox.BUILDENV = "dev"
   

   Since it is an exported string, can also be interpolated with -ldflags at build/run time:

   LDFLAGS="-X ${GOPATH:-$HOME/go}/src/github.com/oblq/sprbox.BUILDENV=develop"
   go build -ldflags "${LDFLAGS}" -v -o ./api_bin ./api
   

2. The environment variable 'BUILD_ENV':

   // sprbox.EnvVarKey is 'BUILD_ENV'
   os.Setenv(sprbox.EnvVarKey, "dev")
   

3. The Git branch name (Gitflow supported).
   By default the working dir is used, you can pass a different git repository path for this:

   sprbox.VCS = sprbox.NewRepository("path/to/repo")
   println(sprbox.VCS.BranchName) // Commit, Tag, Build, Path and Error
   sprbox.VCS.PrintInfo()
   

4. When you run tests the environment will be set automatically to 'testing' if not set manually and no git repo is found in the project root.

Every environment has a set of default RegEx:

Production  = []string{"production", "master"}
Staging     = []string{"staging", "release/*", "hotfix/*"}
Testing     = []string{"testing", "test"}
Development = []string{"development", "develop", "dev", "feature/*"}
Local       = []string{"local"}

...and they can be edited:

sprbox.Testing.SetExps([]string{"testing", "test"})
sprbox.Testing.AppendExp("feature/f*")
println("matched:", sprbox.Testing.MatchTag("feature/f5"))

Finally you can check the current env in code with:

if sprbox.Env() == sprbox.Production { 
    doSomething() 
}

sprbox.Env().PrintInfo()

Working with directories

Sparebox offer two utility funcs to work with directories.

1. EnvSubDir()
   ...pretty much self-explanatory:

   sprbox.EnvSubDir("static") // -> "static/<environment>" (eg.: "static/staging")
   

2. CompiledPath()
   If the current build-environment has RunCompiled == true sprbox.CompiledPath() returns the path base, so static files can stay side by side with the executable while it is possible to have a different location when the program is launched with go run.
   This allow to manage multiple packages in one project during development, for instance using a config path in the parent dir, side by side with the packages, while having the same config folder side by side with the executable where needed.

   sprbox.BUILDENV = sprbox.Development.ID()   
   
   sprbox.Development.RunCompiled = false   
   sprbox.CompiledPath("../static_files/config") // -> "../static_files/config"
   
   sprbox.Development.RunCompiled = true   
   sprbox.CompiledPath("../static_files/config") // -> "config"
   

   A simple usage example is:

   sprbox.LoadToolBox(&myToolBox, sprbox.CompiledPath("../config"))
   

   By default only Production and Staging environments have RunCompiled true.

Using your package in sprbox

To start using your package in sprbox you just need to implement the configurable interface:

type configurable interface {
	SpareConfig([]string) error
}

// optional, allow to load the package from a slice or a map directly.
type configurableInCollection interface {
	SpareConfigBytes([]byte) error
}

Example:

type MyPackage struct {
	Something string `yaml:"something"`
}

// SpareConfig is the sprbox 'configurable' interface implementation.
// (mp *MyPackage) is automatically initialized with a pointer to MyPackage{}
// so it will never be nil, but needs configuration.
func (mp *MyPackage) SpareConfig(configFiles []string) (err error) {
	var config *MyPackageConfig
	err = sprbox.LoadConfig(&cfg, configFiles...)    	
	mp.DoSomethingWithConfig(config)
	return
}

// SpareConfigBytes optionally allow to load MyPackage inside a slice or a map directly.
func (mp *MyPackage) SpareConfigBytes(configBytes []byte) (err error) {
	var config *MyPackageConfig
	err = sprbox.Unmarshal(configBytes, &cfg)
	mp.DoSomethingWithConfig(config)
	return
}

Add sprbox in your repo topics and/or the 'sprbox-ready' badge if you like it:

Embed third-party packages in sprbox

Suppose we want to embed packagex.StructX:

type StructX struct {
	*packagex.StructX
}

func (sx *StructX) SpareConfig(configFiles []string) (err error) {
	var cfg packagex.Config
	err = sprbox.LoadConfig(&cfg, configFiles...)    	
	sx.StructX = packagex.NewStructX(cfg)
	return
}

From here on you can use the StructX in a toolbox with automatic init/config:

type ToolBox struct {
	SX StructX
}

var App ToolBox

func init() {
	// ./config must contain SX.(yml|yaml|json|toml) config file in that case.
	sprbox.LoadToolBox(&App, "./config") 
	
	// Call any of the packagex.StructX's funcs on SX.
	// Initialized and configured.
	App.SX.DoSomething()
}

Agnostic, layered, config unmarshaling

Given that project structure:

├── config
│   ├── pg.yaml
│   └── pg.production.yaml
└── main.go

pg.yaml:

port: 2222

pg.production.yaml:

port: 2345

...to unmarshal that config files to a struct you just need to call sprbox.LoadConfig(&pgConfig, "config/pg.yaml"):

package main

import (
	"fmt"
	"os"
	
	"github.com/oblq/sprbox"
)

type PostgresConfig struct{
	// Environment vars overrides both default values and config file provided values.
	DB       string `sprbox:"env=POSTGRES_DB,default=postgres"`
	User     string `sprbox:"env=POSTGRES_USER,default=postgres"`
	// If no value is found that will return an error: 'required'.
	Password string `sprbox:"env=POSTGRES_PASSWORD,required"`
	Port     int    `sprbox:"default=5432"`
} 

func main() {
	os.Setenv("POSTGRES_PASSWORD", "123_only_known_by_me")
	
	// Setting 'production' build-environment,
	// so 'pg.production.yml' will override 'pg.yml'.
	sprbox.BUILDENV = sprbox.Production.ID() // -> 'production'
	
	var pgConfig PostgresConfig	
	if err := sprbox.LoadConfig(&pgConfig, "config/pg.yaml"); err != nil {
		fmt.Println(err)
	}
	
	fmt.Printf("%#v\n", pgConfig) 
	// Config{
	//      DB:         "postgres"
	//      User:       "postgres"
	//      Password:   "123_only_known_by_me"
	//      Port:       2345
	// }
}

Depending on the build environment, trying to load config/pg.yml will also load config/pg.<environment>.yml (eg.: cfg.production.yml).
If any environment-specific file will be found, for the current environment, that will override the generic one.

It is possible to load multiple separated config files, also of different type, so components configs can be reused. Be aware that:

1. YAML files uses lowercased keys by default, unless you define a custom field tag (struct field Postgres will become "postgres", while in TOML or JSON it will remain "Postgres").
2. The default map interface is map[interface{}]interface{} in YAML, not map[string]interface{} as in JSON or TOML.

func main() {
	// File extension can be omitted:
	var pusherConfig PushNotificationsConfig	
	sprbox.LoadConfig(&pusherConfig, "config/pusher.yml", "config/postgres.json")
}

The file extension in the file path can be omitted, since sprbox can load YAML, TOML and JSON files it will search for cfg.* using RegEx, the config file itself must have an extension.

Also, LoadConfig() will parse text/template placeholders in config files, the key used in placeholders must match the key of the config interface, case-sensitive:

type Config struct {
	Base string
	URL string
}

base: "https://example.com"
url: "{{.Base}}/api/v1" # -> will be parsed to: "https://example.com/api/v1"

Examples

• example

To start it run:

make example

Ready packages

Included:

• common
  • Services Service/micro-service/monolith abstraction, get services URL, Proxy, Version, Basepath, hold custom Data etc...

External:

• Workerful Full-featured worker-pool implementation.

Vendored packages

• gopkg.in/yaml.v2
• github.com/BurntSushi/toml

Author

• Marco Muratori

License

SpareBox is available under the MIT license. See the LICENSE file for more information.

Documentation

Overview

Package sprbox is an agnostic config parser (supporting YAML, TOML, JSON and Environment vars) and a toolbox factory with automatic configuration based on your build environment.

Index

• Constants
• Variables
• func CompiledPath(path string) string
• func EnvSubDir(path string) string
• func GetInfo(w http.ResponseWriter, r *http.Request)
• func LoadConfig(config interface{}, files ...string) (err error)
• func LoadToolBox(toolBox interface{}, configPath string) (err error)
• func PrintInfo()
• func SetColoredLogs(colored bool)
• func SetDebug(enabled bool)
• func SetFileSearchCaseSensitive(caseSensitive bool)
• func Unmarshal(data []byte, config interface{}) (err error)
• type Environment
  • func Env() *Environment
  • func (e *Environment) AppendExp(exp string)
  • func (e *Environment) ID() string
  • func (e *Environment) Info() string
  • func (e *Environment) MatchTag(tag string) bool
  • func (e *Environment) PrintInfo()
  • func (e *Environment) SetExps(exps []string)
• type Repository
  • func NewRepository(path string) *Repository
  • func (r *Repository) Info() string
  • func (r *Repository) PrintInfo()
  • func (r *Repository) UpdateInfo()

Constants

const EnvVarKey = "BUILD_ENV"

EnvVarKey is the environment variable that determine the build environment in sprbox.

Variables

var (
	// BUILDENV define the current environment.
	// Can be defined by code or, since it is an exported string,
	// can be interpolated with -ldflags at build/run time:
	// 	go build -ldflags "-X github.com/oblq/goms/env.TAG=develop" -v -o ./api_bin ./api
	//
	// If TAG is empty then the environment variable 'BUILD_ENV' will be used.
	//
	// If also the 'BUILD_ENV' environment variable is empty,
	// if you have setup the VCS var, then the git.BranchName will be used.
	// Git-Flow automatic environment selection based on branch name is also supported.
	// Here the default environment RegEx, you can customize them as you want:
	//  - Production 	exps: Exps{"production", "master"}
	//	- Staging 	exps: Exps{"staging", "release/*", "hotfix/*"}
	//	- Testing 	exps: Exps{"testing", "test"}
	//	- Development 	exps: Exps{"development", "develop", "dev", "feature/*"}
	//	- Local 	exps: Exps{"local"}
	BUILDENV = ""

	// VCS is the project version control system.
	// By default it uses the working directory.
	VCS = NewRepository("./")
)

var (
	Production  = &Environment{id: "production", exps: []string{"production", "master"}, RunCompiled: true}
	Staging     = &Environment{id: "staging", exps: []string{"staging", "release/*", "hotfix/*"}, RunCompiled: true}
	Testing     = &Environment{id: "testing", exps: []string{"testing", "test"}, RunCompiled: false}
	Development = &Environment{id: "development", exps: []string{"development", "develop", "dev", "feature/*"}, RunCompiled: false}
	Local       = &Environment{id: "local", exps: []string{"local"}, RunCompiled: false}
)

Default environment's configuration

Functions

func CompiledPath

func CompiledPath(path string) string

CompiledPath returns the path base if RunCompiled == true for the environment in use so that static files can stay side by side with the executable while it is possible to have a different location when the program is launched with `go run`. This allow to manage multiple packages in one project during development, for instance using a config path in the parent dir, side by side with the packages, while having the same config folder side by side with the executable where needed.

Can be used in:

sprbox.LoadToolBox(&myToolBox, sprbox.CompiledPath("../config"))

Example:

sprbox.Development.RunCompiled = false
sprbox.BUILDENV = sprbox.Development.ID()
sprbox.CompiledPath("../static_files/config") // -> "../static_files/config"

sprbox.Development.RunCompiled = true
sprbox.BUILDENV = sprbox.Development.ID()
sprbox.CompiledPath("../static_files/config") // -> "config"

By default only Production and Staging environments have RunCompiled = true.

func EnvSubDir

func EnvSubDir(path string) string

EnvSubDir returns <path>/<environment>

func GetInfo

func GetInfo(w http.ResponseWriter, r *http.Request)

GetInfo print info in console from an http request.

func LoadConfig

func LoadConfig(config interface{}, files ...string) (err error)

LoadConfig will unmarshal all the matched config files to the config interface.

Build-environment specific files will override generic files. The latest files will override the earliest.

Will also parse struct flags.

func LoadToolBox

func LoadToolBox(toolBox interface{}, configPath string) (err error)

LoadToolBox initialize and (eventually) configure the provided struct pointer looking for the config files in the provided configPath.

func PrintInfo

func PrintInfo()

PrintInfo print some useful info about the environment and git.

func SetColoredLogs

func SetColoredLogs(colored bool)

SetColoredLogs toggle colors in console.

func SetDebug

func SetDebug(enabled bool)

SetDebug will print detailed logs in console.

func SetFileSearchCaseSensitive

func SetFileSearchCaseSensitive(caseSensitive bool)

SetFileSearchCaseSensitive toggle case sensitive cinfig files search.

func Unmarshal

func Unmarshal(data []byte, config interface{}) (err error)

Unmarshal will unmarshal []byte to interface for yaml, toml and json data formats.

Will also parse struct flags.

Types

type Environment

type Environment struct {

	// RunCompiled true means that the program run from
	// a precompiled binary for that environment.
	// CompiledPath() returns the path base if RunCompiled == true
	// so that static files can stay side by side with the executable
	// while it is possible to have a different location when the
	// program is launched with `go run`.
	//
	// By default only Production and Staging environments have RunCompiled = true.
	RunCompiled bool
	// contains filtered or unexported fields
}

Environment struct.

func Env

func Env() *Environment

Env returns the current selected environment by matching the privateTAG variable against the environments RegEx.

func (*Environment) AppendExp

func (e *Environment) AppendExp(exp string)

AppendExp add a regular expression to match that environment.

func (*Environment) ID

func (e *Environment) ID() string

ID returns the environment id, which are also a valid tag for the current environment.

func (*Environment) Info

func (e *Environment) Info() string

Info return some environment info.

func (*Environment) MatchTag

func (e *Environment) MatchTag(tag string) bool

MatchTag check if the passed tag match that environment, a tag may be the branch name or the machine hostname or whatever you want.

func (*Environment) PrintInfo

func (e *Environment) PrintInfo()

PrintInfo print some environment info in console.

func (*Environment) SetExps

func (e *Environment) SetExps(exps []string)

SetExps set regular expressions to match that environment.

type Repository

type Repository struct {
	Path                           string
	BranchName, Commit, Build, Tag string
	Error                          error
}

Repository represent a git repository

func NewRepository

func NewRepository(path string) *Repository

NewRepository return a new Repository instance for the given path

func (*Repository) Info

func (r *Repository) Info() string

Info return Git repository info.

func (*Repository) PrintInfo

func (r *Repository) PrintInfo()

PrintInfo print git data in console.

func (*Repository) UpdateInfo

func (r *Repository) UpdateInfo()

UpdateInfo grab git info and set 'Error' var eventually.