cnfg

README

golift.io/cnfg

Procedures for parsing configs files and environment variables into data structures. Works a whole lot like json.Unmarshal Quick explanation on how the env variable mapping works below. See GODOC for a working code example.

Supports almost every possible type, including slices, maps, slices of maps, maps of slices, pointers of slices to maps of slices!

Please open an issue if you run into a bug or an unsupported type.

type Shelter struct {
	Title  string    `xml:"title"`
	Sym    float64   `xml:"sym"`
	People []*Person `xml:"people"`
	Dogs   []*Dog    `xml:"dogs"`
}

type Person struct {
	Name    string `xml:"name"`
	Present bool   `xml:"present"`
	Age     int    `xml:"age"`
	ID      int64  `xml:"id"`
}

type Dog struct {
	Name    string
	Elapsed config.Duration
	Owners  []string
}

type Config struct {
	*Shelter `xml:"shelter"`
}

The above struct can be configured with the following environment variables, assuming you set prefix := "APP" when you call UnmarshalENV(). Slices use env vars with numbers in them, starting at 0 and going to infinity, or the last env var provided + 1, whichever comes first. It just works. The ... and ++ indicate that those parameters belong to slices, and many items may be appended or overridden.

APP_SHELTER_TITLE
APP_SHELTER_SYM
APP_SHELTER_PEOPLE_0_NAME
APP_SHELTER_PEOPLE_0_PRESENT
APP_SHELTER_PEOPLE_0_AGE
APP_SHELTER_PEOPLE_0_ID
APP_SHELTER_PEOPLE_1_NAME
...
APP_SHELTER_PEOPLE_10_ID ++

APP_SHELTER_DOGS_0_NAME
APP_SHELTER_DOGS_0_ELAPSED
APP_SHELTER_DOGS_0_OWNERS_0
...
APP_SHELTER_DOGS_0_OWNERS_10 ++

APP_SHELTER_DOGS_1_NAME
APP_SHELTER_DOGS_1_ELAPSED
APP_SHELTER_DOGS_1_OWNERS_0
APP_SHELTER_DOGS_1_OWNERS_1 ++

If you passed in the Shelter struct instead of Config, all the of the SHELTER_ portions of the tags would be omitted. You can also set which struct tag to use by creating an &ENV{} pointer and setting Tag and/or Pfx . Tag defaults to "xml", but you could set it to "env" and make custom names for env variables. The env var prefix Pfx is optional, but recommended.

Documentation

Overview

Package cnfg provide basic procedures to parse a config file into a struct, and more powerfully, parse a slew of environment variables into the same or a different struct. These two procedures can be used one after the other in either order (the latter overrides parts of the former).

If this package interests you, pull requests and feature requests are welcomed!

I consider this package the pinacle example of how to configure small Go applications from a file. You can put your configuration into any file format: XML, YAML, JSON, TOML, and you can override any struct member using an environment variable. As it is now, the (env) code lacks map{} support but pretty much any other base type and nested member is supported. Adding more/the rest will happen in time. I created this package because I got tired of writing custom env parser code for every app I make. This simplifies all the heavy lifting and I don't even have to think about it now.

Index

• Constants
• func UnmarshalENV(i interface{}, prefix string) (bool, error)
• func UnmarshalFile(c interface{}, configFile ...string) error
• func UnmarshalMap(pairs map[string]string, i interface{}) (bool, error)
• type Duration
  • func (d *Duration) UnmarshalText(b []byte) (err error)
• type ENV
  • func (e *ENV) Unmarshal(i interface{}) (bool, error)
  • func (e *ENV) UnmarshalMap(pairs map[string]string, i interface{}) (bool, error)
• type ENVUnmarshaler
• type Pairs
  • func MapEnvPairs(prefix string, pairs []string) Pairs
  • func (p *Pairs) Get(prefix string) Pairs

Examples

• ENV.Unmarshal (Simple)
• ENVUnmarshaler

Constants

const ENVTag = "xml"

ENVTag is the tag to look for on struct members. You may choose to use a custom tag by creating an &ENV{} struct with a different Tag. "env" is popular, but I chose "xml" because the nouns are generally singular, and those look good as env variables. "xml" is also convenient because it's brief and doesn't add yet another struct tag. Those lines can get long quickly.

Functions

func UnmarshalENV

func UnmarshalENV(i interface{}, prefix string) (bool, error)

UnmarshalENV copies environment variables into configuration values. This is useful for Docker users that find it easier to pass ENV variables than a specific configuration file. Uses reflection to find struct tags.

func UnmarshalFile

func UnmarshalFile(c interface{}, configFile ...string) error

UnmarshalFile parses a configuration file (of any format) into a config struct. This is a shorthand method for calling Unmarshal against the json, xml, yaml or toml packages. If the file name contains an appropriate suffix it is unmarshaled with the corresponding package. If the suffix is missing, TOML is assumed. Works with multiple files, so you can have stacked configurations.

func UnmarshalMap

func UnmarshalMap(pairs map[string]string, i interface{}) (bool, error)

UnmarshalMap parses and processes a map of key/value pairs as though they were environment variables. Useful for testing, or unmarshaling values from places other than environment variables. This version of UnmarshalMap assumes default tag ("xml") and no prefix: ""

Types

type Duration

type Duration struct{ time.Duration }

Duration is useful if you need to load a time Duration from a config file into your application. Use the config.Duration type to support automatic unmarshal from all sources. If you do not use a config file, do not use this type because the environment unmarshaler supports time.Duration natively.

func (*Duration) UnmarshalText

func (d *Duration) UnmarshalText(b []byte) (err error)

UnmarshalText parses a duration type from a config file. This method works with the Duration type to allow unmarshaling of durations from files and env variables in the same struct. You won't generally call this directly.

type ENV

type ENV struct {
	Tag string // Struct tag name.
	Pfx string // ENV var prefix.
}

ENV allows you to parse environment variables using an object instead of global state. This package allows using the default ENVTag from global state, or you can pass in your own using this struct. See the UnmarshalENV function (it's 1 line) for an example of how to use this.

func (*ENV) Unmarshal

func (e *ENV) Unmarshal(i interface{}) (bool, error)

Unmarshal parses and processes environment variables into the provided interface. Uses the Prefix and Tag name from the &ENV{} struct values.

Example (Simple)

Complete working example for UnmarshalENV()

// Systems is used to show an example of how to access nested slices.
type System struct {
	Name   string `env:"name"`
	Signal *[]int `env:"signal"`
}

// Config represents your application's environment variable based config inputs.
// Works with or without pointers.
type Config struct {
	Debug    bool           `env:"debug"`
	Users    []string       `env:"user"`
	Interval *time.Duration `env:"interval"`
	Systems  []*System      `env:"system"`
}

// Make a pointer to your struct with some default data.
// Maybe this data came from a config file? Using ParseFile()!
c := &Config{
	Debug: true,
	Users: []string{"me", "you", "them"},
}

// Okay set some ENV variables. Pretend you did this in bash.
os.Setenv("APP_DEBUG", "false")   // turn off debug
os.Setenv("APP_USER_1", "dad")    // replace "you" with "dad"
os.Setenv("APP_USER_3", "mom")    // add "mom"
os.Setenv("APP_INTERVAL", "7m1s") // don't forget the interval!!

// This adds (creates) systems and signals in sub-slices.
os.Setenv("APP_SYSTEM_0_NAME", "SysWon")
os.Setenv("APP_SYSTEM_1_NAME", "SysToo")
os.Setenv("APP_SYSTEM_1_SIGNAL_0", "12")
// You can add as many as you like, as long as they are in numerical order.
os.Setenv("APP_SYSTEM_1_SIGNAL_1", "77")

fmt.Printf("BEFORE => Debug: %v, Interval: %v, Users: %v, Systems: %v\n",
	c.Debug, c.Interval, c.Users, c.Systems)

// Make a ENV Decoder with special tag and prefix.
env := &ENV{Tag: "env", Pfx: "APP"}

// Run Unmarshal to parse the values into your config pointer:
ok, err := env.Unmarshal(c)
if err != nil {
	panic(err)
}

// And optionally, do something with the "ok" return value.
// If you wanted to overwrite ALL configs if ANY env variables are present
// you could use ok to make and if statement that does that.
if ok {
	fmt.Println("~ Environment variables were parsed into the config!")
}

// If you don't set an env variable for it, it will stay nil.
// Same for structs and slices.
if c.Interval == nil {
	fmt.Printf("You forgot to set an interval!")
	return
}

fmt.Printf("AFTER => Debug: %v, Interval: %v, Users: %v\n", c.Debug, *c.Interval, c.Users)
// We added some systems, check them!
for i, s := range c.Systems {
	fmt.Printf(" %v: System Name: %v, Signals: %v\n", i, s.Name, s.Signal)
}

Output:

BEFORE => Debug: true, Interval: <nil>, Users: [me you them], Systems: []
~ Environment variables were parsed into the config!
AFTER => Debug: false, Interval: 7m1s, Users: [me dad them mom]
 0: System Name: SysWon, Signals: <nil>
 1: System Name: SysToo, Signals: &[12 77]

func (*ENV) UnmarshalMap

func (e *ENV) UnmarshalMap(pairs map[string]string, i interface{}) (bool, error)

UnmarshalMap parses and processes a map of key/value pairs as though they were environment variables. Useful for testing, or unmarshaling values from places other than environment variables. Use this version of UnmarshalMap if you need to change the tag or prefix.

type ENVUnmarshaler

type ENVUnmarshaler interface {
	UnmarshalENV(tag, envval string) error
}

ENVUnmarshaler allows custom unmarshaling on a custom type. If your type implements this, it will be called and the logic stops there.

Example

This simple example shows how you may use the ENVUnmarshaler interface. This shows how to use two environment variables to set one custom value.

package main

import (
	"fmt"
	"os"
	"strconv"
	"time"
)

// TimeX uses two environment variables to multiply a duration.
type TimeX struct {
	time.Duration
}

// This is a test to make sure our struct satisfies the interface.
var _ ENVUnmarshaler = (*TimeX)(nil)

type AppConfig struct {
	Name    string `xml:"name"`
	Special TimeX  `xml:"in"`
}

func (t *TimeX) UnmarshalENV(tag, val string) error {
	xTag := tag + "_X"

	xString, ok := os.LookupEnv(xTag)
	if !ok {
		xString = "1"
	}

	multiplier, err := strconv.Atoi(xString)
	if err != nil {
		return fmt.Errorf("multiplier invalid %s: %v", xTag, err)
	}

	t.Duration, err = time.ParseDuration(val)
	if err != nil {
		return fmt.Errorf("duration invalid %s: %v", tag, err)
	}

	t.Duration *= time.Duration(multiplier)

	return nil
}

// This simple example shows how you may use the ENVUnmarshaler interface.
// This shows how to use two environment variables to set one custom value.
func main() {
	c := &AppConfig{}

	os.Setenv("APP_IN", "5m")
	os.Setenv("APP_IN_X", "10")
	os.Setenv("APP_NAME", "myApp")

	_, err := UnmarshalENV(c, "APP")
	if err != nil {
		panic(err)
	}

	fmt.Printf("%s starts in %v", c.Name, c.Special)
}

Output:

myApp starts in 50m0s

type Pairs

type Pairs map[string]string

Pairs represents pairs of environment variables.

func MapEnvPairs

func MapEnvPairs(prefix string, pairs []string) Pairs

MapEnvPairs turns the pairs returned by os.Environ() into a map[string]string. Providing a prefix returns only variables with that prefix.

func (*Pairs) Get

func (p *Pairs) Get(prefix string) Pairs

Get allows getting only specific env variables by prefix. The prefix is trimmed before returning.