envconfig

README

envconfig

envconfig is a library which allows you to parse your configuration from environment variables and fill an arbitrary struct.

See the example to understand how to use it, it's pretty simple.

Supported types

• Almost all standard types plus time.Duration are supported by default.
• Slices and arrays
• Arbitrary structs
• Custom types via the Unmarshaler interface.

How does it work

envconfig takes the hierarchy of your configuration struct and the names of the fields to create a environment variable key.

For example:

var conf struct {
    Name string
    Shard struct {
        Host string
        Port int
    }
}

This will check for those 3 keys:

• NAME or name
• SHARD_HOST, or shard_host
• SHARD_PORT, or shard_port

Flexible key naming

envconfig supports having underscores in the key names where there is a word boundary. Now, that term is not super explicit, so let me show you an example:

var conf struct {
    Cassandra struct {
        SSLCert string
        SslKey string
    }
}

This will check all of the following keys:

• CASSANDRA_SSL_CERT, CASSANDRA_SSLCERT, cassandra_ssl_cert, cassandra_sslcert
• CASSANDRA_SSL_KEY, CASSANDRA_SSLKEY, cassandra_ssl_key, cassandra_sslkey

If that is not good enough, look just below.

Custom environment variable names

envconfig supports custom environment variable names:

var conf struct {
    Name string `envconfig:"myName"`
}

Default values

envconfig supports default values:

var conf struct {
    Name string `envconfig:"default=Vincent"`
}

Optional values

envconfig supports optional values:

var conf struct {
    Name string `envconfig:"optional"`
}

Skipping fields

envconfig supports skipping struct fields:

var conf struct {
    Internal string `envconfig:"-"`
}

Combining multiple options in one tag

You can of course combine multiple options:

var conf struct {
    Name string `envconfig:"default=Vincent,myName"`
}

Slices or arrays

With slices or arrays, the same naming is applied for the slice. To put multiple elements into the slice or array, you need to separate them with a , (will probably be configurable in the future, or at least have a way to escape)

For example:

var conf struct {
    Ports []int
}

This will check for the key PORTS:

• if your variable is 9000 the slice will contain only 9000
• if your variable is 9000,100 the slice will contain 9000 and 100

For slices of structs, it's a little more complicated. The same splitting of slice elements is done with a comma, however, each token must follow a specific format like this: {<first field>,<second field>,...}

For example:

var conf struct {
    Shards []struct {
        Name string
        Port int
    }
}

This will check for the key SHARDS. Example variable content: {foobar,9000},{barbaz,20000}

This will result in two struct defined in the Shards slice.

If you want to set default value for slice or array, you have to use ; as separator, instead of ,:

var conf struct {
    Ports []int `envconfig:"default=9000;100"`
}

Same for slices of structs:

var conf struct {
    Shards []struct {
        Name string
        Port int
    } `envconfig:"default={foobar;localhost:2929};{barbaz;localhost:2828}"`
}

Development state

I consider envconfig to be pretty much done.

It has been used extensively at Batch for more than 5 years now without much problems, with no need for new features either.

So, while I will keep maintaining this library (fixing bugs, making it compatible with new versions of Go and so on) for the foreseeable future, I don't plan on adding new features.

But I'm open to discussion so if you have a need for a particular feature we can discuss it.

Documentation

Overview

Package envconfig implements a configuration reader which reads each value from an environment variable.

The basic idea is that you define a configuration struct, like this:

var conf struct {
    Addr string
    Port int
    Auth struct {
        Key      string
        Endpoint string
    }
    Partitions []int
    Shards     []struct {
        Name string
        Id   int
    }
}

Once you have that, you need to initialize the configuration:

if err := envconfig.Init(&conf); err != nil {
    log.Fatalln(err)
}

Then it's just a matter of setting the environment variables when calling your binary:

ADDR=localhost PORT=6379 AUTH_KEY=foobar ./mybinary

Layout of the conf struct

Your conf struct must follow the following rules:

• no unexported fields by default (can turn off with Options.AllowUnexported)
• only supported types (no map fields for example)

Naming of the keys

By default, envconfig generates all possible keys based on the field chain according to a flexible naming scheme.

The field chain is how you access your field in the configuration struct. For example:

var conf struct {
    Shard struct {
        Name string
    }
}

With that struct, you access the name field via the chain *Shard.Name*

The default naming scheme takes that and transforms it into the following:

• SHARD_NAME
• shard_name

It can handles more complicated cases, with multiple words in one field name. It needs to be in the correct case though, for example:

var conf struct {
    Cassandra struct {
        SSLCert string
        SslKey  string
    }
}

With that struct, you access the name field via the chain *Cassandra.SSLCert* or *Cassandra.SslKey*

The default naming scheme takes that and transforms it into the following:

• CASSANDRA_SSL_CERT, cassandra_ssl_cert, CASSANDRA_SSLCERT, cassandra_sslcert
• CASSANDRA_SSL_KEY, cassandra_ssl_key, CASSANDRA_SSLKEY, cassandra_sslkey

And, if that is not good enough for you, you always have the option to use a custom key:

var conf struct {
    Cassandra struct {
        Name string `envconfig:"cassandraMyName"`
    }
}

Now envconfig will only ever checks the environment variable _cassandraMyName_.

Content of the variables

There are three types of content for a single variable:

• for simple types, a single string representing the value, and parseable into the type.
• for slices or arrays, a comma-separated list of strings. Each string must be parseable into the element type of the slice or array.
• for structs, a comma-separated list of specially formatted strings representing structs.

Example of a valid slice value:

foo,bar,baz

The format for a struct is as follow:

• prefixed with {
• suffixed with }
• contains a comma-separated list of field values, in the order in which they are defined in the struct

Example of a valid struct value:

type MyStruct struct {
    Name    string
    Id      int
    Timeout time.Duration
}

{foobar,10,120s}

Example of a valid slice of struct values:

{foobar,10,120s},{barbaz,20,50s}

Special case for bytes slices

For bytes slices, you generally don't want to type out a comma-separated list of byte values.

For this use case, we support base64 encoded values.

Here's an example:

var conf struct {
    Data []byte
}

os.Setenv("DATA", "Rk9PQkFS")

This will decode DATA to FOOBAR and put that into conf.Data.

Optional values

Sometimes you don't absolutely need a value. Here's how we tell envconfig a value is optional:

var conf struct {
    Name string `envconfig:"optional"`
}

Skipped fields

Sometimes you want a field to be skipped entirely.

    var conf struct {
	Internal string `envconfig:"-"`
    }

Default values

Often times you have configuration keys which almost never changes, but you still want to be able to change them.

In such cases, you might want to provide a default value.

Here's to do this with envconfig:

var conf struct {
    Timeout time.Duration `envconfig:"default=1m"`
}

Combining options

You can of course combine multiple options. The syntax is simple enough, separate each option with a comma.

For example:

var conf struct {
    Timeout time.Duration `envconfig:"default=1m,myTimeout"`
}

This would give you the default timeout of 1 minute, and lookup the myTimeout environment variable.

Supported types

envconfig supports the following list of types:

• bool
• string
• intX
• uintX
• floatX
• time.Duration
• pointers to all of the above types

Notably, we don't (yet) support complex types simply because I had no use for it yet.

Custom unmarshaler

When the standard types are not enough, you will want to use a custom unmarshaler for your types.

You do this by implementing Unmarshaler on your type. Here's an example:

type connectionType uint

const (
    tlsConnection connectionType = iota
    insecureConnection
)

func (t *connectionType) Unmarshal(s string) error {
    switch s {
        case "tls":
            *t = tlsConnection
        case "insecure":
            *t = insecureConnection
        default:
            return fmt.Errorf("unable to unmarshal %s to a connection type", s)
    }

    return nil
}

Index

• Variables
• func Init(conf interface{}) error
• func InitWithOptions(conf interface{}, opts Options) error
• func InitWithPrefix(conf interface{}, prefix string) error
• type Options
• type Unmarshaler

Examples

• Init
• InitWithPrefix

Variables

var (
	// ErrUnexportedField is the error returned by the Init* functions when a field of the config struct is not exported and the option AllowUnexported is not used.
	ErrUnexportedField = errors.New("envconfig: unexported field")
	// ErrNotAPointer is the error returned by the Init* functions when the configuration object is not a pointer.
	ErrNotAPointer = errors.New("envconfig: value is not a pointer")
	// ErrInvalidValueKind is the error returned by the Init* functions when the configuration object is not a struct.
	ErrInvalidValueKind = errors.New("envconfig: invalid value kind, only works on structs")
)

Functions

func Init

func Init(conf interface{}) error

Init reads the configuration from environment variables and populates the conf object. conf must be a pointer

Example

package main

import (
	"fmt"
	"os"
	"time"

	"github.com/vrischmann/envconfig"
)

func main() {
	var conf struct {
		MySQL struct {
			Host     string
			Port     int
			Database struct {
				User     string
				Password string
				Name     string
			}
			Params struct {
				Charset string `envconfig:"-"`
			}
		}
		Log struct {
			Path   string `envconfig:"default=/var/log/mylog.log"`
			Rotate bool   `envconfig:"logRotate"`
		}
		NbWorkers int
		Timeout   time.Duration
		Cassandra struct {
			SSLCert string
			SSLKey  string
		}
	}

	os.Setenv("MYSQL_HOST", "localhost")
	os.Setenv("MYSQL_PORT", "3306")
	os.Setenv("MYSQL_DATABASE_USER", "root")
	os.Setenv("MYSQL_DATABASE_PASSWORD", "foobar")
	os.Setenv("MYSQL_DATABASE_NAME", "default")
	os.Setenv("logRotate", "true")
	os.Setenv("NBWORKERS", "10")
	os.Setenv("TIMEOUT", "120s")
	os.Setenv("CASSANDRA_SSL_CERT", "/etc/cassandra/ssl.crt")
	os.Setenv("CASSANDRA_SSL_KEY", "/etc/cassandra/ssl.key")

	if err := envconfig.Init(&conf); err != nil {
		fmt.Printf("err=%s\n", err)
	}

	fmt.Println(conf.MySQL.Database.User)
	fmt.Println(conf.Log.Rotate)
	fmt.Println(conf.Timeout)
	fmt.Println(conf.Log.Path)
	fmt.Println(conf.Cassandra.SSLCert)
	fmt.Println(conf.Cassandra.SSLKey)
}

Output:

root
true
2m0s
/var/log/mylog.log
/etc/cassandra/ssl.crt
/etc/cassandra/ssl.key

func InitWithOptions

func InitWithOptions(conf interface{}, opts Options) error

InitWithOptions reads the configuration from environment variables and populates the conf object. conf must be a pointer.

func InitWithPrefix

func InitWithPrefix(conf interface{}, prefix string) error

InitWithPrefix reads the configuration from environment variables and populates the conf object. conf must be a pointer. Each key read will be prefixed with the prefix string.

Example

package main

import (
	"fmt"
	"os"

	"github.com/vrischmann/envconfig"
)

func main() {
	var conf struct {
		Name string
	}

	os.Setenv("NAME", "")
	os.Setenv("FOO_NAME", "")

	os.Setenv("NAME", "foobar")

	err := envconfig.InitWithPrefix(&conf, "FOO")
	fmt.Println(err)

	os.Setenv("FOO_NAME", "foobar")
	err = envconfig.InitWithPrefix(&conf, "FOO")
	fmt.Println(err)

	fmt.Println(conf.Name)
}

Output:

envconfig: keys FOO_NAME, foo_name not found
<nil>
foobar

Types

type Options

type Options struct {
	// Prefix allows specifying a prefix for each key.
	Prefix string

	// AllOptional determines whether to not throw errors by default for any key
	// that is not found. AllOptional=true means errors will not be thrown.
	AllOptional bool

	// LeaveNil specifies whether to not create new pointers for any pointer fields
	// found within the passed config. Rather, it behaves such that if and only if
	// there is a) a non-empty field in the value or b) a non-empty value that
	// the pointer is pointing to will a new pointer be created. By default,
	// LeaveNil=false will create all pointers in all structs if they are nil.
	//
	//	var X struct {
	//		A *struct{
	//			B string
	//		}
	//	}
	//	envconfig.InitWithOptions(&X, Options{LeaveNil: true})
	//
	// $ ./program
	//
	// X.A == nil
	//
	// $ A_B="string" ./program
	//
	// X.A.B="string" // A will not be nil
	LeaveNil bool

	// AllowUnexported allows unexported fields to be present in the passed config.
	AllowUnexported bool
}

Options is used to customize the behavior of envconfig. Use it with InitWithOptions.

type Unmarshaler

type Unmarshaler interface {
	Unmarshal(s string) error
}

Unmarshaler is the interface implemented by objects that can unmarshal a environment variable string of themselves.