cleanenv

README

Clean Env

Minimalistic configuration reader

Overview

This is a simple configuration reading tool. It just does the following:

• reads and parses configuration structure from the file
• reads and overwrites configuration structure from environment variables
• writes a detailed variable list to help output

Content

• Installation
• Usage
  • Read Configuration
  • Read Environment Variables Only
  • Update Environment Variables
  • Description
• Model Format
• Supported types
• Custom Functions
  • Custom Value Setter
  • Custom Value Update
• Supported File Formats
• Integration
  • Flag
• Examples
• Contribution
• Thanks

Installation

To install the package run

go get -u github.com/ilyakaznacheev/cleanenv

Usage

The package is oriented to be simple in use and explicitness.

The main idea is to use a structured configuration variable instead of any sort of dynamic set of configuration fields like some libraries does, to avoid unnecessary type conversions and move the configuration through the program as a simple structure, not as an object with complex behavior.

There are just several actions you can do with this tool and probably only things you want to do with your config if your application is not too complicated.

• read configuration file
• read environment variables
• read some environment variables again

Read Configuration

You can read a configuration file and environment variables in a single function call.

import "github.com/ilyakaznacheev/cleanenv"

type ConfigDatabase struct {
    Port     string `yaml:"port" env:"PORT" env-default:"5432"`
    Host     string `yaml:"host" env:"HOST" env-default:"localhost"`
    Name     string `yaml:"name" env:"NAME" env-default:"postgres"`
    User     string `yaml:"user" env:"USER" env-default:"user"`
    Password string `yaml:"password" env:"PASSWORD"`
}

var cfg ConfigDatabase

err := cleanenv.ReadConfig("config.yml", &cfg)
if err != nil {
    ...
}

This will do the following:

1. parse configuration file according to YAML format (yaml tag in this case);
2. reads environment variables and overwrites values from the file with the values which was found in the environment (env tag);
3. if no value was found on the first two steps, the field will be filled with the default value (env-default tag) if it is set.

Read Environment Variables Only

Sometimes you don't want to use configuration files at all, or you may want to use .env file format instead. Thus, you can limit yourself with only reading environment variables:

import "github.com/ilyakaznacheev/cleanenv"

type ConfigDatabase struct {
    Port     string `env:"PORT" env-default:"5432"`
    Host     string `env:"HOST" env-default:"localhost"`
    Name     string `env:"NAME" env-default:"postgres"`
    User     string `env:"USER" env-default:"user"`
    Password string `env:"PASSWORD"`
}

var cfg ConfigDatabase

err := cleanenv.ReadEnv(&cfg)
if err != nil {
    ...
}

Update Environment Variables

Some environment variables may change during the application run. To get the new values you need to mark these variables as updatable with the tag env-upd and then run the update function:

import "github.com/ilyakaznacheev/cleanenv"

type ConfigRemote struct {
    Port     string `env:"PORT" env-upd`
    Host     string `env:"HOST" env-upd`
    UserName string `env:"USERNAME"`
}

var cfg ConfigRemote

cleanenv.ReadEnv(&cfg)

// ... some actions in-between

err := cleanenv.UpdateEnv(&cfg)
if err != nil {
    ...
}

Here remote host and port may change in a distributed system architecture. Fields cfg.Port and cfg.Host can be updated in the runtime from corresponding environment variables. You can update them before the remote service call. Field cfg.UserName will not be changed after the initial read, though.

Description

You can get descriptions of all environment variables to use them in the help documentation.

import "github.com/ilyakaznacheev/cleanenv"

type ConfigServer struct {
    Port     string `env:"PORT" env-description:"server port"`
    Host     string `env:"HOST" env-description:"server host"`
}

var cfg ConfigRemote

help, err := cleanenv.GetDescription(&cfg, nil)
if err != nil {
    ...
}

You will get the following:

Environment variables:
  PORT  server port
  HOST  server host

Model Format

Library uses tags to configure the model of configuration structure. There are the following tags:

• env="<name>" - environment variable name (e.g. env="PORT");
• env-upd - flag to mark a field as updatable. Run UpdateEnv(&cfg) to refresh updatable variables from environment;
• env-required - flag to mark a field as required. If set will return an error during environment parsing when the flagged as required field is empty (default Go value). Tag env-default is ignored in this case;
• env-default="<value>" - default value. If the field wasn't filled from the environment variable default value will be used instead;
• env-separator="<value>" - custom list and map separator. If not set, the default separator , will be used;
• env-description="<value>" - environment variable description;
• env-layout="<value>" - parsing layout (for types like time.Time);
• env-prefix="<value>" - prefix for all fields of nested structure (only for nested structures);

Supported types

There are following supported types:

• int (any kind);
• float (any kind);
• string;
• boolean;
• slices (of any other supported type);
• maps (of any other supported type);
• time.Duration;
• time.Time (layout by default is RFC3339, may be overridden by env-layout);
• *time.Location (time zone parsing depends on running machine);
• any type that implements encoding.TextUnmarshaler;
• any type implementing cleanenv.Setter interface.

Custom Functions

To enhance package abilities you can use some custom functions.

Custom Value Setter

To make custom type allows to set the value from the environment variable, you need to implement the Setter interface on the field level:

type MyField string

func (f *MyField) SetValue(s string) error  {
    if s == "" {
        return fmt.Errorf("field value can't be empty")
    }
    *f = MyField("my field is: "+ s)
    return nil
}

type Config struct {
    Field MyField `env="MY_VALUE"`
}

SetValue method should implement conversion logic from string to custom type.

Custom Value Update

You may need to execute some custom field update logic, e.g. for remote config load.

Thus, you need to implement the Updater interface on the structure level:

type Config struct {
    Field string
}

func (c *Config) Update() error {
    newField, err := SomeCustomUpdate()
    f.Field = newField
    return err
}

Supported File Formats

There are several most popular config file formats supported:

• YAML (.yaml, .yml)
• JSON (.json)
• TOML (.toml)
• EDN (.edn)
• ENV (.env)

Note:

• while using .env file the library will set corresponding data to process environment variables. It will override existing variables with the same keys in the process environment.

Integration

The package can be used with many other solutions. To make it more useful, we made some helpers.

Flag

You can use the cleanenv help together with Golang flag package.

// create some config structure
var cfg config 

// create flag set using `flag` package
fset := flag.NewFlagSet("Example", flag.ContinueOnError)

// get config usage with wrapped flag usage
fset.Usage = cleanenv.FUsage(fset.Output(), &cfg, nil, fset.Usage)

fset.Parse(os.Args[1:])

Examples

type Config struct {
    Port string `yaml:"port" env:"PORT" env-default:"8080"`
    Host string `yaml:"host" env:"HOST" env-default:"localhost"`
}

var cfg Config

err := ReadConfig("config.yml", &cfg)
if err != nil {
    ...
}

This code will try to read and parse the configuration file config.yml as the structure is described in the Config structure. Then it will overwrite fields from available environment variables (PORT, HOST).

For more details check the example directory.

Version Support Policy

We support the last 7 versions of Golang. E.g. if the current version is 1.19, we test compatibility with all versions from 1.19 to 1.13.

If you use an older version of Golang in your project, please use an older library version.

Contribution

The tool is open-sourced under the MIT license.

If you find some error, want to add something or ask a question - feel free to create an issue and/or make a pull request.

Guidelines for contribution may be found in CONTRIBUTING.md.

Any contribution is welcome.

Thanks

Big thanks to a project kelseyhightower/envconfig for inspiration.

The logo was made by alexchoffy.

Blog Posts

Clean Configuration Management in Golang.

Documentation

Overview

Package cleanenv gives you a single tool to read application configuration from several sources with ease.

Features

- read from several file formats (YAML, JSON, TOML, ENV, EDN) and parse into the internal structure;

- read environment variables into the internal structure;

- output environment variable list with descriptions into help output;

- custom variable readers (e.g. if you want to read from remote config server, etc).

Usage

You can just prepare the config structure and fill it from the config file and environment variables.

type Config struct {
	Port string `yaml:"port" env:"PORT" env-default:"8080"`
	Host string `yaml:"host" env:"HOST" env-default:"localhost"`
}

var cfg Config

ReadConfig("config.yml", &cfg)

Help output

You can list all of your environment variables by means of help output:

type ConfigServer struct {
	Port     string `env:"PORT" env-description:"server port"`
	Host     string `env:"HOST" env-description:"server host"`
}

var cfg ConfigRemote

help, err := cleanenv.GetDescription(&cfg, nil)
if err != nil {
	...
}

// setup help output
f := flag.NewFlagSet("Example app", 1)
fu := f.Usage
f.Usage = func() {
	fu()
	envHelp, _ := cleanenv.GetDescription(&cfg, nil)
	fmt.Fprintln(f.Output())
	fmt.Fprintln(f.Output(), envHelp)
}

f.Parse(os.Args[1:])

Then run go run main.go -h and the output will include:

Environment variables:
	PORT  server port
	HOST  server host

For more detailed information check examples and example tests.

Example (Setter)

Example_setter uses type with a custom setter to parse environment variable data

package main

import (
	"fmt"
	"os"

	"github.com/ilyakaznacheev/cleanenv"
)

// MyField1 is an example type with a custom setter
type MyField1 string

func (f *MyField1) SetValue(s string) error {
	if s == "" {
		return fmt.Errorf("field value can't be empty")
	}
	*f = MyField1("my field is: " + s)
	return nil
}

func (f MyField1) String() string {
	return string(f)
}

// MyField2 is an example type with encoding.TextUnmarshaler implementation.
type MyField2 string

func (f *MyField2) UnmarshalText(p []byte) error {
	if len(p) == 0 {
		return fmt.Errorf("field value can't be empty")
	}
	*f = MyField2("my field is: " + string(p))
	return nil
}

func main() {
	type config struct {
		Default string   `env:"ONE"`
		Custom1 MyField1 `env:"TWO"`
		Custom2 MyField2 `env:"THREE"`
	}

	var cfg config

	os.Setenv("ONE", "test1")
	os.Setenv("TWO", "test2")
	os.Setenv("THREE", "test3")

	cleanenv.ReadEnv(&cfg)
	fmt.Printf("%+v\n", cfg)
}

Output:

{Default:test1 Custom1:my field is: test2 Custom2:my field is: test3}

Example (Updater)

Example_updater uses a type with a custom updater

package main

import (
	"fmt"
	"os"

	"github.com/ilyakaznacheev/cleanenv"
)

// ConfigUpdate is a type with a custom updater
type ConfigUpdate struct {
	Default string `env:"DEFAULT"`
	Custom  string
}

func (c *ConfigUpdate) Update() error {
	c.Custom = "custom"
	return nil
}

func main() {
	var cfg ConfigUpdate

	os.Setenv("DEFAULT", "default")

	cleanenv.ReadEnv(&cfg)
	fmt.Printf("%+v\n", cfg)
}

Output:

{Default:default Custom:custom}

Index

• Constants
• func FUsage(w io.Writer, cfg interface{}, headerText *string, usageFuncs ...func()) func()
• func GetDescription(cfg interface{}, headerText *string) (string, error)
• func ParseJSON(r io.Reader, str interface{}) error
• func ParseTOML(r io.Reader, str interface{}) error
• func ParseYAML(r io.Reader, str interface{}) error
• func ReadConfig(path string, cfg interface{}) error
• func ReadEnv(cfg interface{}) error
• func UpdateEnv(cfg interface{}) error
• func Usage(cfg interface{}, headerText *string, usageFuncs ...func()) func()
• type Setter
• type Updater

Examples

• Package (Setter)
• Package (Updater)
• GetDescription
• GetDescription (CustomHeaderText)
• GetDescription (Defaults)
• GetDescription (VariableList)
• ReadEnv
• UpdateEnv
• Usage

Constants

const (
	// TagEnv name of the environment variable or a list of names
	TagEnv = "env"

	// TagEnvLayout value parsing layout (for types like time.Time)
	TagEnvLayout = "env-layout"

	// TagEnvDefault default value
	TagEnvDefault = "env-default"

	// TagEnvSeparator custom list and map separator
	TagEnvSeparator = "env-separator"

	// TagEnvDescription environment variable description
	TagEnvDescription = "env-description"

	// TagEnvUpd flag to mark a field as updatable
	TagEnvUpd = "env-upd"

	// TagEnvRequired flag to mark a field as required
	TagEnvRequired = "env-required"

	// TagEnvPrefix аlag to specify prefix for structure fields
	TagEnvPrefix = "env-prefix"
)

Supported tags

const (
	// DefaultSeparator is a default list and map separator character
	DefaultSeparator = ","
)

Functions

func FUsage

func FUsage(w io.Writer, cfg interface{}, headerText *string, usageFuncs ...func()) func()

FUsage prints configuration help into the custom output. Other usage instructions can be wrapped in and executed before this usage function

func GetDescription

func GetDescription(cfg interface{}, headerText *string) (string, error)

GetDescription returns a description of environment variables. You can provide a custom header text.

Example

ExampleGetDescription builds a description text from structure tags

package main

import (
	"fmt"

	"github.com/ilyakaznacheev/cleanenv"
)

func main() {
	type config struct {
		One   int64   `env:"ONE" env-description:"first parameter"`
		Two   float64 `env:"TWO" env-description:"second parameter"`
		Three string  `env:"THREE" env-description:"third parameter"`
	}

	var cfg config

	text, err := cleanenv.GetDescription(&cfg, nil)
	if err != nil {
		panic(err)
	}

	fmt.Println(text)
}

Output:

Environment variables:
  ONE int64
    	first parameter
  THREE string
    	third parameter
  TWO float64
    	second parameter

Example (CustomHeaderText)

ExampleGetDescription_customHeaderText builds a description text from structure tags with custom header string

package main

import (
	"fmt"

	"github.com/ilyakaznacheev/cleanenv"
)

func main() {
	type config struct {
		One   int64   `env:"ONE" env-description:"first parameter"`
		Two   float64 `env:"TWO" env-description:"second parameter"`
		Three string  `env:"THREE" env-description:"third parameter"`
	}

	var cfg config

	header := "Custom header text:"

	text, err := cleanenv.GetDescription(&cfg, &header)
	if err != nil {
		panic(err)
	}

	fmt.Println(text)
}

Output:

Custom header text:
  ONE int64
    	first parameter
  THREE string
    	third parameter
  TWO float64
    	second parameter

Example (Defaults)

ExampleGetDescription_defaults builds a description text from structure tags with description of default values

package main

import (
	"fmt"

	"github.com/ilyakaznacheev/cleanenv"
)

func main() {
	type config struct {
		One   int64   `env:"ONE" env-description:"first parameter" env-default:"1"`
		Two   float64 `env:"TWO" env-description:"second parameter" env-default:"2.2"`
		Three string  `env:"THREE" env-description:"third parameter" env-default:"test"`
	}

	var cfg config

	text, err := cleanenv.GetDescription(&cfg, nil)
	if err != nil {
		panic(err)
	}

	fmt.Println(text)
}

Output:

Environment variables:
  ONE int64
    	first parameter (default "1")
  THREE string
    	third parameter (default "test")
  TWO float64
    	second parameter (default "2.2")

Example (VariableList)

ExampleGetDescription_variableList builds a description text from structure tags with description of alternative variables

package main

import (
	"fmt"

	"github.com/ilyakaznacheev/cleanenv"
)

func main() {
	type config struct {
		FirstVar int64 `env:"ONE,TWO,THREE" env-description:"first found parameter"`
	}

	var cfg config

	text, err := cleanenv.GetDescription(&cfg, nil)
	if err != nil {
		panic(err)
	}

	fmt.Println(text)
}

Output:

Environment variables:
  ONE int64
    	first found parameter
  THREE int64 (alternative to ONE)
    	first found parameter
  TWO int64 (alternative to ONE)
    	first found parameter

func ParseJSON

func ParseJSON(r io.Reader, str interface{}) error

ParseJSON parses JSON from reader to data structure

func ParseTOML

func ParseTOML(r io.Reader, str interface{}) error

ParseTOML parses TOML from reader to data structure

func ParseYAML

func ParseYAML(r io.Reader, str interface{}) error

ParseYAML parses YAML from reader to data structure

func ReadConfig

func ReadConfig(path string, cfg interface{}) error

ReadConfig reads configuration file and parses it depending on tags in structure provided. Then it reads and parses

Example:

type ConfigDatabase struct {
	Port     string `yaml:"port" env:"PORT" env-default:"5432"`
	Host     string `yaml:"host" env:"HOST" env-default:"localhost"`
	Name     string `yaml:"name" env:"NAME" env-default:"postgres"`
	User     string `yaml:"user" env:"USER" env-default:"user"`
	Password string `yaml:"password" env:"PASSWORD"`
}

var cfg ConfigDatabase

err := cleanenv.ReadConfig("config.yml", &cfg)
if err != nil {
    ...
}

func ReadEnv

func ReadEnv(cfg interface{}) error

ReadEnv reads environment variables into the structure.

Example

ExampleReadEnv reads environment variables or default values into the structure

package main

import (
	"fmt"
	"os"

	"github.com/ilyakaznacheev/cleanenv"
)

func main() {
	type config struct {
		Port     string `env:"PORT" env-default:"5432"`
		Host     string `env:"HOST" env-default:"localhost"`
		Name     string `env:"NAME" env-default:"postgres"`
		User     string `env:"USER" env-default:"user"`
		Password string `env:"PASSWORD"`
	}

	var cfg config

	os.Setenv("PORT", "5050")
	os.Setenv("NAME", "redis")
	os.Setenv("USER", "tester")
	os.Setenv("PASSWORD", "*****")

	cleanenv.ReadEnv(&cfg)
	fmt.Printf("%+v\n", cfg)

}

Output:

{Port:5050 Host:localhost Name:redis User:tester Password:*****}

func UpdateEnv

func UpdateEnv(cfg interface{}) error

UpdateEnv rereads (updates) environment variables in the structure.

Example

ExampleUpdateEnv updates variables in the configuration structure. Only variables with `env-upd:""` tag will be updated

package main

import (
	"fmt"
	"os"

	"github.com/ilyakaznacheev/cleanenv"
)

func main() {
	type config struct {
		One int64 `env:"ONE"`
		Two int64 `env:"TWO" env-upd:""`
	}

	var cfg config

	// set environment variables
	os.Setenv("ONE", "1")
	os.Setenv("TWO", "2")

	// read environment variables into the structure
	cleanenv.ReadEnv(&cfg)
	fmt.Printf("%+v\n", cfg)

	// update environment variables
	os.Setenv("ONE", "11")
	os.Setenv("TWO", "22")

	// update only updatable environment variables in the structure
	cleanenv.UpdateEnv(&cfg)
	fmt.Printf("%+v\n", cfg)

}

Output:

{One:1 Two:2}
{One:1 Two:22}

func Usage

func Usage(cfg interface{}, headerText *string, usageFuncs ...func()) func()

Usage returns a configuration usage help. Other usage instructions can be wrapped in and executed before this usage function. The default output is STDERR.

Example

package main

import (
	"flag"
	"os"

	"github.com/ilyakaznacheev/cleanenv"
)

func main() {
	os.Stderr = os.Stdout //replace STDERR with STDOUT for test

	type config struct {
		One   int64   `env:"ONE" env-description:"first parameter"`
		Two   float64 `env:"TWO" env-description:"second parameter"`
		Three string  `env:"THREE" env-description:"third parameter"`
	}

	// setup flags
	fset := flag.NewFlagSet("Example", flag.ContinueOnError)

	fset.String("p", "8080", "service port")
	fset.String("h", "localhost", "service host")

	var cfg config
	customHeader := "My sweet variables:"

	// get config usage with wrapped flag usage and custom header string
	u := cleanenv.Usage(&cfg, &customHeader, fset.Usage)

	// print usage to STDERR
	u()

}

Output:

Usage of Example:
  -h string
    	service host (default "localhost")
  -p string
    	service port (default "8080")

My sweet variables:
  ONE int64
    	first parameter
  THREE string
    	third parameter
  TWO float64
    	second parameter

Types

type Setter

type Setter interface {
	SetValue(string) error
}

Setter is an interface for a custom value setter.

To implement a custom value setter you need to add a SetValue function to your type that will receive a string raw value:

type MyField string

func (f *MyField) SetValue(s string) error {
	if s == "" {
		return fmt.Errorf("field value can't be empty")
	}
	*f = MyField("my field is: " + s)
	return nil
}

type Updater

type Updater interface {
	Update() error
}

Updater gives an ability to implement custom update function for a field or a whole structure