dotconfig

README

Dotconfig

This package aims to simplify configuration from environment variables. In local development, we can supply a .env file with key/value pairs. When deployed, values come from a secret manager. This is similar to joho/godotenv but the aim here is to not only read the .env file but use reflection to produce a config struct. We also support optional/required fields in the struct and default values.

Usage

Create a .env file in your current working directory with the following contents:

MAX_BYTES_PER_REQUEST='1024'
# Double quotes are fine
API_VERSION="1.19"
# All of these are valie for booleans:
# 1, t, T, TRUE, true, True, 0, f, F, FALSE, false, False
IS_DEV='1'
# Raw values with no quotes are also fine
STRIPE_SECRET=sk_test_insertkeyhere
# Right now supporting newlines via "\n" in strings:
WELCOME_MESSAGE='Hello,\nWelcome to the app!\n-The App Dev Team'

You can read from this file and initialize your config with values with the following code:

package main

import (
	"fmt"

	"github.com/DeanPDX/dotconfig"
)

// Our AppConfig with env struct tags:
type AppConfig struct {
	MaxBytesPerRequest int     `env:"MAX_BYTES_PER_REQUEST" default:"2048"` // Defaults to 2048
	APIVersion         float64 `env:"API_VERSION,required"` // Required to be present and not empty string
	IsDev              bool    `env:"IS_DEV,optional"` // Optional: defaults to zero value
	StripeSecret       string  `env:"STRIPE_SECRET"`
	WelcomeMessage     string  `env:"WELCOME_MESSAGE"`
}

func Main() {
	config, err := dotconfig.FromFileName[AppConfig](".env")
	if err != nil {
		fmt.Printf("Error: %v.", err)
	}
	// Config is ready to use. Don't print to console in a real 
	// app. But for the purposes of testing:
	fmt.Println(config)
}

So for local dev we can use this .env file. But when you deploy your app, you set these values from environment variables / secret managers. Your app that consumes this config struct doesn't have to concern itself with where the values came from.

If your key value pairs are coming from a source other than a file, or you want to control file IO yourself, you can call FromReader instead and pass in a io.Reader. There is a runnable example of that in the godoc.

Error Handling and Options

By default, file IO errors in dotconfig.FromFileName won't produce an error. This is because when you are running in the cloud with a secret manager, not finding a .env file is the happy path. If you want to return errors from os.Open you can do so with an option:

config, err := dotconfig.FromFileName[AppConfig](".env", dotconfig.ReturnFileErrors)

By default, if your struct contains fields that don't have an env:"MY_ENV" tag, we assume you want us to ignore those fields. If you want missing env tags to produce errors, use the dotconfig.EnforceStructTags option:

config, err := dotconfig.FromFileName[AppConfig](".env", dotconfig.EnforceStructTags)

dotconfig.FromFileName and dotconfig.FromReader both return multiple wrapped errors. If you want to print all errors to the console you can do that:

type AppConfig struct {
	ForgotToAddStructTag string
	UnsupportedType   	 complex64 `env:"UNSUPPORTED_TYPE"`
}
config, err := dotconfig.FromFileName[AppConfig](".env", dotconfig.EnforceStructTags)
if err != nil {
	fmt.Printf("Error %v.", err)
}
// Output:
// Error: multiple errors:
//  - missing struct tag on field: ForgotToAddStructTag
//  - unsupported field type: complex64

Sometimes you want more fine-grained control of error handling (because certain states you can recover from). If you want to handle each error type, you can use dotconfig.Errors in conjunction with errors.Unwrap and errors.Is. Here's an example where each error type is being handled:

type MyConfig struct {}
_, err := dotconfig.FromFileName[MyConfig](".env", dotconfig.EnforceStructTags)
if err != nil {
	// Get error slice from err
	errs := dotconfig.Errors(err)
	for _, err := range errs {
		// Handle various error types however you want
		switch {
		case errors.Is(errors.Unwrap(err), dotconfig.ErrMissingEnvVar):
			// Handle missing environment variable
		case errors.Is(errors.Unwrap(err), dotconfig.ErrMissingStructTag):
			// Handle missing struct tag
		case errors.Is(errors.Unwrap(err), dotconfig.ErrUnsupportedFieldType):
			// Handle unsupported field type
		case errors.Is(errors.Unwrap(err), dotconfig.ErrMissingRequiredField):
			// Handle missing required field
		}
	}
}

Contributing

Contributions are always welcome. Have a new idea or find a bug? Submit a pull request or create an issue!

This package is being used in production and any future updates should maintain backwards compatibility. This is why we have options; to allow us to introduce optional new features while maintaining backwards compatibility.

Documentation

Overview

Package dotconfig implements loading/parsing of .env files into configs structs.

Example (DefaultValues)

package main

import (
	"fmt"
	"strings"

	"github.com/DeanPDX/dotconfig"
)

type ConfigWithDefaults struct {
	MaxBytesPerRequest int     `env:"MAX_BYTES" default:"2048"`
	IsDev              bool    `env:"DEVELOPMENT,optional"`
	WelcomeMessage     string  `env:"APP_HELLO" default:"Hey!"`
	AppVersion         float64 `env:"APP_VERSION" default:"1.0"`
}

func main() {
	r := strings.NewReader(`APP_VERSION=2.38`)
	conf, err := dotconfig.FromReader[ConfigWithDefaults](r, dotconfig.EnforceStructTags)
	if err != nil {
		fmt.Printf("Didn't expect error. Got %v.", err)
	}
	fmt.Println("App config loaded")
	fmt.Println("Max bytes:", conf.MaxBytesPerRequest)   // 2048 from default tag
	fmt.Println("Is dev?", conf.IsDev)                   // False because optional so zero value
	fmt.Println("Welcome message:", conf.WelcomeMessage) // "Hey!" from default tag
	fmt.Println("App version:", conf.AppVersion)         // 2.38 because ENV value overrides default
}

Output:
App config loaded
Max bytes: 2048
Is dev? false
Welcome message: Hey!
App version: 2.38

Index

• Variables
• func Errors(err error) []error
• func FromFileName[T any](name string, opts ...DecodeOption) (T, error)
• func FromReader[T any](r io.Reader, opts ...DecodeOption) (T, error)
• type DecodeOption

Examples

• Package (DefaultValues)
• Errors
• FromReader

Variables

var (
	ErrConfigMustBeStruct   = errors.New("config must be struct")
	ErrMissingStructTag     = errors.New("missing struct tag on field")
	ErrMissingEnvVar        = errors.New("key not present in ENV")
	ErrMissingRequiredField = errors.New("field must have non-zero value")
	ErrUnsupportedFieldType = errors.New("unsupported field type")
)

Functions

func Errors

func Errors(err error) []error

Errors returns a slice containing zero or more errors that the supplied error is composed of. If the error is nil, a nil slice is returned.

Example usage:

type myconfig struct{/*...*/}
conf, err := dotconfig.FromFileName[myconfig](".env")
// If we want to extract errors as a slice:
errors := dotconfig.Errors(err)

Example

package main

import (
	"errors"
	"fmt"
	"strings"

	"github.com/DeanPDX/dotconfig"
)

type ConfigWithErrors struct {
	StripeSecret   string     `env:"SHOULD_BE_MISSING"`
	Complex        complex128 `env:"COMPLEX"`
	WelcomeMessage string
	RequiredField  string `env:"REQUIRED_FIELD,required"`
}

const exampleErrorsEnv = `
COMPLEX=asdf
REQUIRED_FIELD="" # Will cause error because zero-value`

func main() {
	r := strings.NewReader(exampleErrorsEnv)
	_, err := dotconfig.FromReader[ConfigWithErrors](r, dotconfig.EnforceStructTags)
	if err != nil {
		// Get error slice from err
		errs := dotconfig.Errors(err)
		for _, err := range errs {
			// Handle various error types however you want
			switch {
			case errors.Is(errors.Unwrap(err), dotconfig.ErrMissingEnvVar):
				// Handle missing environment variable
				fmt.Printf("Missing env variable: %v\n", err)
			case errors.Is(errors.Unwrap(err), dotconfig.ErrMissingStructTag):
				// Handle missing struct tag
				fmt.Printf("Missing struct tag: %v\n", err)
			case errors.Is(errors.Unwrap(err), dotconfig.ErrUnsupportedFieldType):
				// Handle unsupported field
				fmt.Printf("Unsupported type: %v\n", err)
			case errors.Is(errors.Unwrap(err), dotconfig.ErrMissingRequiredField):
				// Handle required field
				fmt.Printf("Required field can't be zero value: %v\n", err)
			}
		}
	}
}

Output:
Missing env variable: key not present in ENV: SHOULD_BE_MISSING
Unsupported type: unsupported field type: complex128
Missing struct tag: missing struct tag on field: WelcomeMessage
Required field can't be zero value: field must have non-zero value: REQUIRED_FIELD

func FromFileName

func FromFileName[T any](name string, opts ...DecodeOption) (T, error)

FromFileName will call os.Open on the supplied name and will then call FromReader. By default this will ignore file access errors. This is usually desired behavior because in live environments, config will come from a secret manager and os.Open will fail. If you *want* to return file errors, use opts:

type myconfig struct{/*...*/}
conf, err := dotconfig.FromFileName[myconfig](".env", dotconfig.ReturnFileIOErrors)

See FromReader for supported types and expected file format. And if you want to control your own file access or read from something other than a file, you can call FromReader directly with an io.Reader.

func FromReader

func FromReader[T any](r io.Reader, opts ...DecodeOption) (T, error)

FromReader will read from r and call os.Setenv to set environment variables based on key value pairs in r.

Expected format for the pairs is:

KEY='value enclosed in single quotes'
# Comments are fine as are blank lines

# You also don't need single/double quotes at all if you prefer
DATA_SOURCE_NAME=postgres://username:password@localhost:5432/database_name
DOUBLE_QUOTES="sk_test_asDF!"
MULTI_LINE='line1\nline2\nline3'

Currently newlines are supported as "\n" in string values. In the future might look in to more advanced escaping, etc. but this suits our needs for the time being.

Example

package main

import (
	"fmt"
	"strings"

	"github.com/DeanPDX/dotconfig"
)

type AppConfig struct {
	MaxBytesPerRequest int     `env:"MAX_BYTES_PER_REQUEST"`
	APIVersion         float64 `env:"API_VERSION"`
	IsDev              bool    `env:"IS_DEV"`
	StripeSecret       string  `env:"STRIPE_SECRET"`
	WelcomeMessage     string  `env:"WELCOME_MESSAGE"`
}

const appConfigSample = `
MAX_BYTES_PER_REQUEST="1024"
API_VERSION=1.19
# All of these are valie for booleans:
# 1, t, T, TRUE, true, True, 0, f, F, FALSE, false, False
IS_DEV='1'
STRIPE_SECRET='sk_test_insertkeyhere'
# Right now supporting newlines via "\n" in strings:
WELCOME_MESSAGE='Hello,\nWelcome to the app!\n-The App Dev Team'`

func main() {
	config, err := dotconfig.FromReader[AppConfig](strings.NewReader(appConfigSample))
	if err != nil {
		fmt.Printf("Didn't expect error. Got %v.", err)
	}
	// Don't do this in the real world, as your config will
	// have secrets from a secret manager and you don't want
	// to print them to the console.
	fmt.Printf("App config loaded.\nMax Bytes: %v. Version: %v. Dev? %v. Stripe Secret: %v.\nWelcome Message:\n%v",
		config.MaxBytesPerRequest, config.APIVersion, config.IsDev, config.StripeSecret, config.WelcomeMessage)
}

Output:
App config loaded.
Max Bytes: 1024. Version: 1.19. Dev? true. Stripe Secret: sk_test_insertkeyhere.
Welcome Message:
Hello,
Welcome to the app!
-The App Dev Team

Types

type DecodeOption

type DecodeOption int

const (
	ReturnFileIOErrors  DecodeOption = iota // Return file IO errors
	EnforceStructTags                       // Make sure all fields in config struct have `env` struct tags
	AllowWhitespace                         // Allow leading/trailing whitespace in string values
	SkipNewlineDecoding                     // Don't turn "\n" into newlines
)