goenv

package module
Version: v0.0.0-...-ce47a9b Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Dec 12, 2017 License: Apache-2.0 Imports: 6 Imported by: 2

README

Golang Marshaler from Environment Variables

Build Status GoDoc Go Report Card codecov

Golang (1.5+) package for marshalling objects from environment variable values. There are many like packages. The one that inspires this API the most is the json package. The idea is that configuration objects are stored as environment variables, especially when running as a containerised application.

Example

The following is a full and well contrived application that showcases how to use the API to retrieve data from environments. The application

  • retrieves a list of things to say and wait time (via config)
  • for everything to be said, it prints a simple statement

Disclaimer: this is a silly toy application. Don't use this as an example of good application design. It isn't.

package main


import (
    "fmt"
    "github.com/evilwire/go-env"
    "time"
)


type SomeConfig struct {
        User string `env:"USER_NAME"`
        
        Application struct {
                WaitDuration time.Duration `env:"WAIT_DURATION"`
                
                // this is best obtained from flags, but you can equally
                // retrieve this from environment variables
                ThingsToSay []string `env:"THINGS_TO_SAY"`
        } `env:"APP_"`
}


// Sets up my application by reading the configs
func setup(env goenv.EnvReader) (*SomeConfig, error) {
        // create a Marshaler using our lovely default, which knows
        // how to marshal a set of things
        marshaller := goenv.DefaultEnvMarshaler{
                env,
        }
        
        // instantiate an empty config 
        config := SomeConfig{} 
        err := marshaller.Unmarshal(&config)
        if err != nil {
            return nil, err
        }
        
        return &config, nil
}


func main() {
        config, err := setup(goenv.NewOsEnvReader())
        if err != nil {
                panic(err)
        }
        
        for _, line := range config.Application.ThingsToSay {
                fmt.Printf("%s says: ", config.User)
                fmt.Println(line)
                time.Sleep(config.Application.WaitDuration)
        }
        
        fmt.Println("We're done!")
}

Compile your application, say silly-app, and run your application

USER_NAME='Michael Bluth' \
APP_WAIT_DURATION=2s \
APP_THINGS_TO_SAY='hiya,how are you,bye' /path/to/silly-app
Customising the UnmarshalEnv method

One of the cases that I encounter is using AWS KMS to manage secrets. For example, you may want to store KMS-encrypted credentials in a distributed version control source such GitHub, and passed into your application directly via environment variables.

We want to use the following function to decrypt KMS-encrypted secrets

package whatever

import (
        "github.com/aws/aws-sdk-go/service/kms"
        "github.com/aws/aws-sdk-go/service/kms/kmsiface"
        "encoding/base64"
)


// Uses KMS to decrypt an encrypted, base64 encoded secret (string) 
// by base64 decoding and KMS decrypting the bugger
func KMSDecrypt(secret string, kmsClient kmsiface.KMSAPI) (string, error) {
        b64Secret, err := base64.StdEncoding.DecodeString(secret)
        if err != nil {
                // handle
                return "", err
        }
        response, err := kmsClient.Decrypt(&kms.DecryptInput {
                CiphertextBlob: b64Secret,
                
                // additional context???
        })
        if err != nil {
            return "", err
        }
        
        return string(response.Plaintext), nil
}

Let's make a custom marshal method that ingests KMS-encrypted

package whatever

import (
    "github.com/evilwire/go-env"
    "github.com/aws/aws-sdk-go/service/kms"
    "github.com/aws/aws-sdk-go/aws/session"
    "github.com/aws/aws-sdk-go/aws"
)

type KMSEncryptedConfig struct {
        Username string `env:"USER"`
        Password string
        
        // and other fields that one might be able
        // to get from things
}


func (config *KMSEncryptedConfig) MarshalEnv(env goenv.EnvReader) error {
        tempConfig := struct {
                KMSEncryptedConfig
                KMSPassword string `env:"KMS_PASSWORD"`
        }{}
        
        marshaller := goenv.DefaultEnvMarshaler{ env }
        err := marshaller.Unmarshal(&tempConfig)
        if err != nil {
                return err
        }
        
        password, err := KMSDecrypt(tempConfig.KMSPassword, &kms.New(session.New(aws.Config{
            // configuration of some sort
        })))
        if err != nil {
                return err
        }
        
        config.Username = tempConfig.Username
        config.Password = password
        // more copying...
        
        return nil
}

Now you can write an application and accepts KMS passwords and not have to worry:

package main


import (
        "whatever"
        "github.com/evilwire/go-env"
)


type Config struct {
        DbCredentials *whatever.KMSEncryptedConfig `env:"DB_"`
        
        // other types of configs
}


func doStuff(config *Config) error {
        // do stuff
        
        return nil
}


func setup(env goenv.EnvReader) (*Config, error) {
        config := Config {}
        marshaller := goenv.DefaultEnvMarshaler{ env }
        
        err := marshaller.Unmarshal(&config)
        if err != nil {
                return nil, err
        }
        
        // does setuppy things...
        
        return config, nil
}


func main() {
        config, err := setup(goenv.NewOsEnvReader())
        if err != nil {
                panic(err)
        }
        
        // does stuff with that config
        panic(doStuff(config))
}

Now compile your application as, say app, and run the application as:

DB_KMS_PASSWORD=ABCD1234abcd1234 \
DB_USER=mbluth \
#... \
app

Documentation

Overview

Package goenv supports unmarshalling objects from environment variables by defining tags appended to fields. The idea here is that there are a lot of applications whose config objects are serialised from environment variable values.

Consider the following example

type CassandraConfig struct {
	Hosts 		[]string `env: "CASSANDRA_HOSTS"`
	Port  		int	 `env: "CASSANDRA_PORT"`
	Consistency	string	 `env: "CASSANDRA_CONSISTENCY"`
 }

func main() {
	// setting up the config
	unmarshaller := DefaultEnvMarshaler {
		Environment: NewOsEnvReader(),
	}
	config := CassandraConfig{}
	unmarshaller.Unmarshal(&config)

	// application logic
	// ...
 }

We believe that the above is pretty straightforward and has a similar flavor to the `encoding/json` library.

At this juncture, the unmarshalling is not thread-safe. Explicit synchronisation logic is needed to achieve atomicity in code.

Package goenv defines an object that parses values from strings.

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

This section is empty.

Types

type DefaultEnvMarshaler

type DefaultEnvMarshaler struct {
	Environment EnvReader
}

DefaultEnvMarshaler - An unmarshaller that uses the DefaultParser and a specific environment reader to unmarshal primitive and derived values.

func (*DefaultEnvMarshaler) Unmarshal

func (marshaler *DefaultEnvMarshaler) Unmarshal(i interface{}) error

Unmarshal - Unmarshals a given value from environment variables. It accepts a pointer to a given object, and either succeeds in unmarshalling the object or returns an error.

Usage:

 import "github.com/evilwire/go-env"

 type CassandraConfig struct {
	Hosts 		[]string `env: "CASSANDRA_HOSTS"`
	Port  		int	 `env: "CASSANDRA_PORT"`
	Consistency	string	 `env: "CASSANDRA_CONSISTENCY"`
 }

 func main() {
	// setting up the config
	unmarshaller := goenv.DefaultEnvMarshaler {
		Environment: goenv.NewOsEnvReader(),
	}
	config := CassandraConfig{}
	unmarshaller.Unmarshal(&config)

	// application logic
	// ...
 }

type DefaultParser

type DefaultParser struct{}

DefaultParser - A default way to parse a string into a specific primitive or pointer.

func (*DefaultParser) ParseType

func (marshaler *DefaultParser) ParseType(str string, t reflect.Type) (reflect.Value, error)

ParseType - Parses a string value for a specific type given by reflect.Type. For example, ParseType might accept str="2" and reflect.Type=reflect.Uint and parses the uint value of 2 returned as reflect.Value.

In this particular case, we parse all numeric types, pointers, strings, booleans, arrays and slices. The method handles Durations differently, though under the hood, the type is treated the same way as int64. In particular, we parse durations of the form `1m3s` and more generally, expects the string to be parse-able via ParseDuration.

If the object isn't one of the supported types, it throws an error.

func (*DefaultParser) Unmarshal

func (marshaler *DefaultParser) Unmarshal(val string, i interface{}) error

Unmarshal - Unmarshals a string into any one of the string-parseable types, which include (pointers of) numeric types, strings, booleans, arrays and slices. The method also handles Duration separately.

The method throws an error if the underlying interface is unsettable (see https://golang.org/pkg/reflect/#Value.CanSet), or if parsing for the value resulted in error

type EnvReader

type EnvReader interface {

	// look up the value for a particular env variable
	// returning false if the variable is not registered
	LookupEnv(string) (string, bool)

	// returns whether or not env variables are set in
	// the environment; returning a collection of env
	// variables that are missing
	HasKeys([]string) (bool, []string)
}

EnvReader is an interface for expressing the ability to look up values from the environment via environment variables (LookupEnv) and the ability to query the existence of many environment variables at once.

type EnvUnmarshaler

type EnvUnmarshaler interface {
	UnmarshalEnv(EnvReader) error
}

EnvUnmarshaler is an interface for any object that defines the UnmarshalEnv method, i.e. a method that accepts an EnvReader and can unmarshal from environment variable values from the EnvReader

type Marshaler

type Marshaler interface {
	Unmarshal(interface{}) error
}

Marshaler - An interface for any object that implements the Unmarshal method.

type OsEnvReader

type OsEnvReader struct {
	// contains filtered or unexported fields
}

OsEnvReader is an environment variable reader that implements that EnvReader interface by using the os.LookupEnv method.

func NewOsEnvReader

func NewOsEnvReader() *OsEnvReader

NewOsEnvReader creates a new instance of OsEnvReader

func (*OsEnvReader) HasKeys

func (env *OsEnvReader) HasKeys(keys []string) (bool, []string)

HasKeys - Returns whether or not a set of environment variables have corresponding values along with a list of environment variables that do not have values.

func (*OsEnvReader) LookupEnv

func (env *OsEnvReader) LookupEnv(key string) (string, bool)

LookupEnv - Lookup a certain environment variable by name. Returns the value of the environment variable if the variable exists and has an assigned value. Otherwise, returns an unspecific value, and the exists flag is set to false.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL