config

Documentation

Overview

Package config provides supports to a fully customizable layered configuration stack.

Introduction

Configuration in go lives in many ways, with benefit and drawbacks for each. Flags, for example, offer self-explanatory help messages, but it can be cumbersome if the size of non-default entries is large. Environmental variables, on the other hand, are recommended by twelve-factor app methodology for its language neutrality and secrecy, but it lacks type safety and configuration hierarchy.

Package config doesn't hold strong opinion on how you should structure your configurations. Rather, it allows you to define a configuration stack. For instance, You can put flags at first, envs at second, and configuration files at the third place. You are free to adjust the order in any other way or left out the layer you don't need.

Package config also supports hot reload. If the desired signal triggers, the whole configuration stack will be reloaded in the same sequence you bootstrap them. Thus, a third place configuration file will not overwrite your flags and envs in first and second place in the reload.

Usage

See the example for how to create a typical configuration stack. After the configuration is created, you can access it with the interface in contract.ConfigAccessor.

type ConfigAccessor interface {
  String(string) string
  Int(string) int
  Strings(string) []string
  Bool(string) bool
  Get(string) interface{}
  Float64(string) float64
  Unmarshal(path string, o interface{}) error
}

Package config uses koanf (https://github.com/knadh/koanf) to achieve many of its features. The configuration stack can be build with a rich set of already available provider and parsers in koanf. See https://github.com/knadh/koanf/blob/master/README.md for more info.

Integrate

Package config is part of the core. When using package core, the config is bootstrapped in the initialization phase. Package core's constructor inherits some of the options for configuration stack. See package core for more info.

A command is provided to export the default configuration:

go run main.go config init -o ./config/config.yaml

Best Practice

In general you should not pass contract.ConfigAccessor or config.KoanfAdapter to your services. You should only pass unmarshalled strings and structs that matters. You don't want your service unnecessarily depend on package config.

The only exception is when you need configuration hot reload. In this case, you have to pass the contract.ConfigAccessor to your service, and access the config repeatedly in each request/iteration of you service.

Future scope

Remote configuration store such as zookeeper, consul and ETCD can be adopted in this package in the same manner as the file provider. To avoid dependency bloat, they might live in their own subpackage.

Example (ConfigurationStack)

package main

import (
	"flag"
	"fmt"

	"github.com/DoNewsCode/core/config"
	"github.com/knadh/koanf/providers/basicflag"
	"github.com/knadh/koanf/providers/confmap"
)

func main() {
	fs := flag.NewFlagSet("config", flag.ContinueOnError)
	fs.String("foo", "", "foo value")
	fs.Parse([]string{"-foo", "bar"})
	conf, _ := config.NewConfig(
		config.WithProviderLayer(
			basicflag.Provider(fs, "."), nil,
		),
		config.WithProviderLayer(
			confmap.Provider(map[string]interface{}{"foo": "baz"}, "."), nil,
		),
	)
	// We have two layers of configuration, the first one from flags and the second one from a map.
	// Both of them defined "foo". The first one should take precedence.
	fmt.Println(conf.String("foo"))
}

Output:

bar

Example (Minimum)

package main

import (
	"fmt"

	"github.com/DoNewsCode/core/config"
	"github.com/knadh/koanf/parsers/json"
	"github.com/knadh/koanf/providers/rawbytes"
)

func main() {
	conf, _ := config.NewConfig(
		config.WithProviderLayer(
			rawbytes.Provider([]byte(`{"foo": "bar"}`)), json.Parser(),
		),
	)
	fmt.Println(conf.String("foo"))
}

Output:

bar

Index

• func WithAccessor(unmarshaler contract.ConfigUnmarshaler) contract.ConfigAccessor
• type AppName
  • func NewAppNameFromConf(conf contract.ConfigAccessor) AppName
  • func (a AppName) String() string
• type CodecParser
  • func (c CodecParser) Marshal(m map[string]interface{}) ([]byte, error)
  • func (c CodecParser) Unmarshal(bytes []byte) (map[string]interface{}, error)
• type ConfigIn
• type Duration
  • func (d Duration) IsZero() bool
  • func (d Duration) MarshalJSON() ([]byte, error)
  • func (d Duration) MarshalText() ([]byte, error)
  • func (d Duration) MarshalYAML() (interface{}, error)
  • func (d *Duration) UnmarshalJSON(b []byte) error
  • func (d *Duration) UnmarshalText(text []byte) error
  • func (d *Duration) UnmarshalYAML(value *yaml.Node) error
• type Env
  • func NewEnv(env string) Env
  • func NewEnvFromConf(conf contract.ConfigUnmarshaler) Env
  • func (e Env) IsDevelopment() bool
  • func (e Env) IsLocal() bool
  • func (e Env) IsProduction() bool
  • func (e Env) IsStaging() bool
  • func (e Env) IsTesting() bool
  • func (e Env) String() string
• type ExportedConfig
• type KoanfAdapter
  • func NewConfig(options ...Option) (*KoanfAdapter, error)
  • func (k *KoanfAdapter) Bool(s string) bool
  • func (k *KoanfAdapter) Duration(s string) time.Duration
  • func (k *KoanfAdapter) Float64(s string) float64
  • func (k *KoanfAdapter) Get(s string) interface{}
  • func (k *KoanfAdapter) Int(s string) int
  • func (k *KoanfAdapter) Reload() error
  • func (k *KoanfAdapter) Route(s string) contract.ConfigAccessor
  • func (k *KoanfAdapter) String(s string) string
  • func (k *KoanfAdapter) Strings(s string) []string
  • func (k *KoanfAdapter) Unmarshal(path string, o interface{}) error
  • func (k *KoanfAdapter) Watch(ctx context.Context) error
• type MapAdapter
  • func (m MapAdapter) Route(s string) contract.ConfigUnmarshaler
  • func (m MapAdapter) Unmarshal(path string, o interface{}) (err error)
• type Module
  • func New(p ConfigIn) (Module, error)
  • func (m Module) ProvideCommand(command *cobra.Command)
  • func (m Module) ProvideRunGroup(group *run.Group)
• type Option
  • func WithDelimiter(delimiter string) Option
  • func WithDispatcher(dispatcher contract.Dispatcher) Option
  • func WithProviderLayer(provider koanf.Provider, parser koanf.Parser) Option
  • func WithValidators(validators ...Validator) Option
  • func WithWatcher(watcher contract.ConfigWatcher) Option
• type ProviderSet
• type Validator

Examples

• Package (ConfigurationStack)
• Package (Minimum)

Functions

func WithAccessor

func WithAccessor(unmarshaler contract.ConfigUnmarshaler) contract.ConfigAccessor

WithAccessor upgrade contract.ConfigUnmarshaler to contract.ConfigAccessor by wrapping the original unmarshaler.

Types

type AppName

type AppName string

AppName represents the name the application. It is primarily used as a symbol for dependency injection.

func NewAppNameFromConf

func NewAppNameFromConf(conf contract.ConfigAccessor) AppName

NewAppNameFromConf reads the name of application from configuration's "name" entry.

func (AppName) String

func (a AppName) String() string

String gives the string form of AppName.

type CodecParser

type CodecParser struct {
	Codec contract.Codec
}

CodecParser implements the Parser interface. It converts any contract.Codec to a valid config parser.

func (CodecParser) Marshal

func (c CodecParser) Marshal(m map[string]interface{}) ([]byte, error)

Marshal converts the map to bytes.

func (CodecParser) Unmarshal

func (c CodecParser) Unmarshal(bytes []byte) (map[string]interface{}, error)

Unmarshal converts the bytes to map

type ConfigIn

type ConfigIn struct {
	di.In

	Conf            contract.ConfigAccessor
	Dispatcher      contract.Dispatcher `optional:"true"`
	ExportedConfigs []ExportedConfig    `group:"config"`
}

ConfigIn is the injection parameter for config.New.

type Duration

type Duration struct {
	time.Duration
}

Duration is a type that describe a time duration. It is suitable for use in configurations as it implements a variety of serialization methods.

func (Duration) IsZero

func (d Duration) IsZero() bool

IsZero returns true if the Duration is the zero value.

func (Duration) MarshalJSON

func (d Duration) MarshalJSON() ([]byte, error)

MarshalJSON implements json.Marshaler

func (Duration) MarshalText

func (d Duration) MarshalText() ([]byte, error)

MarshalText implements encoding.TextMarshaler

func (Duration) MarshalYAML

func (d Duration) MarshalYAML() (interface{}, error)

MarshalYAML implements yaml.Marshaler

func (*Duration) UnmarshalJSON

func (d *Duration) UnmarshalJSON(b []byte) error

UnmarshalJSON implements json.Unmarshaler

func (*Duration) UnmarshalText

func (d *Duration) UnmarshalText(text []byte) error

UnmarshalText implements encoding.TextUnmarshaler

func (*Duration) UnmarshalYAML

func (d *Duration) UnmarshalYAML(value *yaml.Node) error

UnmarshalYAML implements yaml.Unmarshaler

type Env

type Env string

Env is the environment of the application. It is primarily used as dependency injection symbol

const (
	// local
	EnvLocal Env = "local"
	// testing
	EnvTesting Env = "testing"
	// development
	EnvDevelopment Env = "development"
	// staging
	EnvStaging Env = "staging"
	// production
	EnvProduction Env = "production"
	// unknown
	EnvUnknown Env = "unknown"
)

global static variables for Env

func NewEnv

func NewEnv(env string) Env

NewEnv takes in environment string and returns a Env type. It does some "best-effort" normalization internally. For example, prod, PROD, production and PRODUCTION produces the same type. It is recommended to use one of "production", "staging", "development", "local", or "testing" as output to avoid unexpected outcome.

func NewEnvFromConf

func NewEnvFromConf(conf contract.ConfigUnmarshaler) Env

NewEnvFromConf reads the name of application from configuration's "env" entry.

func (Env) IsDevelopment

func (e Env) IsDevelopment() bool

IsDevelopment returns true if the environment is development

func (Env) IsLocal

func (e Env) IsLocal() bool

IsLocal returns true if the environment is local

func (Env) IsProduction

func (e Env) IsProduction() bool

IsProduction returns true if the environment is production

func (Env) IsStaging

func (e Env) IsStaging() bool

IsStaging returns true if the environment is staging

func (Env) IsTesting

func (e Env) IsTesting() bool

IsTesting returns true if the environment is testing

func (Env) String

func (e Env) String() string

String returns the string form of the environment. This is a lowercase full word, such as production.

type ExportedConfig

type ExportedConfig struct {
	Owner    string
	Data     map[string]interface{}
	Comment  string
	Validate Validator
}

ExportedConfig is a struct that outlines a set of configuration. Each module is supposed to emit ExportedConfig into DI, and Package config should collect them.

type KoanfAdapter

type KoanfAdapter struct {
	K *koanf.Koanf
	// contains filtered or unexported fields
}

KoanfAdapter is a implementation of contract.Config based on Koanf (https://github.com/knadh/koanf).

func NewConfig

func NewConfig(options ...Option) (*KoanfAdapter, error)

NewConfig creates a new *KoanfAdapter.

func (*KoanfAdapter) Bool

func (k *KoanfAdapter) Bool(s string) bool

Bool returns the bool value of a given key path or false if the path does not exist or if the value is not a valid bool representation. Accepted string representations of bool are the ones supported by strconv.ParseBool.

func (*KoanfAdapter) Duration

func (k *KoanfAdapter) Duration(s string) time.Duration

Duration returns the time.Duration value of a given key path or its zero value if the path does not exist or if the value is not a valid float64.

func (*KoanfAdapter) Float64

func (k *KoanfAdapter) Float64(s string) float64

Float64 returns the float64 value of a given key path or 0 if the path does not exist or if the value is not a valid float64.

func (*KoanfAdapter) Get

func (k *KoanfAdapter) Get(s string) interface{}

Get returns the raw, uncast interface{} value of a given key path in the config map. If the key path does not exist, nil is returned.

func (*KoanfAdapter) Int

func (k *KoanfAdapter) Int(s string) int

Int returns the int value of a given key path or 0 if the path does not exist or if the value is not a valid int.

func (*KoanfAdapter) Reload

func (k *KoanfAdapter) Reload() error

Reload reloads the whole configuration stack. It reloads layer by layer, so if an error occurred, Reload will return early and abort the rest of the reloading.

func (*KoanfAdapter) Route

func (k *KoanfAdapter) Route(s string) contract.ConfigAccessor

Route cuts the config map at a given key path into a sub map and returns a new contract.ConfigAccessor instance with the cut config map loaded. For instance, if the loaded config has a path that looks like parent.child.sub.a.b, `Route("parent.child")` returns a new contract.ConfigAccessor instance with the config map `sub.a.b` where everything above `parent.child` are cut out.

func (*KoanfAdapter) String

func (k *KoanfAdapter) String(s string) string

String returns the string value of a given key path or "" if the path does not exist or if the value is not a valid string

func (*KoanfAdapter) Strings

func (k *KoanfAdapter) Strings(s string) []string

Strings returns the []string slice value of a given key path or an empty []string slice if the path does not exist or if the value is not a valid string slice.

func (*KoanfAdapter) Unmarshal

func (k *KoanfAdapter) Unmarshal(path string, o interface{}) error

Unmarshal unmarshals a given key path into the given struct using the mapstructure lib. If no path is specified, the whole map is unmarshalled. `koanf` is the struct field tag used to match field names.

func (*KoanfAdapter) Watch

func (k *KoanfAdapter) Watch(ctx context.Context) error

Watch uses the internal watcher to watch the configuration reload signals. This function should be registered in the run group. If the watcher is nil, this call will block until context expired.

type MapAdapter

type MapAdapter map[string]interface{}

MapAdapter implements ConfigUnmarshaler and ConfigRouter. It is primarily used for testing

func (MapAdapter) Route

func (m MapAdapter) Route(s string) contract.ConfigUnmarshaler

Route implements contract.ConfigRouter

func (MapAdapter) Unmarshal

func (m MapAdapter) Unmarshal(path string, o interface{}) (err error)

type Module

type Module struct {
	// contains filtered or unexported fields
}

Module is the configuration module that bundles the reload watcher and exportConfig commands. This module triggers ReloadedEvent on configuration change.

func New

func New(p ConfigIn) (Module, error)

New creates a new config module. It contains the init command.

func (Module) ProvideCommand

func (m Module) ProvideCommand(command *cobra.Command)

ProvideCommand provides the config related command.

func (Module) ProvideRunGroup

func (m Module) ProvideRunGroup(group *run.Group)

ProvideRunGroup runs the configuration watcher.

type Option

type Option func(option *KoanfAdapter)

Option is the functional option type for KoanfAdapter

func WithDelimiter

func WithDelimiter(delimiter string) Option

WithDelimiter changes the default delimiter of Koanf. See Koanf's doc to learn more about delimiters.

func WithDispatcher

func WithDispatcher(dispatcher contract.Dispatcher) Option

WithDispatcher changes the default dispatcher of Koanf.

func WithProviderLayer

func WithProviderLayer(provider koanf.Provider, parser koanf.Parser) Option

WithProviderLayer is an option for *KoanfAdapter that adds a layer to the bottom of the configuration stack. This option can be used multiple times, thus forming the whole stack. The layer on top has higher priority.

func WithValidators

func WithValidators(validators ...Validator) Option

WithValidators changes the validators of Koanf.

func WithWatcher

func WithWatcher(watcher contract.ConfigWatcher) Option

WithWatcher is an option for *KoanfAdapter that adds a config watcher. The watcher should notify the configurations whenever a reload event is triggered.

type ProviderSet

type ProviderSet struct {
	Parser   koanf.Parser
	Provider koanf.Provider
}

ProviderSet is a configuration layer formed by a parser and a provider.

type Validator

type Validator func(data map[string]interface{}) error

Validator is a method to verify if config is valid. If it is not valid, the returned error should contain a human readable description of why.