config

package module
v1.4.0 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Nov 20, 2019 License: MIT Imports: 11 Imported by: 229

README

🎣 config GoDoc Build Status Coverage Status

Convenient, injection-friendly YAML configuration.

Installation

go get -u go.uber.org/config

Note that config only supports the two most recent minor versions of Go.

Quick Start

// Model your application's configuration using a Go struct.
type cfg struct {
    Parameter string
}

// Two sources of configuration to merge.
base := strings.NewReader("module: {parameter: foo}")
override := strings.NewReader("module: {parameter: bar}")

// Merge the two sources into a Provider. Later sources are higher-priority.
// See the top-level package documentation for details on the merging logic.
provider, err := config.NewYAML(config.Source(base), config.Source(override))
if err != nil {
    panic(err) // handle error
}

var c cfg
if err := provider.Get("module").Populate(&c); err != nil {
  panic(err) // handle error
}

fmt.Printf("%+v\n", c)
// Output:
// {Parameter:bar}

Development Status: Stable

All APIs are finalized, and no breaking changes will be made in the 1.x series of releases. Users of semver-aware dependency management systems should pin config to ^1.

Released under the MIT License.

Documentation

Overview

Package config is an encoding-agnostic configuration abstraction. It supports merging multiple configuration files, expanding environment variables, and a variety of other small niceties. It currently supports YAML, but may be extended in the future to support more restrictive encodings like JSON or TOML.

Merging Configuration

It's often convenient to separate configuration into multiple files; for example, an application may want to first load some universally-applicable configuration and then merge in some environment-specific overrides. This package supports this pattern in a variety of ways, all of which use the same merge logic.

Simple types (numbers, strings, dates, and anything else YAML would consider a scalar) are merged by replacing lower-priority values with higher-priority overrides. For example, consider this merge of base.yaml and override.yaml: 

# base.yaml
some_key: foo

# override.yaml
some_key: bar

# merged result
some_key: bar

Slices, arrays, and anything else YAML would consider a sequence are also replaced. Again merging base.yaml and override.yaml: 

# base.yaml
some_key: [foo, bar]

# override.yaml
some_key: [baz, quux]

# merged output
some_key: [baz, quux]

Maps are recursively deep-merged, handling scalars and sequences as described above. Consider a merge between a more complex set of YAML files: 

  # base.yaml
  some_key:
    foo: bar
    foos: [1, 2]

  # override.yaml
  some_key:
    baz: quux
    foos: [3, 4]

 # merged output
	some_key:
	  foo: bar      # from base.yaml
	  baz: quux     # from override.yaml
	  foos: [3, 4]  # from override.yaml

In all cases, explicit nils (represented in YAML with a tilde) override any pre-existing configuration. For example, 

# base.yaml
foo: {bar: baz}

# override.yaml
foo: ~

# merged output
foo: ~

Strict Unmarshalling

By default, the NewYAML constructor enables gopkg.in/yaml.v2's strict unmarshalling mode. This prevents a variety of common programmer errors, especially when deep-merging loosely-typed YAML files. In strict mode, providers throw errors if keys are duplicated in the same configuration source, all keys aren't used when populating a struct, or a merge encounters incompatible data types. This behavior can be disabled with the Permissive option.

To maintain backward compatibility, all other constructors default to permissive unmarshalling.

Quote Strings

YAML allows strings to appear quoted or unquoted, so these two lines are identical: 

foo: bar
"foo": "bar"

However, the YAML specification special-cases some unquoted strings. Most obviously, true and false are interpreted as Booleans (unless quoted). Less obviously, yes, no, on, off, and many variants of these words are also treated as Booleans (see http://yaml.org/type/bool.html for the complete specification).

Correctly deep-merging sources requires this package to unmarshal and then remarshal all YAML, which implicitly converts these special-cased unquoted strings to their canonical representation. For example, 

foo: yes  # before merge
foo: true # after merge

Quoting special-cased strings prevents this surprising behavior.

Deprecated APIs

Unfortunately, this package was released with a variety of bugs and an overly large API. The internals of the configuration provider have been completely reworked and all known bugs have been addressed, but many duplicative exported functions were retained to preserve backward compatibility. New users should rely on the NewYAML constructor. In particular, avoid NewValue - it's unnecessary, complex, and may panic.

Deprecated functions are documented in the format expected by the staticcheck linter, available at https://staticcheck.io/.

Example 
package main

import (
	"fmt"
	"strings"

	"go.uber.org/config"
)

func main() {
	// A struct to represent the configuration of a self-contained unit of your
	// application.
	type cfg struct {
		Parameter string
	}

	// Two sources of YAML configuration to merge. We could also use
	// config.Static to supply some configuration as a Go struct.
	base := strings.NewReader("module: {parameter: foo}")
	override := strings.NewReader("module: {parameter: bar}")

	// Merge the two sources into a Provider. Later sources are higher-priority.
	// See the top-level package documentation for details on the merging logic.
	provider, err := config.NewYAML(config.Source(base), config.Source(override))
	if err != nil {
		panic(err)
	}

	var c cfg
	if err := provider.Get("module").Populate(&c); err != nil {
		panic(err)
	}

	fmt.Printf("%+v\n", c)
}

Output:

{Parameter:bar}

Index

Examples

Constants

View Source 
const Root = ""

Root is a virtual key that accesses the entire configuration. Using it as the key when calling Provider.Get or Value.Get returns the whole configuration.

View Source 
const Version = "1.4.0"

Version is the current semantic version.

Variables

This section is empty.

Functions

This section is empty.

Types

type LookupFunc added in v1.2.0 

type LookupFunc = func(string) (string, bool)

A LookupFunc behaves like os.LookupEnv: it uses the supplied string as a key into some key-value store and returns the value and whether the key was present.

type NopProvider 

type NopProvider struct{}

NopProvider is a no-op provider.

func (NopProvider) Get 

func (n NopProvider) Get(_ string) Value

Get returns a value with no configuration available.

func (NopProvider) Name 

func (NopProvider) Name() string

Name implements Provider.

type Provider 

type Provider interface {
	Name() string         // name of the configuration store
	Get(key string) Value // retrieves a portion of the configuration, see Value for details
}

Provider is an abstraction over a configuration store, such as a collection of merged YAML, JSON, or TOML files.

func NewProviderGroup 

func NewProviderGroup(name string, providers ...Provider) (Provider, error)

NewProviderGroup composes multiple providers, with later providers overriding earlier ones. The merge logic is described in the package-level documentation. To preserve backward compatibility, the resulting provider disables strict unmarshalling.

Prefer using NewYAML instead of this where possible. NewYAML gives you strict unmarshalling by default and allows use of other options at the same time.

func NewScopedProvider 

func NewScopedProvider(prefix string, provider Provider) Provider

NewScopedProvider wraps a provider and adds a prefix to all Get calls.

func NewStaticProvider deprecated 

func NewStaticProvider(data interface{}) (Provider, error)

NewStaticProvider serializes a Go data structure to YAML, then loads it into a provider. To preserve backward compatibility, the resulting provider disables strict unmarshalling.

Deprecated: use NewYAML and the Static option directly. This enables strict unmarshalling by default and allows use of other options at the same time.

func NewStaticProviderWithExpand deprecated 

func NewStaticProviderWithExpand(data interface{}, lookup LookupFunc) (Provider, error)

NewStaticProviderWithExpand serializes a Go data structure to YAML, expands any environment variable references using the supplied lookup function, then loads the result into a provider. See the Expand option for a description of the environment variable replacement syntax. To preserve backward compatibility, the resulting provider disables strict unmarshalling.

Deprecated: use NewYAML and the Static and Expand options directly. This enables strict unmarshalling by default and allows use of other options at the same time.

func NewYAMLProviderFromBytes deprecated 

func NewYAMLProviderFromBytes(yamls ...[]byte) (Provider, error)

NewYAMLProviderFromBytes merges multiple YAML-formatted byte slices into a single provider. Later configuration blobs override earlier ones using the merge logic described in the package-level documentation. To preserve backward compatibility, the resulting provider disables strict unmarshalling.

Deprecated: use NewYAML with the Source and Expand options directly. This enables strict unmarshalling by default and allows use of other options at the same time.

func NewYAMLProviderFromFiles deprecated 

func NewYAMLProviderFromFiles(filenames ...string) (Provider, error)

NewYAMLProviderFromFiles opens and merges multiple YAML files into a single provider. Later files override earlier files using the merge logic described in the package-level documentation. To preserve backward compatibility, the resulting provider disables strict unmarshalling.

Deprecated: use NewYAML and the File option directly. This enables strict unmarshalling by default and allows use of other options at the same time.

func NewYAMLProviderFromReader deprecated 

func NewYAMLProviderFromReader(readers ...io.Reader) (Provider, error)

NewYAMLProviderFromReader merges multiple YAML-formatted io.Readers into a single provider. Later readers override earlier ones using the merge logic described in the package-level documentation. To preserve backward compatibility, the resulting provider disables strict unmarshalling.

Deprecated: use NewYAML and the Source option directly. This enables strict unmarshalling by default and allows use of other options at the same time.

func NewYAMLProviderFromReaderWithExpand deprecated 

func NewYAMLProviderFromReaderWithExpand(lookup LookupFunc, readers ...io.Reader) (Provider, error)

NewYAMLProviderFromReaderWithExpand merges multiple YAML-formatted io.Readers, expands any environment variable references using the supplied lookup function, and then loads the result into a provider. Later readers override earlier readers using the merge logic described in the package-level documentation. See the Expand option for a description of the environment variable replacement syntax. To preserve backward compatibility, the resulting provider disables strict unmarshalling.

Deprecated: use NewYAML and the Source and Expand options directly. This enables strict unmarshalling by default and allows use of other options at the same time.

func NewYAMLProviderWithExpand deprecated 

func NewYAMLProviderWithExpand(lookup LookupFunc, filenames ...string) (Provider, error)

NewYAMLProviderWithExpand opens and merges multiple YAML-formatted files, expands any environment variable references using the supplied lookup function, and then loads the result into a provider. Later readers override earlier readers using the merge logic described in the package-level documentation. See the Expand option for a description of the environment variable replacement syntax. To preserve backward compatibility, the resulting provider disables strict unmarshalling.

Deprecated: use NewYAML and the File and Expand options directly. This enables strict unmarshalling by default and allows use of other options at the same time.

type Value 

type Value struct {
	// contains filtered or unexported fields
}

A Value is a subset of a provider's configuration.

func NewValue deprecated 

func NewValue(p Provider, key string, value interface{}, found bool) Value

NewValue is a highly error-prone constructor preserved only for backward compatibility. If value and found don't match the contents of the provider at the supplied key, it panics.

Deprecated: this internal constructor was mistakenly exported in the initial release of this package, but its behavior was often very surprising. To guarantee sane behavior without changing the function signature, input validation and panics were added in version 1.2. In all cases, it's both safer and less verbose to use Provider.Get directly.

func (Value) Get 

func (v Value) Get(path string) Value

Get dives further into the configuration, pulling out more deeply nested values. The supplied path is split on periods, and each segment is treated as a nested map key. For example, if the current value holds the YAML configuration 

foo:
  bar:
    baz: quux

then a call to Get("foo.bar") will hold the YAML mapping 

baz: quux

func (Value) HasValue deprecated 

func (v Value) HasValue() bool

HasValue checks whether any configuration is available at this key.

It doesn't distinguish between configuration supplied during provider construction and configuration applied by WithDefault. If the value has explicitly been set to nil, HasValue is true.

Deprecated: this function has little value and is often confusing. Rather than checking whether a value has any configuration available, Populate a struct with appropriate defaults and zero values.

func (Value) Populate 

func (v Value) Populate(target interface{}) error

Populate unmarshals the value into the target struct, much like json.Unmarshal or yaml.Unmarshal. When populating a struct with some fields already set, data is deep-merged as described in the package-level documentation.

func (Value) Source 

func (v Value) Source() string

Source returns the name of the value's provider.

func (Value) String 

func (v Value) String() string

func (Value) Value deprecated 

func (v Value) Value() interface{}

Value unmarshals the configuration into interface{}.

Deprecated: in a strongly-typed language, unmarshaling configuration into interface{} isn't helpful. It's safer and easier to use Populate with a strongly-typed struct.

func (Value) WithDefault deprecated 

func (v Value) WithDefault(d interface{}) (Value, error)

WithDefault supplies a default configuration for the value. The default is serialized to YAML, and then the existing configuration sources are deep-merged into it using the merge logic described in the package-level documentation. Note that applying defaults requires re-expanding environment variables, which may have unexpected results if the environment changes after provider construction.

Deprecated: the deep-merging behavior of WithDefault is complex, especially when applied multiple times. Instead, create a Go struct, set any defaults directly on the struct, then call Populate.

Example 
package main

import (
	"fmt"

	"go.uber.org/config"
)

func main() {
	provider, err := config.NewYAML(config.Static(map[string]string{
		"key": "value",
	}))
	if err != nil {
		panic(err)
	}
	// Using config.Root as a key retrieves the whole configuration.
	base := provider.Get(config.Root)

	// Applying a default is equivalent to serializing it to YAML, writing the
	// serialized bytes to default.yaml, and then merging the existing
	// configuration into default.yaml. Maps are deep-merged!
	//
	// Since we're setting the default for a key that's not already in the
	// configuration, new_key will now be set to new_value. From now on, it's
	// impossible to tell whether the value of new_key came from the original
	// provider or a call to WithDefault.
	defaulted, err := base.WithDefault(map[string]string{
		"new_key": "new_value",
	})
	if err != nil {
		panic(err)
	}
	fmt.Println(defaulted)

	// If we try to use WithDefault again to set different defaults for the two
	// existing keys, nothing happens - since both keys already have scalar
	// values, those values overwrite the new defaults in the merge. See the
	// package-level documentation for a more detailed discussion of the merge
	// logic.
	again, err := defaulted.WithDefault(map[string]string{
		"key":     "ignored",
		"new_key": "ignored",
	})
	if err != nil {
		panic(err)
	}
	fmt.Println(again)
}

Output:

type YAML added in v1.2.0 

type YAML struct {
	// contains filtered or unexported fields
}

YAML is a provider that reads from one or more YAML sources. Many aspects of the resulting provider's behavior can be altered by passing functional options.

By default, the YAML provider attempts to proactively catch common mistakes by enabling gopkg.in/yaml.v2's strict mode. See the package-level documentation on strict unmarshalling for details.

When populating Go structs, values produced by the YAML provider correctly handle all struct tags supported by gopkg.in/yaml.v2. See https://godoc.org/gopkg.in/yaml.v2#Marshal for details.

func NewYAML added in v1.2.0 

func NewYAML(options ...YAMLOption) (*YAML, error)

NewYAML constructs a YAML provider. See the various YAMLOptions for available tweaks to the default behavior.

func (*YAML) Get added in v1.2.0 

func (y *YAML) Get(key string) Value

Get retrieves a value from the configuration. The supplied key is treated as a period-separated path, with each path segment used as a map key. For example, if the provider contains the YAML 

foo:
  bar:
    baz: hello

then Get("foo.bar") returns a value holding 

baz: hello

To get a value holding the entire configuration, use the Root constant as the key.

func (*YAML) Name added in v1.2.0 

func (y *YAML) Name() string

Name returns the name of the provider. It defaults to "YAML".

type YAMLOption added in v1.2.0 

type YAMLOption interface {
	// contains filtered or unexported methods
}

A YAMLOption alters the default configuration of the YAML configuration provider.

func Expand added in v1.2.0 

func Expand(lookup LookupFunc) YAMLOption

Expand enables variable expansion in all non-raw provided sources. The supplied function MUST behave like os.LookupEnv: it looks up a key and returns a value and whether the key was found. Any expansion is deferred until after all sources are merged, so it's not possible to reference different variables in different sources and have the values automatically merged.

Expand allows variable references to take two forms: $VAR or ${VAR:default}. In the first form, variable names MUST adhere to shell naming rules: 

...a word consisting solely of underscores, digits, and alphabetics form
the portable character set. The first character of a name may not be a
digit.

In this form, NewYAML returns an error if any referenced variables aren't found.

In the second form, all characters between the opening curly brace and the first colon are used as the key, and all characters from the colon to the closing curly brace are used as the default value. Keys need not adhere to the shell naming rules above. If a variable isn't found, the default value is used.

$$ is expanded to a literal $.

func File added in v1.2.0 

func File(name string) YAMLOption

File opens a file, uses it as a source of YAML configuration, and closes it once provider construction is complete. Priority, merge, and expansion logic are identical to Source.

func Name added in v1.2.0 

func Name(name string) YAMLOption

Name customizes the name of the provider. The default name is "YAML".

func Permissive added in v1.2.0 

func Permissive() YAMLOption

Permissive disables gopkg.in/yaml.v2's strict mode. It's provided for backward compatibility; to avoid a variety of common mistakes, most users should leave YAML providers in the default strict mode.

In permissive mode, duplicate keys in the same source file are allowed. Later values override earlier ones (note that duplicates are NOT merged, unlike all other merges in this package). Calls to Populate that don't use all keys present in the YAML are allowed. Finally, type conflicts are allowed when merging source files, with later values replacing earlier ones.

func RawSource added in v1.3.0 

func RawSource(r io.Reader) YAMLOption

RawSource adds a source of YAML configuration. Later sources override earlier ones using the merge logic described in the package-level documentation.

Raw sources are not subject to variable expansion. To provide a source with variable expansion enabled, use the Source option.

func Source added in v1.2.0 

func Source(r io.Reader) YAMLOption

Source adds a source of YAML configuration. Later sources override earlier ones using the merge logic described in the package-level documentation.

Sources are subject to variable expansion (via the Expand option). To provide a source that remains unexpanded, use the RawSource option.

func Static added in v1.2.0 

func Static(val interface{}) YAMLOption

Static serializes a Go data structure to YAML and uses the result as a source. If serialization fails, provider construction will return an error. Priority, merge, and expansion logic are identical to Source.

Source Files

View all Source files

Directories

Path Synopsis
internal
merge
unreachable

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.