koanf

README

koanf (pronounced conf; a play on the Japanese Koan) is a library for reading configuration from different sources in different formats in Go applications. It is a cleaner, lighter alternative to spf13/viper with better abstractions and extensibility and fewer dependencies.

koanf comes with built in support for reading configuration from files, command line flags, and environment variables, and can parse JSON, YAML, TOML, and Hashicorp HCL. Any external dependencies are detatched from the core into sub-packages, so only the ones that are explicitly referenced get compiled into an application.

Installation

go get -u github.com/knadh/koanf

Contents

• Concepts
• Reading config from files
• Watching files for changes
• Reading from command line
• Reading environment variables
• Reading raw bytes
• Unmarshalling and marshalling
• Unmarshalling with flat paths
• Setting default values
• Order of merge and key case senstivity
• Custom Providers and Parsers
• Custom Merge Strategies
• List of Providers, Parsers, and functions

Concepts

• koanf.Provider is a generic interface that provides configuration, for example, from files, environment variables, HTTP sources, or anywhere. The configuration can either be raw bytes that a parser can parse, or it can be a nested map[string]interface{} that can be directly loaded.
• koanf.Parser is a generic interface that takes raw bytes, parses, and returns a nested map[string]interface{} representation. For example, JSON and YAML parsers.
• Once loaded into koanf, configuration are values queried by a delimited key path syntax. eg: app.server.port. Any delimiter can be chosen.
• Configuration from multiple sources can be loaded and merged into a koanf instance, for example, load from a file first and override certain values with flags from the command line.

With these two interface implementations, koanf can obtain a configuration from multiple sources and parse any format and make it available to an application.

Reading config from files

package main

import (
	"fmt"
	"log"

	"github.com/knadh/koanf"
	"github.com/knadh/koanf/parsers/json"
	"github.com/knadh/koanf/parsers/yaml"
	"github.com/knadh/koanf/providers/file"
)

// Global koanf instance. Use "." as the key path delimiter. This can be "/" or any character.
var k = koanf.New(".")

func main() {
	// Load JSON config.
	if err := k.Load(file.Provider("mock/mock.json"), json.Parser()); err != nil {
		log.Fatalf("error loading config: %v", err)
	}

	// Load YAML config and merge into the previously loaded config (because we can).
	k.Load(file.Provider("mock/mock.yml"), yaml.Parser())

	fmt.Println("parent's name is = ", k.String("parent1.name"))
	fmt.Println("parent's ID is = ", k.Int("parent1.id"))
}

Watching files for changes

Some providers expose a Watch() method that makes the provider watch for changes in configuration and trigger a callback to reload the configuration. This is not goroutine safe if there are concurrent *Get() calls happening on the koanf object while it is doing a Load(). Such scenarios will need mutex locking.

file, appconfig, vault, consul providers have a Watch() method.

package main

import (
	"fmt"
	"log"

	"github.com/knadh/koanf"
	"github.com/knadh/koanf/parsers/json"
	"github.com/knadh/koanf/parsers/yaml"
	"github.com/knadh/koanf/providers/file"
)

// Global koanf instance. Use "." as the key path delimiter. This can be "/" or any character.
var k = koanf.New(".")

func main() {
	// Load JSON config.
	f := file.Provider("mock/mock.json")
	if err := k.Load(f, json.Parser()); err != nil {
		log.Fatalf("error loading config: %v", err)
	}

	// Load YAML config and merge into the previously loaded config (because we can).
	k.Load(file.Provider("mock/mock.yml"), yaml.Parser())

	fmt.Println("parent's name is = ", k.String("parent1.name"))
	fmt.Println("parent's ID is = ", k.Int("parent1.id"))

	// Watch the file and get a callback on change. The callback can do whatever,
	// like re-load the configuration.
	// File provider always returns a nil `event`.
	f.Watch(func(event interface{}, err error) {
		if err != nil {
			log.Printf("watch error: %v", err)
			return
		}

		// Throw away the old config and load a fresh copy.
		log.Println("config changed. Reloading ...")
		k = koanf.New(".")
		k.Load(f, json.Parser())
		k.Print()
	})

	// Block forever (and manually make a change to mock/mock.json) to
	// reload the config.
	log.Println("waiting forever. Try making a change to mock/mock.json to live reload")
	<-make(chan bool)
}

Reading from command line

The following example shows the use of posflag.Provider, a wrapper over the spf13/pflag library, an advanced commandline lib. For Go's built in flag package, use basicflag.Provider.

package main

import (
	"fmt"
	"log"
	"os"

	"github.com/knadh/koanf"
	"github.com/knadh/koanf/parsers/toml"
	"github.com/knadh/koanf/providers/file"
	"github.com/knadh/koanf/providers/posflag"
	flag "github.com/spf13/pflag"
)

// Global koanf instance. Use "." as the key path delimiter. This can be "/" or any character.
var k = koanf.New(".")

func main() {
	// Use the POSIX compliant pflag lib instead of Go's flag lib.
	f := flag.NewFlagSet("config", flag.ContinueOnError)
	f.Usage = func() {
		fmt.Println(f.FlagUsages())
		os.Exit(0)
	}
	// Path to one or more config files to load into koanf along with some config params.
	f.StringSlice("conf", []string{"mock/mock.toml"}, "path to one or more .toml config files")
	f.String("time", "2020-01-01", "a time string")
	f.String("type", "xxx", "type of the app")
	f.Parse(os.Args[1:])

	// Load the config files provided in the commandline.
	cFiles, _ := f.GetStringSlice("conf")
	for _, c := range cFiles {
		if err := k.Load(file.Provider(c), toml.Parser()); err != nil {
			log.Fatalf("error loading file: %v", err)
		}
	}

	// "time" and "type" may have been loaded from the config file, but
	// they can still be overridden with the values from the command line.
	// The bundled posflag.Provider takes a flagset from the spf13/pflag lib.
	// Passing the Koanf instance to posflag helps it deal with default command
	// line flag values that are not present in conf maps from previously loaded
	// providers.
	if err := k.Load(posflag.Provider(f, ".", k), nil); err != nil {
		log.Fatalf("error loading config: %v", err)
	}

	fmt.Println("time is = ", k.String("time"))
}

Reading environment variables

package main

import (
	"fmt"
	"log"
	"strings"

	"github.com/knadh/koanf"
	"github.com/knadh/koanf/parsers/json"
	"github.com/knadh/koanf/providers/env"
	"github.com/knadh/koanf/providers/file"
)

// Global koanf instance. Use . as the key path delimiter. This can be / or anything.
var k = koanf.New(".")

func main() {
	// Load JSON config.
	if err := k.Load(file.Provider("mock/mock.json"), json.Parser()); err != nil {
		log.Fatalf("error loading config: %v", err)
	}

	// Load environment variables and merge into the loaded config.
	// "MYVAR" is the prefix to filter the env vars by.
	// "." is the delimiter used to represent the key hierarchy in env vars.
	// The (optional, or can be nil) function can be used to transform
	// the env var names, for instance, to lowercase them.
	//
	// For example, env vars: MYVAR_TYPE and MYVAR_PARENT1_CHILD1_NAME
	// will be merged into the "type" and the nested "parent1.child1.name"
	// keys in the config file here as we lowercase the key, 
	// replace `_` with `.` and strip the MYVAR_ prefix so that 
	// only "parent1.child1.name" remains.
	k.Load(env.Provider("MYVAR_", ".", func(s string) string {
		return strings.Replace(strings.ToLower(
			strings.TrimPrefix(s, "MYVAR_")), "_", ".", -1)
	}), nil)

	fmt.Println("name is = ", k.String("parent1.child1.name"))
}

You can also use the env.ProviderWithValue with a callback that supports mutating both the key and value to return types other than a string. For example, here, env values separated by spaces are returned as string slices or arrays. eg: MYVAR_slice=a b c becomes slice: [a, b, c].

	k.Load(env.ProviderWithValue("MYVAR_", ".", func(s string, v string) (string, interface{}) {
		// Strip out the MYVAR_ prefix and lowercase and get the key while also replacing
		// the _ character with . in the key (koanf delimeter).
		key := strings.Replace(strings.ToLower(strings.TrimPrefix(s, "MYVAR_")), "_", ".", -1)

		// If there is a space in the value, split the value into a slice by the space.
		if strings.Contains(v, " ") {
			return key, strings.Split(v, " ")
		}

		// Otherwise, return the plain string.
		return key, v
	}), nil)

Reading from an S3 bucket

// Load JSON config from s3.
if err := k.Load(s3.Provider(s3.Config{
	AccessKey: os.Getenv("AWS_S3_ACCESS_KEY"),
	SecretKey: os.Getenv("AWS_S3_SECRET_KEY"),
	Region:    os.Getenv("AWS_S3_REGION"),
	Bucket:    os.Getenv("AWS_S3_BUCKET"),
	ObjectKey: "dir/config.json",
}), json.Parser()); err != nil {
	log.Fatalf("error loading config: %v", err)
}

Reading raw bytes

The bundled rawbytes Provider can be used to read arbitrary bytes from a source, like a database or an HTTP call.

package main

import (
	"fmt"

	"github.com/knadh/koanf"
	"github.com/knadh/koanf/parsers/json"
	"github.com/knadh/koanf/providers/rawbytes"
)

// Global koanf instance. Use . as the key path delimiter. This can be / or anything.
var k = koanf.New(".")

func main() {
	b := []byte(`{"type": "rawbytes", "parent1": {"child1": {"type": "rawbytes"}}}`)
	k.Load(rawbytes.Provider(b), json.Parser())
	fmt.Println("type is = ", k.String("parent1.child1.type"))
}

Unmarshalling and marshalling

Parsers can be used to unmarshal and scan the values in a Koanf instance into a struct based on the field tags, and to marshal a Koanf instance back into serialized bytes, for example, back to JSON or YAML, to write back to files.

package main

import (
	"fmt"
	"log"

	"github.com/knadh/koanf"
	"github.com/knadh/koanf/parsers/json"
	"github.com/knadh/koanf/providers/file"
)

// Global koanf instance. Use . as the key path delimiter. This can be / or anything.
var (
	k      = koanf.New(".")
	parser = json.Parser()
)

func main() {
	// Load JSON config.
	if err := k.Load(file.Provider("mock/mock.json"), parser); err != nil {
		log.Fatalf("error loading config: %v", err)
	}

	// Structure to unmarshal nested conf to.
	type childStruct struct {
		Name       string            `koanf:"name"`
		Type       string            `koanf:"type"`
		Empty      map[string]string `koanf:"empty"`
		GrandChild struct {
			Ids []int `koanf:"ids"`
			On  bool  `koanf:"on"`
		} `koanf:"grandchild1"`
	}

	var out childStruct

	// Quick unmarshal.
	k.Unmarshal("parent1.child1", &out)
	fmt.Println(out)

	// Unmarshal with advanced config.
	out = childStruct{}
	k.UnmarshalWithConf("parent1.child1", &out, koanf.UnmarshalConf{Tag: "koanf"})
	fmt.Println(out)

	// Marshal the instance back to JSON.
	// The paser instance can be anything, eg: json.Paser(), yaml.Parser() etc.
	b, _ := k.Marshal(parser)
	fmt.Println(string(b))
}

Unmarshalling with flat paths

Sometimes it is necessary to unmarshal an assortment of keys from various nested structures into a flat target structure. This is possible with the UnmarshalConf.FlatPaths flag.

package main

import (
	"fmt"
	"log"

	"github.com/knadh/koanf"
	"github.com/knadh/koanf/parsers/json"
	"github.com/knadh/koanf/providers/file"
)

// Global koanf instance. Use . as the key path delimiter. This can be / or anything.
var k = koanf.New(".")

func main() {
	// Load JSON config.
	if err := k.Load(file.Provider("mock/mock.json"), json.Parser()); err != nil {
		log.Fatalf("error loading config: %v", err)
	}

	type rootFlat struct {
		Type                        string            `koanf:"type"`
		Empty                       map[string]string `koanf:"empty"`
		Parent1Name                 string            `koanf:"parent1.name"`
		Parent1ID                   int               `koanf:"parent1.id"`
		Parent1Child1Name           string            `koanf:"parent1.child1.name"`
		Parent1Child1Type           string            `koanf:"parent1.child1.type"`
		Parent1Child1Empty          map[string]string `koanf:"parent1.child1.empty"`
		Parent1Child1Grandchild1IDs []int             `koanf:"parent1.child1.grandchild1.ids"`
		Parent1Child1Grandchild1On  bool              `koanf:"parent1.child1.grandchild1.on"`
	}

	// Unmarshal the whole root with FlatPaths: True.
	var o1 rootFlat
	k.UnmarshalWithConf("", &o1, koanf.UnmarshalConf{Tag: "koanf", FlatPaths: true})
	fmt.Println(o1)

	// Unmarshal a child structure of "parent1".
	type subFlat struct {
		Name                 string            `koanf:"name"`
		ID                   int               `koanf:"id"`
		Child1Name           string            `koanf:"child1.name"`
		Child1Type           string            `koanf:"child1.type"`
		Child1Empty          map[string]string `koanf:"child1.empty"`
		Child1Grandchild1IDs []int             `koanf:"child1.grandchild1.ids"`
		Child1Grandchild1On  bool              `koanf:"child1.grandchild1.on"`
	}

	var o2 subFlat
	k.UnmarshalWithConf("parent1", &o2, koanf.UnmarshalConf{Tag: "koanf", FlatPaths: true})
	fmt.Println(o2)
}

Marshalling and writing config

It is possible to marshal and serialize the conf map into TOML, YAML etc.

Setting default values.

koanf does not provide any special functions to set default values but uses the Provider interface to enable it.

From a map

The bundled confmap provider takes a map[string]interface{} that can be loaded into a koanf instance.

package main

import (
	"fmt"
	"log"

	"github.com/knadh/koanf"
	"github.com/knadh/koanf/providers/confmap"
	"github.com/knadh/koanf/providers/file"
	"github.com/knadh/koanf/parsers/json"
	"github.com/knadh/koanf/parsers/yaml"
)

// Global koanf instance. Use "." as the key path delimiter. This can be "/" or any character.
var k = koanf.New(".")

func main() {
	// Load default values using the confmap provider.
	// We provide a flat map with the "." delimiter.
	// A nested map can be loaded by setting the delimiter to an empty string "".
	k.Load(confmap.Provider(map[string]interface{}{
		"parent1.name": "Default Name",
		"parent3.name": "New name here",
	}, "."), nil)

	// Load JSON config on top of the default values.
	if err := k.Load(file.Provider("mock/mock.json"), json.Parser()); err != nil {
		log.Fatalf("error loading config: %v", err)
	}

	// Load YAML config and merge into the previously loaded config (because we can).
	k.Load(file.Provider("mock/mock.yml"), yaml.Parser())

	fmt.Println("parent's name is = ", k.String("parent1.name"))
	fmt.Println("parent's ID is = ", k.Int("parent1.id"))
}

From a struct

The bundled structs provider can be used to read data from a struct to load into a koanf instance.

package main

import (
	"fmt"

	"github.com/knadh/koanf"
	"github.com/knadh/koanf/providers/structs"
)

// Global koanf instance. Use "." as the key path delimiter. This can be "/" or any character.
var k = koanf.New(".")

type parentStruct struct {
	Name   string      `koanf:"name"`
	ID     int         `koanf:"id"`
	Child1 childStruct `koanf:"child1"`
}
type childStruct struct {
	Name        string            `koanf:"name"`
	Type        string            `koanf:"type"`
	Empty       map[string]string `koanf:"empty"`
	Grandchild1 grandchildStruct  `koanf:"grandchild1"`
}
type grandchildStruct struct {
	Ids []int `koanf:"ids"`
	On  bool  `koanf:"on"`
}
type sampleStruct struct {
	Type    string            `koanf:"type"`
	Empty   map[string]string `koanf:"empty"`
	Parent1 parentStruct      `koanf:"parent1"`
}

func main() {
	// Load default values using the structs provider.
	// We provide a struct along with the struct tag `koanf` to the
	// provider.
	k.Load(structs.Provider(sampleStruct{
		Type:  "json",
		Empty: make(map[string]string),
		Parent1: parentStruct{
			Name: "parent1",
			ID:   1234,
			Child1: childStruct{
				Name:  "child1",
				Type:  "json",
				Empty: make(map[string]string),
				Grandchild1: grandchildStruct{
					Ids: []int{1, 2, 3},
					On:  true,
				},
			},
		},
	}, "koanf"), nil)

	fmt.Printf("name is = `%s`\n", k.String("parent1.child1.name"))
}

Merge behavior

Default behavior

The default behavior when you create Koanf this way is: koanf.New(delim) that the latest loaded configuration will merge with the previous one.

For example: first.yml

key: [1,2,3]

second.yml

key: 'string'

When second.yml is loaded it will override the type of the first.yml.

If this behavior is not desired, you can merge 'strictly'. In the same scenario, Load will return an error.

package main

import (
	"errors"
	"log"

	"github.com/knadh/koanf"
	"github.com/knadh/koanf/maps"
	"github.com/knadh/koanf/parsers/json"
	"github.com/knadh/koanf/parsers/yaml"
	"github.com/knadh/koanf/providers/file"
)

var conf = koanf.Conf{
	Delim:       ".",
	StrictMerge: true,
}
var k = koanf.NewWithConf(conf)

func main() {
	yamlPath := "mock/mock.yml"
	if err := k.Load(file.Provider(yamlPath), yaml.Parser()); err != nil {
		log.Fatalf("error loading config: %v", err)
	}

	jsonPath := "mock/mock.json"
	if err := k.Load(file.Provider(jsonPath), json.Parser()); err != nil {
		log.Fatalf("error loading config: %v", err)
	}
}

Note: When merging different extensions, each parser can treat his types differently, meaning even though you the load same types there is a probability that it will fail with StrictMerge: true.

For example: merging JSON and YAML will most likely fail because JSON treats integers as float64 and YAML treats them as integers.

Order of merge and key case sensitivity

• Config keys are case-sensitive in koanf. For example, app.server.port and APP.SERVER.port are not the same.
• koanf does not impose any ordering on loading config from various providers. Every successive Load() or Load() merges new config into existing config. That means it is possible to load environment variables first, then files on top of it, and then command line variables on top of it, or any such order.

Custom Providers and Parsers

A Provider can provide a nested map[string]interface{} config that can be loaded into koanf with koanf.Load() or raw bytes that can be parsed with a Parser (loaded using koanf.Load().

Writing Providers and Parsers are easy. See the bundled implementations in the providers and parses directory.

Custom merge strategies

By default, when merging two config sources using Load(), koanf recursively merges keys of nested maps (map[string]interface{}), while static values are overwritten (slices, strings, etc). This behaviour can be changed by providing a custom merge function with the WithMergeFunc option.

package main

import (
	"errors"
	"log"

	"github.com/knadh/koanf"
	"github.com/knadh/koanf/maps"
	"github.com/knadh/koanf/parsers/json"
	"github.com/knadh/koanf/parsers/yaml"
	"github.com/knadh/koanf/providers/file"
)

var conf = koanf.Conf{
	Delim:       ".",
	StrictMerge: true,
}
var k = koanf.NewWithConf(conf)

func main() {
	yamlPath := "mock/mock.yml"
	if err := k.Load(file.Provider(yamlPath), yaml.Parser()); err != nil {
		log.Fatalf("error loading config: %v", err)
	}

	jsonPath := "mock/mock.json"
	if err := k.Load(file.Provider(jsonPath), json.Parser(), koanf.WithMergeFunc(func(src, dest map[string]interface{}) error {
     // Your custom logic, copying values from src into dst
     return nil
    })); err != nil {
		log.Fatalf("error loading config: %v", err)
	}
}

API

Instantiation methods

Method	Description
New(delim string) *Koanf	Returns a new instance of Koanf. delim is the delimiter to use when specifying config key paths, for instance a . for parent.child.key or a / for parent/child/key.
NewWithConf(conf Conf) *Koanf	Returns a new instance of Koanf depending on the Koanf.Conf configurations.

Structs

Struct	Description
Koanf	Koanf is the configuration apparatus.
Conf	Conf is the Koanf configuration, that includes Delim and StrictMerge.
UnmarshalConf	UnmarshalConf represents configuration options used by Unmarshal() to unmarshal conf maps in arbitrary structs

Bundled providers

Package	Provider	Description
providers/file	file.Provider(filepath string)	Reads a file and returns the raw bytes to be parsed.
providers/fs	fs.Provider(f fs.FS, filepath string)	(Experimental) Reads a file from fs.FS and returns the raw bytes to be parsed. The provider requires go v1.16 or higher.
providers/basicflag	basicflag.Provider(f *flag.FlagSet, delim string)	Takes an stdlib flag.FlagSet
providers/posflag	posflag.Provider(f *pflag.FlagSet, delim string)	Takes an spft3/pflag.FlagSet (advanced POSIX compatible flags with multiple types) and provides a nested config map based on delim.
providers/env	env.Provider(prefix, delim string, f func(s string) string)	Takes an optional prefix to filter env variables by, an optional function that takes and returns a string to transform env variables, and returns a nested config map based on delim.
providers/confmap	confmap.Provider(mp map[string]interface{}, delim string)	Takes a premade map[string]interface{} conf map. If delim is provided, the keys are assumed to be flattened, thus unflattened using delim.
providers/structs	structs.Provider(s interface{}, tag string)	Takes a struct and struct tag.
providers/s3	s3.Provider(s3.S3Config{})	Takes a s3 config struct.
providers/rawbytes	rawbytes.Provider(b []byte)	Takes a raw []byte slice to be parsed with a koanf.Parser
providers/vault	vault.Provider(vault.Config{})	Hashicorp Vault provider
providers/appconfig	vault.AppConfig(appconfig.Config{})	AWS AppConfig provider
providers/etcd	etcd.Provider(etcd.Config{})	CNCF etcd provider
providers/consul	consul.Provider(consul.Config{})	Hashicorp Consul provider

Third-party providers

Package	Provider	Description
github.com/defensestation/koanf/providers/secretsmanager	vault.SecretsMananger(secretsmanager.Config{}, f func(s string) string)	AWS Secrets Manager provider, takes map or string as a value from store
github.com/defensestation/koanf/providers/parameterstore	vault.ParameterStore(parameterstore.Config{}, f func(s string) string)	AWS ParameterStore provider, an optional function that takes and returns a string to transform env variables

Bundled parsers

Package	Parser	Description
parsers/json	json.Parser()	Parses JSON bytes into a nested map
parsers/yaml	yaml.Parser()	Parses YAML bytes into a nested map
parsers/toml	toml.Parser()	Parses TOML bytes into a nested map
parsers/dotenv	dotenv.Parser()	Parses DotEnv bytes into a flat map
parsers/hcl	hcl.Parser(flattenSlices bool)	Parses Hashicorp HCL bytes into a nested map. flattenSlices is recommended to be set to true. Read more.
parsers/nestedtext	nestedtext.Parser()	Parses NestedText bytes into a flat map
parsers/hjson	hjson.Parser()	Parses HJSON bytes into a nested map

|

Instance functions

Method	Description
Load(p Provider, pa Parser, opts ...Option) error	Loads config from a Provider. If a koanf.Parser is provided, the config is assumed to be raw bytes that's then parsed with the Parser.
Keys() []string	Returns the list of flattened key paths that can be used to access config values
KeyMap() map[string][]string	Returns a map of all possible key path combinations possible in the loaded nested conf map
All() map[string]interface{}	Returns a flat map of flattened key paths and their corresponding config values
Raw() map[string]interface{}	Returns a copy of the raw nested conf map
Print()	Prints a human readable copy of the flattened key paths and their values for debugging
Sprint()	Returns a human readable copy of the flattened key paths and their values for debugging
Cut(path string) *Koanf	Cuts the loaded nested conf map at the given path and returns a new Koanf instance with the children
Copy() *Koanf	Returns a copy of the Koanf instance
Merge(*Koanf)	Merges the config map of a Koanf instance into the current instance
MergeAt(in *Koanf, path string)	Merges the config map of a Koanf instance into the current instance, at the given key path.
Set(path string, val interface{})	Shorthand wrapper around Merge() for directly overriding the value of a given key path.
Delete(path string)	Delete the value at the given path, and does nothing if path doesn't exist.
Unmarshal(path string, o interface{}) error	Scans the given nested key path into a given struct (like json.Unmarshal) where fields are denoted by the koanf tag
UnmarshalWithConf(path string, o interface{}, c UnmarshalConf) error	Like Unmarshal but with customizable options

Getter functions

Get(path string) interface{}	Returns the value for the given key path, and if it doesn’t exist, returns nil
Exists(path string) bool	Returns true if the given key path exists in the conf map
Int64(path string) int64
Int64s(path string) []int64
Int64Map(path string) map[string]int64
Int(path string) int
Ints(path string) []int
IntMap(path string) map[string]int
Float64(path string) float64
Float64s(path string) []float64
Float64Map(path string) map[string]float64
Duration(path string) time.Duration	Returns the time.Duration value of the given key path if it’s numeric (attempts a parse+convert if string) or a string representation like "3s".
Time(path, layout string) time.Time	Parses the string value of the the given key path with the given layout format and returns time.Time. If the key path is numeric, treats it as a UNIX timestamp and returns its time.Time.
String(path string) string
Strings(path string) []string
StringMap(path string) map[string]string
StringsMap(path string) map[string][]string
Byte(path string) []byte
Bool(path string) bool
Bools(path string) []bool
BoolMap(path string) map[string]bool
MapKeys(path string) []string	Returns the list of keys in any map
Slices(path string) []Koanf	Returns []map[string]interface{}, a slice of confmaps loaded into a slice of new Koanf instances.

Alternative to viper

koanf is a lightweight alternative to the popular spf13/viper. It was written as a result of multiple stumbling blocks encountered with some of viper's fundamental flaws.

• viper breaks JSON, YAML, TOML, HCL language specs by forcibly lowercasing keys.
• Significantly bloats build sizes.
• Tightly couples config parsing with file extensions.
• Has poor semantics and abstractions. Commandline, env, file etc. and various parses are hardcoded in the core. There are no primitives that can be extended.
• Pulls a large number of third party dependencies into the core package. For instance, even if you do not use YAML or flags, the dependencies are still pulled as a result of the coupling.
• Imposes arbitrary ordering conventions (eg: flag -> env -> config etc.)
• Get() returns references to slices and maps. Mutations made outside change the underlying values inside the conf map.
• Does non-idiomatic things such as throwing away O(1) on flat maps.
• Viper treats keys that contain an empty map (eg: my_key: {}) as if they were not set (ie: IsSet("my_key") == false).
• There are a large number of open issues.

Documentation

Index

• type Conf
• type KeyMap
• type Koanf
  • func New(delim string) *Koanf
  • func NewWithConf(conf Conf) *Koanf
  • func (ko *Koanf) All() map[string]interface{}
  • func (ko *Koanf) Bool(path string) bool
  • func (ko *Koanf) BoolMap(path string) map[string]bool
  • func (ko *Koanf) Bools(path string) []bool
  • func (ko *Koanf) Bytes(path string) []byte
  • func (ko *Koanf) Copy() *Koanf
  • func (ko *Koanf) Cut(path string) *Koanf
  • func (ko *Koanf) Delete(path string)
  • func (ko *Koanf) Delim() string
  • func (ko *Koanf) Duration(path string) time.Duration
  • func (ko *Koanf) Exists(path string) bool
  • func (ko *Koanf) Float64(path string) float64
  • func (ko *Koanf) Float64Map(path string) map[string]float64
  • func (ko *Koanf) Float64s(path string) []float64
  • func (ko *Koanf) Get(path string) interface{}
  • func (ko *Koanf) Int(path string) int
  • func (ko *Koanf) Int64(path string) int64
  • func (ko *Koanf) Int64Map(path string) map[string]int64
  • func (ko *Koanf) Int64s(path string) []int64
  • func (ko *Koanf) IntMap(path string) map[string]int
  • func (ko *Koanf) Ints(path string) []int
  • func (ko *Koanf) KeyMap() KeyMap
  • func (ko *Koanf) Keys() []string
  • func (ko *Koanf) Load(p Provider, pa Parser, opts ...Option) error
  • func (ko *Koanf) MapKeys(path string) []string
  • func (ko *Koanf) Marshal(p Parser) ([]byte, error)
  • func (ko *Koanf) Merge(in *Koanf) error
  • func (ko *Koanf) MergeAt(in *Koanf, path string) error
  • func (ko *Koanf) MustBoolMap(path string) map[string]bool
  • func (ko *Koanf) MustBools(path string) []bool
  • func (ko *Koanf) MustBytes(path string) []byte
  • func (ko *Koanf) MustDuration(path string) time.Duration
  • func (ko *Koanf) MustFloat64(path string) float64
  • func (ko *Koanf) MustFloat64Map(path string) map[string]float64
  • func (ko *Koanf) MustFloat64s(path string) []float64
  • func (ko *Koanf) MustInt(path string) int
  • func (ko *Koanf) MustInt64(path string) int64
  • func (ko *Koanf) MustInt64Map(path string) map[string]int64
  • func (ko *Koanf) MustInt64s(path string) []int64
  • func (ko *Koanf) MustIntMap(path string) map[string]int
  • func (ko *Koanf) MustInts(path string) []int
  • func (ko *Koanf) MustString(path string) string
  • func (ko *Koanf) MustStringMap(path string) map[string]string
  • func (ko *Koanf) MustStrings(path string) []string
  • func (ko *Koanf) MustStringsMap(path string) map[string][]string
  • func (ko *Koanf) MustTime(path, layout string) time.Time
  • func (ko *Koanf) Print()
  • func (ko *Koanf) Raw() map[string]interface{}
  • func (ko *Koanf) Set(key string, val interface{}) error
  • func (ko *Koanf) Slices(path string) []*Koanf
  • func (ko *Koanf) Sprint() string
  • func (ko *Koanf) String(path string) string
  • func (ko *Koanf) StringMap(path string) map[string]string
  • func (ko *Koanf) Strings(path string) []string
  • func (ko *Koanf) StringsMap(path string) map[string][]string
  • func (ko *Koanf) Time(path, layout string) time.Time
  • func (ko *Koanf) Unmarshal(path string, o interface{}) error
  • func (ko *Koanf) UnmarshalWithConf(path string, o interface{}, c UnmarshalConf) error
• type Option
  • func WithMergeFunc(merge func(src, dest map[string]interface{}) error) Option
• type Parser
• type Provider
• type UnmarshalConf

Types

type Conf

type Conf struct {
	// Delim is the delimiter to use
	// when specifying config key paths, for instance a . for `parent.child.key`
	// or a / for `parent/child/key`.
	Delim string

	// StrictMerge makes the merging behavior strict.
	// Meaning when loading two files that have the same key,
	// the first loaded file will define the desired type, and if the second file loads
	// a different type will cause an error.
	StrictMerge bool
}

Conf is the Koanf configuration.

type KeyMap

type KeyMap map[string][]string

KeyMap represents a map of flattened delimited keys and the non-delimited parts as their slices. For nested keys, the map holds all levels of path combinations. For example, the nested structure `parent -> child -> key` will produce the map: parent.child.key => [parent, child, key] parent.child => [parent, child] parent => [parent]

type Koanf

type Koanf struct {
	// contains filtered or unexported fields
}

Koanf is the configuration apparatus.

func New

func New(delim string) *Koanf

New returns a new instance of Koanf. delim is the delimiter to use when specifying config key paths, for instance a . for `parent.child.key` or a / for `parent/child/key`.

func NewWithConf

func NewWithConf(conf Conf) *Koanf

NewWithConf returns a new instance of Koanf based on the Conf.

func (*Koanf) All

func (ko *Koanf) All() map[string]interface{}

All returns a map of all flattened key paths and their values. Note that it uses maps.Copy to create a copy that uses json.Marshal which changes the numeric types to float64.

func (*Koanf) Bool

func (ko *Koanf) Bool(path string) bool

Bool returns the bool value of a given key path or false if the path does not exist or if the value is not a valid bool representation. Accepted string representations of bool are the ones supported by strconv.ParseBool.

func (*Koanf) BoolMap

func (ko *Koanf) BoolMap(path string) map[string]bool

BoolMap returns the map[string]bool value of a given key path or an empty map[string]bool if the path does not exist or if the value is not a valid bool map.

func (*Koanf) Bools

func (ko *Koanf) Bools(path string) []bool

Bools returns the []bool slice value of a given key path or an empty []bool slice if the path does not exist or if the value is not a valid bool slice.

func (*Koanf) Bytes

func (ko *Koanf) Bytes(path string) []byte

Bytes returns the []byte value of a given key path or an empty []byte slice if the path does not exist or if the value is not a valid string.

func (*Koanf) Copy

func (ko *Koanf) Copy() *Koanf

Copy returns a copy of the Koanf instance.

func (*Koanf) Cut

func (ko *Koanf) Cut(path string) *Koanf

Cut cuts the config map at a given key path into a sub map and returns a new Koanf instance with the cut config map loaded. For instance, if the loaded config has a path that looks like parent.child.sub.a.b, `Cut("parent.child")` returns a new Koanf instance with the config map `sub.a.b` where everything above `parent.child` are cut out.

func (*Koanf) Delete

func (ko *Koanf) Delete(path string)

Delete removes all nested values from a given path. Clears all keys/values if no path is specified. Every empty, key on the path, is recursively deleted.

func (*Koanf) Delim

func (ko *Koanf) Delim() string

Delim returns delimiter in used by this instance of Koanf.

func (*Koanf) Duration

func (ko *Koanf) Duration(path string) time.Duration

Duration returns the time.Duration value of a given key path assuming that the key contains a valid numeric value.

func (*Koanf) Exists

func (ko *Koanf) Exists(path string) bool

Exists returns true if the given key path exists in the conf map.

func (*Koanf) Float64

func (ko *Koanf) Float64(path string) float64

Float64 returns the float64 value of a given key path or 0 if the path does not exist or if the value is not a valid float64.

func (*Koanf) Float64Map

func (ko *Koanf) Float64Map(path string) map[string]float64

Float64Map returns the map[string]float64 value of a given key path or an empty map[string]float64 if the path does not exist or if the value is not a valid float64 map.

func (*Koanf) Float64s

func (ko *Koanf) Float64s(path string) []float64

Float64s returns the []float64 slice value of a given key path or an empty []float64 slice if the path does not exist or if the value is not a valid float64 slice.

func (*Koanf) Get

func (ko *Koanf) Get(path string) interface{}

Get returns the raw, uncast interface{} value of a given key path in the config map. If the key path does not exist, nil is returned.

func (*Koanf) Int

func (ko *Koanf) Int(path string) int

Int returns the int value of a given key path or 0 if the path does not exist or if the value is not a valid int.

func (*Koanf) Int64

func (ko *Koanf) Int64(path string) int64

Int64 returns the int64 value of a given key path or 0 if the path does not exist or if the value is not a valid int64.

func (*Koanf) Int64Map

func (ko *Koanf) Int64Map(path string) map[string]int64

Int64Map returns the map[string]int64 value of a given key path or an empty map[string]int64 if the path does not exist or if the value is not a valid int64 map.

func (*Koanf) Int64s

func (ko *Koanf) Int64s(path string) []int64

Int64s returns the []int64 slice value of a given key path or an empty []int64 slice if the path does not exist or if the value is not a valid int slice.

func (*Koanf) IntMap

func (ko *Koanf) IntMap(path string) map[string]int

IntMap returns the map[string]int value of a given key path or an empty map[string]int if the path does not exist or if the value is not a valid int map.

func (*Koanf) Ints

func (ko *Koanf) Ints(path string) []int

Ints returns the []int slice value of a given key path or an empty []int slice if the path does not exist or if the value is not a valid int slice.

func (*Koanf) KeyMap

func (ko *Koanf) KeyMap() KeyMap

KeyMap returns a map of flattened keys and the individual parts of the key as slices. eg: "parent.child.key" => ["parent", "child", "key"]

func (*Koanf) Keys

func (ko *Koanf) Keys() []string

Keys returns the slice of all flattened keys in the loaded configuration sorted alphabetically.

func (*Koanf) Load

func (ko *Koanf) Load(p Provider, pa Parser, opts ...Option) error

Load takes a Provider that either provides a parsed config map[string]interface{} in which case pa (Parser) can be nil, or raw bytes to be parsed, where a Parser can be provided to parse. Additionally, options can be passed which modify the load behavior, such as passing a custom merge function.

func (*Koanf) MapKeys

func (ko *Koanf) MapKeys(path string) []string

MapKeys returns a sorted string list of keys in a map addressed by the given path. If the path is not a map, an empty string slice is returned.

func (*Koanf) Marshal

func (ko *Koanf) Marshal(p Parser) ([]byte, error)

Marshal takes a Parser implementation and marshals the config map into bytes, for example, to TOML or JSON bytes.

func (*Koanf) Merge

func (ko *Koanf) Merge(in *Koanf) error

Merge merges the config map of a given Koanf instance into the current instance.

func (*Koanf) MergeAt

func (ko *Koanf) MergeAt(in *Koanf, path string) error

MergeAt merges the config map of a given Koanf instance into the current instance as a sub map, at the given key path. If all or part of the key path is missing, it will be created. If the key path is `""`, this is equivalent to Merge.

func (*Koanf) MustBoolMap

func (ko *Koanf) MustBoolMap(path string) map[string]bool

MustBoolMap returns the map[string]bool value of a given key path or panics if the value is not set or set to default value.

func (*Koanf) MustBools

func (ko *Koanf) MustBools(path string) []bool

MustBools returns the []bool value of a given key path or panics if the value is not set or set to default value.

func (*Koanf) MustBytes

func (ko *Koanf) MustBytes(path string) []byte

MustBytes returns the []byte value of a given key path or panics if the value is not set or set to default value.

func (*Koanf) MustDuration

func (ko *Koanf) MustDuration(path string) time.Duration

MustDuration returns the time.Duration value of a given key path or panics if its not set or set to default value 0.

func (*Koanf) MustFloat64

func (ko *Koanf) MustFloat64(path string) float64

MustFloat64 returns the float64 value of a given key path or panics or panics if its not set or set to default value 0.

func (*Koanf) MustFloat64Map

func (ko *Koanf) MustFloat64Map(path string) map[string]float64

MustFloat64Map returns the map[string]float64 value of a given key path or panics if the value is not set or set to default value.

func (*Koanf) MustFloat64s

func (ko *Koanf) MustFloat64s(path string) []float64

MustFloat64s returns the []Float64 slice value of a given key path or panics if the value is not set or set to default value.

func (*Koanf) MustInt

func (ko *Koanf) MustInt(path string) int

MustInt returns the int value of a given key path or panics or panics if its not set or set to default value of 0.

func (*Koanf) MustInt64

func (ko *Koanf) MustInt64(path string) int64

MustInt64 returns the int64 value of a given key path or panics if the value is not set or set to default value of 0.

func (*Koanf) MustInt64Map

func (ko *Koanf) MustInt64Map(path string) map[string]int64

MustInt64Map returns the map[string]int64 value of a given key path or panics if its not set or set to default value.

func (*Koanf) MustInt64s

func (ko *Koanf) MustInt64s(path string) []int64

MustInt64s returns the []int64 slice value of a given key path or panics if the value is not set or its default value.

func (*Koanf) MustIntMap

func (ko *Koanf) MustIntMap(path string) map[string]int

MustIntMap returns the map[string]int value of a given key path or panics if the value is not set or set to default value.

func (*Koanf) MustInts

func (ko *Koanf) MustInts(path string) []int

MustInts returns the []int slice value of a given key path or panics if the value is not set or set to default value.

func (*Koanf) MustString

func (ko *Koanf) MustString(path string) string

MustString returns the string value of a given key path or panics if its not set or set to default value "".

func (*Koanf) MustStringMap

func (ko *Koanf) MustStringMap(path string) map[string]string

MustStringMap returns the map[string]string value of a given key path or panics if the value is not set or set to default value.

func (*Koanf) MustStrings

func (ko *Koanf) MustStrings(path string) []string

MustStrings returns the []string slice value of a given key path or panics if the value is not set or set to default value.

func (*Koanf) MustStringsMap

func (ko *Koanf) MustStringsMap(path string) map[string][]string

MustStringsMap returns the map[string][]string value of a given key path or panics if the value is not set or set to default value.

func (*Koanf) MustTime

func (ko *Koanf) MustTime(path, layout string) time.Time

MustTime attempts to parse the value of a given key path and return time.Time representation. If the value is numeric, it is treated as a UNIX timestamp and if it's string, a parse is attempted with the given layout. It panics if the parsed time is zero.

func (*Koanf) Print

func (ko *Koanf) Print()

Print prints a key -> value string representation of the config map with keys sorted alphabetically.

func (*Koanf) Raw

func (ko *Koanf) Raw() map[string]interface{}

Raw returns a copy of the full raw conf map. Note that it uses maps.Copy to create a copy that uses json.Marshal which changes the numeric types to float64.

func (*Koanf) Set

func (ko *Koanf) Set(key string, val interface{}) error

Set sets the value at a specific key.

func (*Koanf) Slices

func (ko *Koanf) Slices(path string) []*Koanf

Slices returns a list of Koanf instances constructed out of a []map[string]interface{} interface at the given path.

func (*Koanf) Sprint

func (ko *Koanf) Sprint() string

Sprint returns a key -> value string representation of the config map with keys sorted alphabetically.

func (*Koanf) String

func (ko *Koanf) String(path string) string

String returns the string value of a given key path or "" if the path does not exist or if the value is not a valid string.

func (*Koanf) StringMap

func (ko *Koanf) StringMap(path string) map[string]string

StringMap returns the map[string]string value of a given key path or an empty map[string]string if the path does not exist or if the value is not a valid string map.

func (*Koanf) Strings

func (ko *Koanf) Strings(path string) []string

Strings returns the []string slice value of a given key path or an empty []string slice if the path does not exist or if the value is not a valid string slice.

func (*Koanf) StringsMap

func (ko *Koanf) StringsMap(path string) map[string][]string

StringsMap returns the map[string][]string value of a given key path or an empty map[string][]string if the path does not exist or if the value is not a valid strings map.

func (*Koanf) Time

func (ko *Koanf) Time(path, layout string) time.Time

Time attempts to parse the value of a given key path and return time.Time representation. If the value is numeric, it is treated as a UNIX timestamp and if it's string, a parse is attempted with the given layout.

func (*Koanf) Unmarshal

func (ko *Koanf) Unmarshal(path string, o interface{}) error

Unmarshal unmarshals a given key path into the given struct using the mapstructure lib. If no path is specified, the whole map is unmarshalled. `koanf` is the struct field tag used to match field names. To customize, use UnmarshalWithConf(). It uses the mitchellh/mapstructure package.

func (*Koanf) UnmarshalWithConf

func (ko *Koanf) UnmarshalWithConf(path string, o interface{}, c UnmarshalConf) error

UnmarshalWithConf is like Unmarshal but takes configuration params in UnmarshalConf. See mitchellh/mapstructure's DecoderConfig for advanced customization of the unmarshal behaviour.

type Option

type Option func(*options)

Option is a generic type used to modify the behavior of Koanf.Load.

func WithMergeFunc

func WithMergeFunc(merge func(src, dest map[string]interface{}) error) Option

WithMergeFunc is an option to modify the merge behavior of Koanf.Load. If unset, the default merge function is used.

The merge function is expected to merge map src into dest (left to right).

type Parser

type Parser interface {
	Unmarshal([]byte) (map[string]interface{}, error)
	Marshal(map[string]interface{}) ([]byte, error)
}

Parser represents a configuration format parser.

type Provider

type Provider interface {
	// Read returns the entire configuration as raw []bytes to be parsed.
	// with a Parser.
	ReadBytes() ([]byte, error)

	// Read returns the parsed configuration as a nested map[string]interface{}.
	// It is important to note that the string keys should not be flat delimited
	// keys like `parent.child.key`, but nested like `{parent: {child: {key: 1}}}`.
	Read() (map[string]interface{}, error)
}

Provider represents a configuration provider. Providers can read configuration from a source (file, HTTP etc.)

type UnmarshalConf

type UnmarshalConf struct {
	// Tag is the struct field tag to unmarshal.
	// `koanf` is used if left empty.
	Tag string

	// If this is set to true, instead of unmarshalling nested structures
	// based on the key path, keys are taken literally to unmarshal into
	// a flat struct. For example:
	// “`
	// type MyStuff struct {
	// 	Child1Name string `koanf:"parent1.child1.name"`
	// 	Child2Name string `koanf:"parent2.child2.name"`
	// 	Type       string `koanf:"json"`
	// }
	// “`
	FlatPaths     bool
	DecoderConfig *mapstructure.DecoderConfig
}

UnmarshalConf represents configuration options used by Unmarshal() to unmarshal conf maps into arbitrary structs.