qcl

README

QCL: Quick Config Loader

GO 1.18+ ONLY This library makes use of generics, which are only available in Go 1.18+

qcl is a lightweight library for loading configuration values at runtime. It is designed to have a simple API, robust test suite, zero external dependencies, and be easy to integrate into existing projects. If you are looking for a more full-featured configuration library, check out Viper or Koanf. I've used both and they are great libraries, but I wanted something simpler for my use cases. I currently have no plans to support loading configuration from files, so if your use-case requires that, this library is not for you.

BE ADVISED This library is still under active development, but the API is stable and will not change before 1.0.0. The test suite is pretty extensive, but I'm sure there are still edge cases I haven't thought of. If you find a bug, please open an issue.

Installation

go get github.com/thezmc/qcl

Simple Example

Make sure the environment variables you want to use are set. For example:

export HOST="localhost"
export PORT="8080"

You can also use command line arguments. For example:

go run main.go --port 8081

type Config struct {
  Host string
  Port int
  SSL  bool
}

defaultConfig := Config{
  Host: "anotherhost",
  Port: 9090,
  SSL:  false,
}

conf, _ := qcl.Load(&defaultConfig)

fmt.Printf("Host: %s\n", conf.Host) // "Host: localhost" from environment
fmt.Printf("Port: %d\n", conf.Port) // "Port: 8081" from command line, overrides environment by default
fmt.Printf("SSL: %t\n", conf.SSL)   // "SSL: false" from the "defaultConfig" struct

Default Behavior

By default, the library will use the field names as the environment variable / command-line argument names. The environment variables will be loaded first, followed by the command line arguments. If a value is found in both the environment and command line arguments, the command line argument will take precedence except in the case of slice and map values.

Environment Variables

By default, the library will look for environment variables with the same name as the struct fields, but split along word boundaries:

type Config struct {
  Host   string // "HOST" environment variable
  DBHost string // "DB_HOST" environment variable
  DBPort int    // "DB_PORT" environment variable
}

You can override the environment variable name by using the env tag:

type Config struct {
  Host     string `env:"HoSt"` // "HOST" environment variable
  HTTPPort int    `env:"PORT"` // "PORT" environment variable
}

NOTE: The override is case-insensitive. The library will convert the tag value to uppercase before looking for the environment variable.

Command Line Arguments

By default, the library will look for command line arguments with the same name as the struct fields, but split along word boundaries:

type Config struct {
  Host   string // "--host" command line argument
  DBHost string // "--db-host" command line argument
  DBPort int    // "--db-port" command line argument
}

You can override the command line argument name by using the flag tag:

type Config struct {
  Host     string                // "--host" command line argument
  HTTPPort int    `flag:"port"`  // "--port" command line argument
}

Slice and Map Values

Slices and maps are special cases when it comes to overrides. If a slice or map value is found in the environment or command-line, it will be appended to the slice or map from the default config. For example:

export HOSTS="localhost,otherhost" # separate iterable values with a comma
go run main.go --hosts "yetanotherhost"

type Config struct {
  Hosts []string
}

conf, _ := qcl.Load(&Config{})

fmt.Printf("Hosts: %s\n", conf.Hosts) // "Hosts: [localhost otherhost yetanotherhost]"

Same idea for maps:

export HOSTS="localhost=8080,otherhost=9090" # separate key-value pairs with a comma, separate keys and values with an equals sign
go run main.go --hosts "yetanotherhost=1234"

type Config struct {
  Hosts map[string]int
}

conf := qcl.Load(&Config{})
fmt.Printf("Hosts: %s\n", conf.Hosts) // "Hosts: map[localhost:8080 otherhost:9090 yetanotherhost:1234]"

Nested Structs

Nested structs are also supported. The field name for the nested struct will be used as the prefix for the environment variables and command-line arguments. For example:

type Config struct {
  Host string   // "HOST" environment variable; "--host" command line argument
  DB   struct {
    Host string // "DB_HOST" environment variable; "--db-host" command line argument
    Port int    // "DB_PORT" environment variable; "--db-port" command line argument
  }
}

Embedded Structs

Embedded structs are also supported. The embedded struct will be flattened into the parent struct and so will not have a prefix. For example:

type Config struct {
  User     string // "USER" environment variable; "--user" command line argument
  struct {
    Host string // "HOST" environment variable; "--host" command line argument
    Port int    // "PORT" environment variable; "--port" command line argument
  }
}
// is treated the same as
type Config struct {
  User string // "USER" environment variable; "--user" command line argument
  Host string // "HOST" environment variable; "--host" command line argument
  Port int    // "PORT" environment variable; "--port" command line argument
}

Advanced Usage

Custom Environment Variable Prefix

By default, the library doesn't expect any prefix on your environment variables. You can set a custom environment variable prefix by using the qcl.WithEnvPrefix functional option:

type Config struct {
  Host string // "MYAPP_HOST" environment variable
}

qcl.Load(&Config{}, qcl.UseEnv(qcl.WithEnvPrefix("MYAPP_"))) // the _ on the end is optional. It will be added automatically if not included.

Custom Environment Variable Struct Tag

By default, the env struct tag is used to override environment variable names. You can set a custom environment variable struct tag by using the qcl.WithEnvStructTag functional option:

type Config struct {
  HTTPHost string `envvar:"HOST"` // "HOST" environment variable
}

qcl.Load(&Config{}, qcl.UseEnv(qcl.WithEnvStructTag("envvar")))

NOTE: The override is case-insensitive. The library will convert the tag value to uppercase before looking for the environment variable.

Custom Environment Variable Iterable Separator

By default, iterables are separated by a comma. You can set a custom environment variable iterable separator by using the qcl.WithEnvSeparator functional option:

export HOSTS="localhost|otherhost" # separate iterable values with a pipe

type Config struct {
  Hosts []string // "HOSTS" environment variable
}

qcl.Load(&Config{}, qcl.UseEnv(qcl.WithEnvSeparator("|")))

Extending the Library

Custom Loaders

You can create your own loaders

func UseJSON(path string) LoadOption {
	return func(lc *qcl.LoadConfig) { // Make sure the function you return implements the qcl.LoadOption interface

		// add your source to the list of sources. Be aware of the order since the sources are loaded in the order they are added.
		lc.Sources = append(lc.Sources, "json")
		
		// add your loader to the Loaders map. Be careful not to override any existing loaders: "env" and "flags" are already taken.
		lc.Loaders["json"] = func(config any) error {
			// do your thing...
		}
	}
}

NOTE: The order of the sources is important. The library will load the values from the sources in the order they are defined. If a value is found in multiple sources, the value from the last configured source will be used.

License

MIT

Contributing

Bug reports and pull requests are welcome at the issues page. For major changes, please open an issue first to discuss what you would like to change. Any new feature PRs must include adequate test coverage and documentation. Any features importing packages that aren't in the standard library will not be accepted.

Author

Zach Callahan

Documentation

Index

• Variables
• func Load[T any](defaultConfig *T, opts ...LoadOption) (*T, error)
• func WithEnvPrefix(prefix string) envOption
• func WithEnvSeparator(separator string) envOption
• func WithEnvStructTag(tag string) envOption
• type InvalidMapValueError
  • func (e InvalidMapValueError) Error() string
• type LoadConfig
• type LoadOption
  • func UseEnv(opts ...envOption) LoadOption
  • func UseFlags() LoadOption
• type Loader
• type UnsupportedTypeError
  • func (e UnsupportedTypeError) Error() string

Variables

var (
	NotAMapError    = errors.New("not a map")
	NotASliceError  = errors.New("not a slice")
	ConfigTypeError = errors.New("config must be a pointer to a struct")
)

var DefaultLoadOptions = []LoadOption{
	UseEnv(),
	UseFlags(),
}

DefaultLoadOptions is the default LoadOptions used by the Load function if no LoadOptions are passed into it.

Functions

func Load

func Load[T any](defaultConfig *T, opts ...LoadOption) (*T, error)

Load modifies the pointer it receives with configuration information from the sources specified in the LoadOptions. The Load function are passed to the Load function. The default LoadOptions are:

 DefaultLoadOptions := []LoadOption{
	  qcl.UseEnv(qcl.WithPrefix(""), qcl.WithEnvSeparator(","), qcl.WithEnvStructTag("env")),
	  qcl.UseFlags()
 }

If no LoadOptions are passed to the Load function, the default LoadOptions will be used.

Example:

qcl.Load(&defaultConfig)

is equivalent to:

qcl.Load(&defaultConfig, qcl.DefaultLoadOptions...)

If any LoadOption is passed to the Load function, the default LoadOptions will not be used. The Load function returns a pointer to the configuration struct, and an error.

func WithEnvPrefix

func WithEnvPrefix(prefix string) envOption

WithEnvPrefix allows you to specify a prefix for environment variables. For example, if you specify "FOO" as the prefix, then an environment variable named "FOO_BAR" will be used to set a field named "Bar" in the config struct.

Example:

type Config struct {
	Bar string
}

WithEnvPrefix("FOO")

will set the value of Bar to the value of the environment variable "FOO_BAR".

The default is no prefix.

func WithEnvSeparator

func WithEnvSeparator(separator string) envOption

WithEnvSeparator allows you to specify a custom separator for environment variables that are setting iterables.

Example:

WithEnvSeparator(";")

will allow an environment variable like

export FOO="bar;baz"

to set a field named "Foo" to a slice of strings with the values "bar" and "baz":

type Config struct {
	Foo []string // foo will be set to []string{"bar", "baz"}
}

This also works for maps where the key/value pairs are separated by the separator.

Example:

export FOO="bar=baz;qux=quux"

will set a field named "Foo" to a map with the key/value pairs bar=baz, and qux=quux.

type Config struct {
	Foo map[string]string // foo will be set to map[string]string{"bar": "baz", "qux": "quux"}
}

The default separator is a comma (,)

func WithEnvStructTag

func WithEnvStructTag(tag string) envOption

WithEnvStructTag allows you to specify a custom struct tag to use for environment variable names. By default, the loader looks for the "env" struct tag, but if that's not found the field name itself is used as the environment variable name and in either case, it is split on word boundaries. For example, a field named "FooBar" will be set by the environment variable "FOO_BAR" by default.

Example:

WithEnvStructTag("mytag")

type Config struct {
	FooBar string `mytag:"FOO"` // FooBar will be set by the environment variable "FOO" instead of the default "FOO_BAR"
}

By default, the environment loader looks for a struct tag "env" and in the absence of a struct tag, will use the field name itself.

Types

type InvalidMapValueError

type InvalidMapValueError struct {
	// contains filtered or unexported fields
}

InvalidMapValueError is returned when the number of keys and values in a map do not match.

func (InvalidMapValueError) Error

func (e InvalidMapValueError) Error() string

type LoadConfig

type LoadConfig struct {
	Sources []string          // Sources is a slice of the configuration sources.
	Loaders map[string]Loader // Loaders is a map of the configuration sources and their corresponding loaders.
}

LoadConfig is the configuration struct for the Load function. It contains the configuration sources and the loaders for those sources. Since maps in go are not ordered, the order of the sources is kept in a separate slice. The Load function will iterate over the sources in the Sources slice and call the corresponding loader in the Loaders map.

type LoadOption

type LoadOption func(*LoadConfig) // LoadOption is a function that configures the Load function's LoadConfig. The Load function accepts a variable number of LoadOptions.

func UseEnv

func UseEnv(opts ...envOption) LoadOption

UseEnv allows you to load configuration from environment variables. The environment variables are expected to be in all caps and separated by underscores. For example, a field named "FooBar" will be set by the environment variable "FOO_BAR".

Example:

export FOO_BAR=baz

type Config struct {
	FooBar string
}

var defaultConfig Config

ocl.Load(defaultConfig, ocl.UseEnv())

will set the value of FooBar to the value of the environment variable "FOO_BAR".

func UseFlags

func UseFlags() LoadOption

UseFlags enables configuration from command line flags. Currently, the flag loader is not configurable. It will use the struct field names as the flag names, but lowercased and spit on word boundaries with a dash. For example, the field name "FooBar" will be converted to "foo-bar". You can override the flag name by using the "flag" struct tag. Examples:

type Config struct {
    FooBar string // will look for -foo-bar flag
}
type Config struct {
    FooBar string `flag:"foo"` // will look for -foo flag
}
type Config struct {
    FooBar string `flag:"foo.bar"` // will look for -foo.bar flag
}

By default, calling Load() without any LoadOptions will use the flag loader as well as the environment loader, with the flag loader taking precedence. If you want to use only the flag loader, you can call Load with just the UseFlags option:

Load(&config, UseFlags()) // will only use flags

type Loader

type Loader func(any) error

A Loader is a function that loads the configuration from a specific source.

type UnsupportedTypeError

type UnsupportedTypeError struct {
	// contains filtered or unexported fields
}

UnsupportedTypeError is returned when a field is not a supported type.

func (UnsupportedTypeError) Error

func (e UnsupportedTypeError) Error() string