Documentation
¶
Overview ¶
Package confmaker builds typed configs from the environment. A library declares its config as a struct with env tags; an application loads it, and the library receives a ready value without reading the environment itself.
cfg, err := confmaker.Load[store.Config]()
Load reads one config. A Loader reads several together and reports the problems of all of them in one error. The Fx adapter is the separate module github.com/uchaloop/confmaker/confx.
Overview ¶
A config is a struct with env tags and, optionally, three methods:
type Config struct {
Host string `env:"HOST,notEmpty"`
Timeout time.Duration `env:"TIMEOUT"`
Password secret.Secret `env:"PASSWORD"`
}
func (Config) ConfigName() string { return "store" }
func (c *Config) SetDefaults() { c.Timeout = 30 * time.Second }
func (c Config) Validate() error { ... }
It reads STORE_HOST, STORE_TIMEOUT and STORE_PASSWORD. The tags are plain strings, so the type depends on github.com/uchaloop/secret/v2 for the secret field and not on this package.
Declaring configurations ¶
A field that names a variable in its env tag is read from that variable. A struct field without an env tag is not read itself: its fields are, and those with env tags become variables of the config, under the prefix extended by the field's envPrefix tag, if any. Any other field without an env tag is not configuration. env:"-" excludes a field and everything nested in it.
type Config struct {
Host string `env:"HOST"`
Pool PoolConfig `envPrefix:"POOL_"`
}
STORE_HOST
STORE_POOL_MAX_CONNS
A nested config is nested by value: the variables a config reads are known from its type, which a pointer or a collection could not promise. envPrefix is written into names as it stands, with or without a trailing underscore.
A field may be a string, a bool, any sized integer or float, a time.Duration, a type implementing encoding.TextUnmarshaler, a pointer to one of those, or a slice or map of them. A TextUnmarshaler decodes its own text and reports its own errors; for a manifest or dump it also needs encoding.TextMarshaler. complex, uintptr and []byte are refused. A slice splits on envSeparator ("," by default). A map splits entries the same way and each key from its value on envKeyValSeparator (":" by default). The separator tags apply only to a slice or map read in this plain syntax, not to a scalar, a type with its own text form or a JSON field:
STORE_BROKERS=a:9092,b:9092 STORE_LABELS=env:prod,team:core
A duplicate map key is an error, and a key or value padded with whitespace is reported rather than trimmed.
A set variable replaces its field. An empty variable is read like any other text: a string becomes "", a slice or map becomes empty, a TextUnmarshaler receives empty text, a number, bool or duration fails to parse, and a pointer reads its element the same way.
Each field's variable is read under these tag options:
- required: the variable must be set; an empty value is allowed;
- notEmpty: the variable must be set and its text must be non-empty. It checks the text, not the result: JSON [] is non-empty text for an empty slice. Limits on what a value holds belong in Validate.
A default does not satisfy either option: they concern what the deployment supplies.
Declarations are checked when a config is registered, before any default or variable is read, and a mistake is reported by Loader.Load or Manifest: envDefault (defaults belong in SetDefaults), an unknown option or options without a name, two fields on one variable, envPrefix on anything but a struct nested by value, a secret in a slice, array or map, a config nested through a pointer or collection, an unreadable type, an empty separator or one on a field that is not split, and a variable name no environment can hold (with = or NUL). Secrets are recognised behind any number of pointers.
Names and prefixes ¶
Every config has an instance name. It gives the default prefix - "read-replica" reads READ_REPLICA_* - and labels the config in errors. It comes from WithName or, without it, from the config's ConfigName method (ConfigNamer):
loader.Add[Config]() // "store", STORE_*
loader.Add[Config](confmaker.WithName("replica")) // "replica", REPLICA_*
loader.Add[Config](confmaker.WithPrefix("MAIN_DB_")) // "store", MAIN_DB_*
Names hold lowercase letters, digits and _ - ., and do not start or end with a separator. A config with neither WithName nor ConfigName cannot be registered. Two registrations may not share a name, a prefix, or a variable.
Options come in two kinds: a ConfigOption configures one config, an EnvOption configures a whole load. Passing one where the other is expected does not compile. Load takes both.
Defaults and validation ¶
Loading a config runs these steps:
- SetDefaults, if the config has it, sets defaults in code.
- Each variable that is set replaces its field; an unset variable leaves the default.
- Validate, if the config has it, checks the result. It runs only when every variable of the config parsed.
SetDefaults must be deterministic and free of side effects: loading calls it once per config, and every Manifest call calls it again. Only the top-level config's methods are called.
JSON values ¶
envFormat:"json" reads one variable as JSON, with encoding/json/v2, into a struct, slice, array or map, or a pointer to one, nested to any depth:
type Config struct {
Endpoints []Endpoint `env:"ENDPOINTS" envFormat:"json"`
}
STORE_ENDPOINTS=[{"url":"http://a:9000","timeout":"30s"}]
The value is decoded into the field's type:
- A set variable replaces the whole field; nothing is merged with the default. {"url":"http://a"} leaves timeout zero even when the default had one.
- Member names match exactly, and unknown or duplicate members are errors, unless a field's JSON tag says otherwise: case:ignore matches a name in any case, and an embed map collects members no field names, whatever they are.
- Only json:"-" ignores a field; json:"-," and json:"-,omitempty" name it "-".
- null clears a pointer, slice or map and is an error elsewhere. [] and {} are empty collections. An empty variable is an error.
- A time.Duration is a string such as "30s", in collections too.
- A type with UnmarshalJSON or UnmarshalText reads its own form, and is as strict as it decides to be.
Refused at registration: envFormat on a scalar, an unknown format, envFormat with a separator tag, interface values, []byte and byte arrays, map keys other than strings, integers or text types, any secret type inside the value, ignored fields included (such a secret is refused, not masked), and a struct json/v2 rejects as an object - two fields on one JSON name, for example. The last check decodes {} into each struct type, which runs no method of the config.
An error names the variable and the place inside the value:
config "store": variable "STORE_ENDPOINTS": JSON at "/0/timeout": time: invalid duration "soon"
A manifest renders a JSON default with sorted map keys and nil as null. Whether it loads back to an equal value depends on the type: custom marshalers decide their own form, and omitempty drops an empty slice or map, which then loads back as nil. omitzero keeps an empty non-nil collection.
Loading and lifecycle ¶
Loader.Load runs in this order:
- Invalid options and registrations are reported; the other configs still load.
- Conflicting registrations - two on one name, prefix or variable - end loading here, before any SetDefaults runs or anything is printed.
- One snapshot of the environment is taken.
- SetDefaults runs for each config.
- The dump is written, when WithDump asks for one. A dump that fails is reported and does not stop the checks.
- Variables under a registered prefix that no field reads are reported.
- The environment is applied and Validate runs, for each config.
Everything found is joined into one error, one problem per line, ordered by instance name and prefix rather than by the order of Add calls:
unknown configuration variable "STORE_HSOT" (did you mean "STORE_HOST"?) config "store": required variable "STORE_HOST" is not set
Values are handed out only when the whole set loaded; see Handle.Value. Load runs once. A Loader is safe for concurrent use and holds no lock while ConfigName, SetDefaults, Validate, unmarshalers or the dump writer run; those may call Add or Value on the same Loader but must not call its Load.
Environment and testing ¶
Load reads one snapshot of the environment, shared by loading, the check for unknown variables and the dump. By default it is the process environment; WithEnv replaces it with a map, which suits tests:
cfg, err := confmaker.Load[Config](confmaker.WithEnv(map[string]string{
"STORE_HOST": "localhost",
}))
confmaker reads variables only; reading a file into such a map is left to the caller. AllowUnknown exempts prefixes from the check for unknown variables, for an environment shared with other programs.
Manifest, dump and secrets ¶
Manifest and WithDump describe the same variables for different purposes:
- Manifest returns metadata for generating a .env.example, a config map or a table. It does not read the environment, but it calls SetDefaults and renders each default with MarshalText or the field's plain syntax.
- WithDump prints, while loading, each variable's environment value or default and its source. It is an input report, not proof that loading succeeded. Control characters are escaped.
A default that the field's separators could not carry back is an error. Variable.HasDefault reports a non-zero default and cannot tell an explicit false, 0 or "" from no default. Manifest values are ENV text; a generator escapes them for shell or YAML itself.
A field of a secret type (github.com/uchaloop/secret/v2) is never printed: not in the dump, the manifest or a parse error. Secrets are recognised by type only. A password written into a plain string, such as postgres://user:password@host/db, is printed like any other text; keep it in its own secret field.
Index ¶
Examples ¶
Constants ¶
This section is empty.
Variables ¶
var ( // ErrNotLoaded is returned by [Handle.Value] until [Loader.Load] has finished. ErrNotLoaded = errors.New("configuration is not loaded; call Load first") // ErrRegisteredAfterLoad is returned by [Handle.Value] for a config added // with [Loader.Add] once Load had started. ErrRegisteredAfterLoad = errors.New("config registered after Load started; register every config before loading") )
Functions ¶
func Load ¶
func Load[T any](opts ...LoadOption) (T, error)
Load loads one config of type T, as a Loader with a single registration would. It takes both a ConfigOption (WithName, WithPrefix) and an EnvOption (WithEnv, WithDump, AllowUnknown). Unknown variables are looked for under T's own prefix only. On error it returns the zero T.
cfg, err := confmaker.Load[store.Config]()
cfg, err := confmaker.Load[store.Config](confmaker.WithName("replica"), confmaker.WithEnv(vars))
Example ¶
Load reads one config. WithEnv stands in for the process environment here, so the example runs the same everywhere; an application calls Load without it.
package main
import (
"fmt"
"time"
"github.com/uchaloop/confmaker"
)
// StoreConfig is the kind of config a library declares: plain fields, env tags,
// and no knowledge of where the values come from.
type StoreConfig struct {
Host string `env:"HOST,notEmpty"`
Timeout time.Duration `env:"TIMEOUT"`
}
func (StoreConfig) ConfigName() string { return "store" }
func (c *StoreConfig) SetDefaults() { c.Timeout = 30 * time.Second }
func (c StoreConfig) Validate() error {
if c.Timeout <= 0 {
return fmt.Errorf("timeout must be positive, got %s", c.Timeout)
}
return nil
}
func main() {
cfg, err := confmaker.Load[StoreConfig](confmaker.WithEnv(map[string]string{
"STORE_HOST": "db:5432",
}))
if err != nil {
fmt.Println(err)
return
}
fmt.Println(cfg.Host, cfg.Timeout)
}
Output: db:5432 30s
Example (Errors) ¶
One error reports every problem, with a suggestion for a misspelled variable. Validate would refuse the zero timeout, but it does not run while a variable of the config is missing: its report comes once the host is fixed.
package main
import (
"fmt"
"time"
"github.com/uchaloop/confmaker"
)
// StoreConfig is the kind of config a library declares: plain fields, env tags,
// and no knowledge of where the values come from.
type StoreConfig struct {
Host string `env:"HOST,notEmpty"`
Timeout time.Duration `env:"TIMEOUT"`
}
func (StoreConfig) ConfigName() string { return "store" }
func (c *StoreConfig) SetDefaults() { c.Timeout = 30 * time.Second }
func (c StoreConfig) Validate() error {
if c.Timeout <= 0 {
return fmt.Errorf("timeout must be positive, got %s", c.Timeout)
}
return nil
}
func main() {
_, err := confmaker.Load[StoreConfig](confmaker.WithEnv(map[string]string{
"STORE_HSOT": "db:5432",
"STORE_TIMEOUT": "0s",
}))
fmt.Println(err)
}
Output: unknown configuration variable "STORE_HSOT" (did you mean "STORE_HOST"?) config "store": required variable "STORE_HOST" is not set
Types ¶
type ConfigNamer ¶ added in v0.6.2
type ConfigNamer interface {
ConfigName() string
}
ConfigNamer is implemented by a config that names its default instance, for example "postgres". A config implements it without importing this package. ConfigName is called on a zero value, before SetDefaults, with a value or pointer receiver, and must return a constant name. WithName takes precedence.
type ConfigOption ¶ added in v0.6.2
type ConfigOption interface {
LoadOption
// contains filtered or unexported methods
}
ConfigOption configures one config: WithName and WithPrefix. Loader.Add, Manifest and Load take it.
func WithName ¶ added in v0.6.2
func WithName(name string) ConfigOption
WithName sets the instance name, instead of the config's ConfigName. The name gives the default prefix ("read-replica" reads READ_REPLICA_*) and labels the config in errors. It may be given once per config.
func WithPrefix ¶ added in v0.6.2
func WithPrefix(prefix string) ConfigOption
WithPrefix overrides the environment prefix without changing the instance name. It must contain upper-case letters, digits or underscores and end with an underscore, for example "REPLICA_POSTGRES_".
type EnvOption ¶ added in v0.6.2
type EnvOption interface {
LoadOption
// contains filtered or unexported methods
}
EnvOption configures a whole load: WithEnv, WithDump and AllowUnknown. MakeLoader and Load take it.
func AllowUnknown ¶ added in v0.6.2
AllowUnknown exempts variables starting with the given prefixes from the check for unknown variables, for an environment shared with other programs. The prefixes are copied. An empty prefix is ignored: it would match every variable and turn the check off.
func WithDump ¶ added in v0.6.2
WithDump writes a table to w while loading: each variable the configs declare, its type, its environment value or rendered default, and where the value comes from. It is written even when loading fails, except for conflicting registrations, so it is not proof that loading succeeded. Secrets are shown as set or unset, never printed.
Example ¶
WithDump prints each variable, its value and where it came from.
package main
import (
"fmt"
"os"
"time"
"github.com/uchaloop/confmaker"
)
// StoreConfig is the kind of config a library declares: plain fields, env tags,
// and no knowledge of where the values come from.
type StoreConfig struct {
Host string `env:"HOST,notEmpty"`
Timeout time.Duration `env:"TIMEOUT"`
}
func (StoreConfig) ConfigName() string { return "store" }
func (c *StoreConfig) SetDefaults() { c.Timeout = 30 * time.Second }
func (c StoreConfig) Validate() error {
if c.Timeout <= 0 {
return fmt.Errorf("timeout must be positive, got %s", c.Timeout)
}
return nil
}
func main() {
_, err := confmaker.Load[StoreConfig](
confmaker.WithDump(os.Stdout),
confmaker.WithEnv(map[string]string{"STORE_HOST": "db:5432"}),
)
fmt.Println(err)
}
Output: INSTANCE VARIABLE TYPE VALUE SOURCE store STORE_HOST string db:5432 env store STORE_TIMEOUT time.Duration 30s default <nil>
func WithEnv ¶ added in v0.6.2
WithEnv makes a load read vars instead of the process environment.
- vars is the complete environment: nothing is added from the process.
- nil is an empty environment, not the process environment.
- The map is copied when WithEnv is called; later changes to it are not seen.
- WithEnv may be given once per load; a second one is reported by Loader.Load.
Without WithEnv, a load takes a snapshot of the process environment when it starts.
type Handle ¶ added in v0.6.2
type Handle[T any] struct { // contains filtered or unexported fields }
Handle is a config registered with Loader.Add, read with Handle.Value. The zero Handle belongs to no loader, and its Value returns an error.
func (*Handle[T]) Value ¶ added in v0.6.2
Value returns the loaded config, or an error:
- ErrNotLoaded until Loader.Load has finished;
- the error Load returned, when any config of the set failed - even if this config loaded on its own;
- ErrRegisteredAfterLoad for a config added once Load had started.
The struct is returned by value, but maps, slices and pointers inside it are shared by every caller. Treat a loaded config as read-only.
type LoadOption ¶ added in v0.6.2
type LoadOption interface {
// contains filtered or unexported methods
}
LoadOption is an option Load takes: every ConfigOption and every EnvOption.
type Loader ¶ added in v0.6.2
type Loader struct {
// contains filtered or unexported fields
}
Loader loads a set of configs together: configs are registered with Loader.Add, loaded with Loader.Load, and read with Handle.Value.
A Loader is safe for concurrent use and holds no lock while user code runs: ConfigName, SetDefaults, Validate, unmarshalers and the dump writer may call Add or Value on the same Loader. They must not call its Load, which would wait for the load it is part of.
Example ¶
A Loader loads several configs, here two instances of one type, and reports their problems together.
package main
import (
"fmt"
"time"
"github.com/uchaloop/confmaker"
)
// StoreConfig is the kind of config a library declares: plain fields, env tags,
// and no knowledge of where the values come from.
type StoreConfig struct {
Host string `env:"HOST,notEmpty"`
Timeout time.Duration `env:"TIMEOUT"`
}
func (StoreConfig) ConfigName() string { return "store" }
func (c *StoreConfig) SetDefaults() { c.Timeout = 30 * time.Second }
func (c StoreConfig) Validate() error {
if c.Timeout <= 0 {
return fmt.Errorf("timeout must be positive, got %s", c.Timeout)
}
return nil
}
func main() {
loader := confmaker.MakeLoader(confmaker.WithEnv(map[string]string{
"STORE_HOST": "db:5432",
"REPLICA_HOST": "replica:5432",
}))
primary := loader.Add[StoreConfig]() // STORE_*
replica := loader.Add[StoreConfig](confmaker.WithName("replica")) // REPLICA_*
if err := loader.Load(); err != nil {
fmt.Println(err)
return
}
primaryCfg, _ := primary.Value()
replicaCfg, _ := replica.Value()
fmt.Println(primaryCfg.Host, replicaCfg.Host)
}
Output: db:5432 replica:5432
func MakeLoader ¶ added in v0.6.2
MakeLoader returns an empty Loader. opts configure the whole load: WithEnv, WithDump, AllowUnknown. A nil option is reported by Loader.Load.
func (*Loader) Add ¶ added in v0.6.2
func (l *Loader) Add[T any](opts ...ConfigOption) *Handle[T]
Add registers a config of type T and returns the Handle to read it from. The instance name comes from WithName or T's ConfigName method.
An invalid registration - no name, a bad tag, a secret inside a JSON value - is reported by Loader.Load with every other problem. Once Load has started, Add registers nothing and calls no method of T; the handle's Value returns ErrRegisteredAfterLoad.
func (*Loader) Load ¶ added in v0.6.2
Load loads every registered config and returns every problem as one error: invalid options and registrations, conflicts between registrations, unknown variables, variables that did not parse, and the Validate errors of every config whose variables parsed. The steps are listed in the package documentation.
Load runs once. Calling it again returns the same error, or nil, without reading the environment again; a call made while another goroutine loads waits for that result. If user code panics during the load, the panic propagates from the Load that ran it, and later calls return an error.
type Variable ¶ added in v0.6.2
type Variable struct {
// Name is the full variable name, prefix included.
Name string
// Type is the Go type of the field it fills.
Type string
// Required reports whether the variable must be set: the field is declared
// required or notEmpty.
Required bool
// NotEmpty reports whether the variable must also hold a non-empty value: the
// field is declared notEmpty. A required field accepts X= as a deliberate
// empty value.
NotEmpty bool
// Secret reports whether the field holds a secret value.
Secret bool
// Default is the value the field holds after SetDefaults, rendered as the
// text a variable would carry. It is empty for a secret.
Default string
// HasDefault reports whether the field is non-zero before the environment is
// applied. A default of false, 0 or "" is indistinguishable from no default
// and is reported as false.
HasDefault bool
}
Variable describes one environment variable a config reads. The set of them comes from the same compiled schema used to load it, so a variable listed here is exactly a variable the config reads.
func Manifest ¶ added in v0.6.2
func Manifest[T any](opts ...ConfigOption) ([]Variable, error)
Manifest returns the variables a config of type T reads, in declaration order, for generating a .env.example, a config map or a documentation table. The options are those of Loader.Add, so WithName and WithPrefix give the same names as a registration with the same options.
Manifest does not read the environment. It does call T's SetDefaults, and renders each default with MarshalText or the field's plain syntax; a default that cannot be rendered is an error. The result is metadata: Variable.Default is ENV text, not escaped for a shell or YAML.
A declaration Loader.Load would refuse is refused here too, before SetDefaults runs.
Example ¶
Manifest lists the variables and their defaults without reading the environment. The output is ENV names and values, not an escaped shell script.
package main
import (
"fmt"
"time"
"github.com/uchaloop/confmaker"
)
// StoreConfig is the kind of config a library declares: plain fields, env tags,
// and no knowledge of where the values come from.
type StoreConfig struct {
Host string `env:"HOST,notEmpty"`
Timeout time.Duration `env:"TIMEOUT"`
}
func (StoreConfig) ConfigName() string { return "store" }
func (c *StoreConfig) SetDefaults() { c.Timeout = 30 * time.Second }
func (c StoreConfig) Validate() error {
if c.Timeout <= 0 {
return fmt.Errorf("timeout must be positive, got %s", c.Timeout)
}
return nil
}
func main() {
variables, err := confmaker.Manifest[StoreConfig]()
if err != nil {
fmt.Println(err)
return
}
for _, v := range variables {
fmt.Printf("%s=%s\n", v.Name, v.Default)
}
}
Output: STORE_HOST= STORE_TIMEOUT=30s