README
¶
Alligotor
The zero configuration configuration package.
Install
go get github.com/brumhard/alligotor
What is Alligotor?
Alligotor is designed to be used as the configuration source for executables (not commands in a command line application) for example for api servers or any other long-running applications that need a startup config.
It takes only a few lines of code to get going, and it supports:
- setting defaults just like you're used to from for example json unmarshalling (see this example)
- reading from YAML and JSON files from io.Reader, local file system or fs.FS
- reading from environment variables
- reading from command line flags
- defining custom source to load config from your preferred source (e.g. etcd)
- extremely simple API
- support for every type (by implementing TextUnmarshaler) and out of the box support for many common ones
- autogenerated property names for each child property in the config, but still configurable via struct tags
- set overwrite order by defining the sources in the preferred order in
alligotor.New()
Why Alligotor?
There are a lot of configuration packages for Go that give you the ability to load you configuration from several sources like env vars, command line flags or config files.
Alligotor was designed to have the least configuration effort possible (autogenerating the property names for the source trough reflection) while still keeping it customizable. So for example if a config struct looks like the following:
type cfg struct {
API struct {
Port int
}
}
The port value will be loaded by default from the env variable <PREFIX>_API_PORT and the flag --api-port without the
need to set that explicitly.
That's why if you keep the package defaults you only need one function call, and your config struct definition to fill this struct with values from environment variables, several config files and command line flags or your defined custom source.
Known unsupported usecases
Read directly into properties of embedded structs
Generally embedded structs are supported but certain use cases don't work. So for example in the following struct:
type DB struct {
Host string
}
type Config {
DB
}
You can set the value for the DB.Host with the env variable <PREFIX>_DB_HOST but not with <PREFIX>_HOST directly.
Arrays
Since there is no nice way of representing arrays in all config sources (for example environment variables) it's currently not supported in these sources.
The ReadersSource on the other hand can easily read arrays.
Minimal example
package main
import (
"github.com/brumhard/alligotor"
"go.uber.org/zap/zapcore"
"time"
)
func main() {
// define the config struct
cfg := struct {
SomeList []string
SomeMap map[string]string
API struct {
Enabled bool
LogLevel zapcore.Level
}
DB struct {
HostName string
Timeout time.Duration
}
}{
// could define defaults here
}
// get the values
_ = alligotor.Get(&cfg)
}
Just like with the json package alligotor only supports setting public properties since it relies on reflection.
Custom setup
As alligotor aims for good customizability, the Collector's constructor supports as many sources as you like. Included in the package are one source for env vars, one for config files (supporting readers, local file system or fs.FS) and one for cli flags (see sources).
It is shown in the following example.
// all predefined sources
_ = alligotor.New(
alligotor.NewFilesSource("./test_config.*"),
alligotor.NewEnvSource("TEST"),
alligotor.NewFlagsSource(),
)
// only from env vars with prefix "TEST" and custom separator
_ = alligotor.New(
alligotor.NewEnvSource("TEST", alligotor.WithEnvSeparator("::")),
)
As shown in the latter example, the sources support an option to set a custom separator. In case it is not set explicitly, it will be set to the defaults:
- env vars:
_(underscore) - cli flags:
.(dash)
Sources
For each of the following sources the following example config struct is used.
// example struct
type Config struct {
Enabled bool
Sub struct {
Port int
}
}
Environment variables
The source for environment variables can be used as follows:
_ = alligotor.New(
alligotor.NewEnvSource("TEST", alligotor.WithEnvSeparator("::")),
)
It supports setting a custom prefix as well as a custom separator. The separator is needed for nested config structs.
So for example for the example struct from above and the defined source configuration the value for the Port field
will be read from TEST::SUB::PORT.
Like with the other sources the properties name to look up can be changed by adding a struct tag. In that case
add config:"env=something" as a struct tag for the Port field and it will be read from TEST::SUB::SOMETHING.
Commandline flags
The source for command line flags can be used as follows:
_ = alligotor.New(
alligotor.NewFlagsSource(alligotor.WithFlagSeparator(".")),
)
It supports setting a custom separator, that is needed for nested structs.
So for example for the example struct from above and the defined source configuration the value for the Port field
will be read from --sub.port.
Like with the other sources the properties name to look up can be changed by adding a struct tag. In that case
add config:"flag=something" as a struct tag for the Port field and it will be read from --sub.something.
In addition the struct tag can be defined as config:"flag=p" to set the short name for the flag (-p) or any of
config:"flag=p some" or config:"flag=some p" to overwrite the name and the short name.
To set a flags usage string in addition to the config struct tag also the description struct tag is read and set as
the flags usage that is returned when the user requests help with --help or -h.
Files
The source for files can be used in one of the following ways:
_ = alligotor.New(
// any io.Reader is supported
alligotor.NewReadersSource(strings.NewReader(`{"key":"value"}`))
)
_ = alligotor.New(
// reads from local fs in this case
alligotor.NewFilesSource("dir/example_config.*", "test2/config.yml"),
)
_ = alligotor.New(
// fsys has a type implementing fs.FS in this case
alligotor.NewFSFilesSource(fsys, "dir/example_config.*", "test2/config.yml")
)
NewReadersSource reads the config file from any io.Reader so for example a file or an http endpoint.
NewFilesSource and NewFSFilesSource are simple wrappers around the ReadersSource to find the files using glob
patterns on any filesystem (either local FS or fs.FS). This differentiation is used since os.DirFS does not support
propper relative and absolute paths for the local filesystem.
Reading from files works as expected (just like json or yaml unmarshaling). The only difference is that it looks for fields in a case-insensitive manner.
Of course also here the name can be defined by setting the struct tag to for example config="file=something" which
works just like the json or yaml struct tag.
Currently, only yaml and json files are supported but others will be added if needed.
Struct tags
Struct tags are used to overwrite the name for the env source that is generated by default. They are defined in the following format:
type Config struct {
Enabled bool `config:"key=value,key2=value2"`
}
where key could for example be file or env. The struct tag can also be consumed from custom sources from the Field
property Field.Configs(), which contains a map from struct tag key to value.
Custom
Custom sources can be added by implementing the following interfaces. For an example on how to implement a config source take a look at the env source, which implements reading from environment variables in less than 100 lines of code.
ConfigSource
Each config source needs to implement at least the following interface.
type ConfigSource interface {
Read(field *Field) (interface{}, error)
}
As shown it contains only one method that receives a Field instance and returns the value that was found for the field. For sources that only support setting values as strings (like for example environment variables) just return a byte slice containing the string and it will automatically be converted to the target type if possible. Any other type is used directly leading to an error on type mismatch.
You should not return structs directly since this could lead to errors if some struct properties are set and others are not. This would then overwrite the target with the zero value, which is not intended.
The received fields match directly to the fields in the config. So for example for a config struct like the following:
type Config struct {
Sub struct {
Field string
}
}
two fields will be send to the Read function, one containing the whole sub struct and one referencing only the Field
property. The structs are included to enable structs that implement the TextUnmarshaler interface. If no value is
found for a specific field nil should be returned in order to not override any existing value for that field with an
empty one.
ConfigSourceInitializer
If the custom config source depends on some initialization before reading the fields the ConfigSourceInitializer
interface can be implemented as well. The method is invoked right before calling the Read function. In the existing
sources this is used for example to read the the files to not do it for every field or read in the environment
variables.
type ConfigSourceInitializer interface {
Init(fields []Field) error
}
Documentation
¶
Overview ¶
Example ¶
package main
import (
"fmt"
"log"
"os"
"path"
"strings"
"time"
"github.com/brumhard/alligotor"
)
type Config struct {
WillStayDefault string // not overwritten in any source -> will keep default
SomeList []string `description:"list that can ba assigned in format: a,b,c"` // description used for flag usage
SomeMap map[string]string `config:"env=MAP"` // overwrite the env var to read
SomeCustomType SomeCustomType `config:"env=custom"` // implements text.Unmarshaler
API struct {
Enabled *bool // pointers to basic types are also supported
}
DB struct {
HostName string `config:"flag=h host"` // overwrite the cli flags to read, h is shortname
Timeout time.Duration `config:"flag=time"` // only overwrite long name
}
TimeStamp time.Time `config:"file=custom"` // implements text.Unmarshaler, overwrite key in file
Everything string `config:"env=every,flag=e every,file=every"` // set overwrites for every source
}
func main() {
dir, _ := os.MkdirTemp("", "testing")
defer os.RemoveAll(dir)
jsonBytes := []byte(`{
"custom": "2007-01-02T15:04:05Z",
"db": {
"timeout": "2m0s"
}
}`)
os.Args = []string{"cmdName", "--somelist", "a,b,c", "--api.enabled", "true", "-h", "dbhost", "--every", "every"}
_ = os.Setenv("TEST_MAP", "a=a,b=b,c=c")
_ = os.Setenv("TEST_DB_TIMEOUT", "1m0s")
_ = os.Setenv("TEST_CUSTOM", "key=value")
filePath := path.Join(dir, "example_config.json")
_ = os.WriteFile(filePath, jsonBytes, os.ModePerm)
cfg := Config{WillStayDefault: "yessir"}
// The order of sources will also set the order in which the sources overwrite each other.
// That's why the db timeout set in the json is overwritten with the one set in env variable.
cfgReader := alligotor.New(
alligotor.NewFilesSource(path.Join(dir, "example_config.*")),
alligotor.NewEnvSource("TEST"),
alligotor.NewFlagsSource(),
)
// There's also a default reader, that can be used with alligotor.Get().
if err := cfgReader.Get(&cfg); err != nil {
log.Fatal(err)
}
fmt.Println(
cfg.WillStayDefault, cfg.SomeList, cfg.SomeMap, cfg.SomeCustomType,
*cfg.API.Enabled, cfg.DB.HostName, cfg.DB.Timeout, cfg.TimeStamp.UTC(),
cfg.Everything,
)
}
type SomeCustomType struct {
key string
value string
}
func (s *SomeCustomType) UnmarshalText(text []byte) error {
split := strings.SplitN(string(text), "=", 2)
for i := range split {
split[i] = strings.TrimSpace(split[i])
}
s.key = split[0]
s.value = split[1]
return nil
}
func (s SomeCustomType) String() string {
return fmt.Sprintf("%s=%s", s.key, s.value)
}
Output: yessir [a b c] map[a:a b:b c:c] key=value true dbhost 1m0s 2007-01-02 15:04:05 +0000 UTC every
Index ¶
Examples ¶
Constants ¶
This section is empty.
Variables ¶
var ( ErrPointerExpected = errors.New("expected a pointer as input") ErrStructExpected = errors.New("expected pointer to struct as input") ErrTypeMismatch = errors.New("type mismatch when trying to assign") ErrDuplicateConfigKey = errors.New("key already used for a config source") )
var ( ErrMalformedFlagConfig = errors.New("malformed flag config strings") ErrHelp = errors.New("help requested") )
var DefaultCollector = &Collector{ Sources: []ConfigSource{ NewFilesSource("./config.*"), NewEnvSource(""), NewFlagsSource(), }, }
DefaultCollector is the default Collector and is used by Get.
var ErrFileFormatNotSupported = errors.New("file format not supported or malformed content")
Functions ¶
func Get ¶
func Get(v interface{}) error
Get is a wrapper around DefaultCollector.Get. All predefined configuration sources are enabled. For environment variables it uses no prefix and "_" as the separator. For flags it use "-" as the separator. For config files it uses "config" as the basename and searches in the current directory.
Types ¶
type Collector ¶
type Collector struct {
Sources []ConfigSource
}
Collector is the root struct that implements the main package api. The only method that can be called is Collector.Get to unmarshal the found configuration values from the configured sources into the provided struct. If the default configuration suffices your needs you can just use the package level Get function instead without initializing a new Collector struct.
The order in which the different configuration sources overwrite each other can be configured by the order in which the sources are defined. The default is the following: defaults -> config files -> environment variables -> command line flags (each source is overwritten by the following source)
To define defaults for the config variables it can just be predefined in the struct that the configuration is supposed to be unmarshalled into. Properties that are not set in any of the configuration sources will keep the preset value.
Since environment variables and flags are purely text based it also supports types that implement the encoding.TextUnmarshaler interface like for example zapcore.Level and logrus.Level. On top of that custom implementations are already baked into the package to support duration strings using time.ParseDuration() and time using time.Parse() as well as string slices ([]string) in the format val1,val2,val3 and string maps (map[string]string) in the format key1=val1,key2=val2.
func New ¶ added in v0.1.2
func New(sources ...ConfigSource) *Collector
New returns a new Collector. It accepts multiple configuration sources that implement the ConfigSource interface. If no sources are present the resulting Collector won't have any configuration sources and return the input struct without any changes in the Collector.Get method.
func (*Collector) Get ¶
Get is the main package function and can be used by its wrapper Get or on a defined Collector struct. It expects a pointer to the config struct to write the config variables from the configured source to. If the input param is not a pointer to a struct, Get will return an error.
Get looks for config variables in all defined sources. Further usage details can be found in the examples or the Collector struct's documentation.
type ConfigSource ¶ added in v0.2.0
ConfigSource consists of one method that gets a certain field and should return its value. If this value is a string and should be parsed (for example env variables can only be retrieved as a string but could also resemble an int value or even a string slice), a []byte should be returned.
If anything else than a byte slice is returned the given value will be used as is and if there's a type mismatch an error will be reported.
type ConfigSourceInitializer ¶ added in v0.2.0
type ConfigSourceInitializer interface {
// Init should be called right before Read to initialize stuff.
// Some things shouldn't be initialized in the constructor since the environment or files (the config source)
// could be altered in the time between constructing a config source and calling the Read method.
Init(fields []Field) error
}
ConfigSourceInitializer is an optional interface to implement and can be used to initialize the config source before reading the fields one by one with the Read method of ConfigSource.
type EnvOption ¶ added in v0.1.2
type EnvOption func(*EnvSource)
EnvOption takes an EnvSource as input and modifies it.
func WithEnvSeparator ¶ added in v0.1.2
WithEnvSeparator adds a custom separator to an EnvSource struct.
type EnvSource ¶ added in v0.2.0
type EnvSource struct {
// contains filtered or unexported fields
}
EnvSource is used to read the configuration from environment variables. prefix can be defined to look for environment variables with a certain prefix. separator is used for nested structs and also for the Prefix. As an example: If prefix is set to "example", the separator is set to "_" and the config struct's field is named Port, it will by default look for the environment variable "EXAMPLE_PORT".
func NewEnvSource ¶ added in v0.2.0
NewEnvSource returns a new EnvSource. prefix defines the prefix to be prepended to the automatically generated names when looking for the environment variables. prefix can be empty. It accepts an EnvOption to override the default env separator.
type Field ¶ added in v0.2.0
type Field struct {
// contains filtered or unexported fields
}
Field is a struct to hold all information for a struct's field that should be filled with configuration.
func (*Field) Description ¶ added in v0.2.0
type FilesSource ¶ added in v0.2.0
type FilesSource struct {
ReadersSource
// contains filtered or unexported fields
}
FilesSource is a wrapper around ReadersSource to automatically find the needed readers in a filesystem. To use the local FS the NewFilesSource can be used to find the files. NewFSFilesSource can be used for other usecases where the FS is present on S3 or an embed.FS. This differentiation was implemented since the os.DirFS does not really support relative and absolute paths easily.
func NewFSFilesSource ¶ added in v0.3.0
func NewFSFilesSource(fsys fs.FS, globs ...string) *FilesSource
func NewFilesSource ¶ added in v0.2.0
func NewFilesSource(globs ...string) *FilesSource
func (*FilesSource) Init ¶ added in v0.2.0
func (s *FilesSource) Init(fields []Field) error
Init tries to find files on the filesystem matching the supplied globs and reads them. Afterwards the underlying ReadersSource is initialized.
type FlagOption ¶ added in v0.1.2
type FlagOption func(*FlagsSource)
FlagOption takes a FlagsSource as input and modifies it.
func WithFlagSeparator ¶ added in v0.1.2
func WithFlagSeparator(separator string) FlagOption
WithFlagSeparator adds a custom separator to a FlagsSource struct.
type FlagsSource ¶ added in v0.2.0
type FlagsSource struct {
// contains filtered or unexported fields
}
FlagsSource is used to read the configuration from command line flags. separator is used for nested structs to construct flag names from parent and child properties recursively.
func NewFlagsSource ¶ added in v0.2.0
func NewFlagsSource(opts ...FlagOption) *FlagsSource
NewFlagsSource returns a new FlagsSource. It accepts a FlagOption to override the default flag separator.
func (*FlagsSource) Init ¶ added in v0.2.0
func (s *FlagsSource) Init(fields []Field) error
Init initializes the fieldToFlagInfos property. It should be used right before calling the Read method to load the latest flags.
func (*FlagsSource) Read ¶ added in v0.2.0
func (s *FlagsSource) Read(field *Field) (interface{}, error)
Read reads the saved flagSet from the Init function and returns the set value for a certain field. If no value is set in the flags it returns nil.
type ReadersSource ¶ added in v0.3.0
type ReadersSource struct {
// contains filtered or unexported fields
}
ReadersSource is used to read configuration from any type that implements the io.Reader interface. The data in the readers should be in one of the supported file formats (currently yml and json). This enables a wide range of usages like for example reading the config from an http endpoint or a file.
The ReadersSource accepts io.Reader to support as many types as possible. To improve the experience with sources that need to be closed it will also check if the supplied type implements io.Closer and closes the reader if it does.
func NewReadersSource ¶ added in v0.3.0
func NewReadersSource(readers ...io.Reader) *ReadersSource
NewReadersSource returns a new ReadersSource that reads from one or more readers. If the input reader slice is empty this will be a noop reader.
func (*ReadersSource) Init ¶ added in v0.3.0
func (s *ReadersSource) Init(_ []Field) error
Init initializes the fileMaps property. It should be used right before calling the Read method to load the latest config files' states.
func (*ReadersSource) Read ¶ added in v0.3.0
func (s *ReadersSource) Read(field *Field) (interface{}, error)
Read reads the saved fileMaps from the Init function and returns the set value for a certain field. If not value is set in the flags it returns nil.
Source Files
Directories