settings

package module
v1.3.1 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Oct 18, 2023 License: MIT Imports: 12 Imported by: 0

README

12-Factor Compliant Application Configuration

godoc license codecov GoReportCard example

This package gathers values (typically used for application settings and configuration) from various sources outside of the application, and layers them together in a single output struct (supplied as an argument) for use by the application. This package supports 12-factor configuration use cases to facilitate cloud-native API and application development.

The package will first attempt to load settings from the following sources in the order arranged below:

  1. a base file (in yaml or json format)
  2. from the default values map (if provided in ReadOptions)
  3. from any command line provided override files (if ArgsFileOverride switches are defined in ReadOptions)
  4. from any environment override files (if EnvOverride and EnvSearchPaths are provided in ReadOptions)
  5. from all command line argument field specific value overrides (if ArgsMap is provided in ReadOptions)
  6. from all environment variable field specific value overrides (if VarsMap is provided in ReadOptions)

Installation

go get -u go.jtlabs.io/settings

Usage

The following snippet demonstrates creating settings.ReadOptions with instructions that the settings.Gather function uses to populate the supplied config struct:

package main

import (
  "log"

  "go.jtlabs.io/settings"
)

type config struct {
  Data struct {
    Name string `json:"name"`
    Host string `json:"host"`
    Port int    `json:"port"`
  } `json:"data"`
  Logging struct {
    Level string `json:"level"`
  } `json:"logging"`
  Server struct {
    Address string `json:"address"`
  } `json:"server"`
}

func main() {
  var c config
  options := settings.Options().
    SetBasePath("./defaults.yaml").
    SetSearchPaths("./", "./config", "./settings").
    SetDefaultsMap(map[string]interface{}{
      "Server.Address": ":3080",
    }).
    SetArgsMap(map[string]string{
      "--data-name": "Data.Name",
      "--data-host": "Data.Host",
      "--data-port": "Data.Port",
    }).
    SetVarsMap(map[string]string{
      "DATA_NAME":      "Data.Name",
      "DATA_HOST":      "Data.Host",
      "DATA_PORT":      "Data.Port",
      "SERVER_ADDRESS": "Server.Address",
    })

  // read in configuration from all sources
  if err := settings.Gather(options, &c); err != nil {
    log.Fatal(err)
  }
}

For a more verbose example along with execution instructions, see examples/example.go.

ReadOptions

ReadOptions are used to instruct the package where to find override values from a base file, a command line override file, an environment override file, command line arguments, or from environment variables.

EnvDefault

This function exists so that default environment variable driven overrides, similar to those defined in settings-lib, can be provided easily to the Gather function.

options := settings.Options().EnvDefault()
settings.Gather(options, &config)
SetArg

Similar to SetArgsMap, this can be used to attach command line arguments, individually, to fields for settings.

options := settings.Options().
  SetArg("--a-flag", "Field.Name")
settings.Gather(options, &config)
SetArgsFileOverride

When providing a value to this method, one can override the underlying settings via one or more specific files that are provided via command line arguments.

options := settings.
  Options().
  SetArgsFileOverride("./path/to/first-file.yml", "./path/to/another/second-file.json")
settings.Gather(options, &config)

The first-file.yml will be read and applied, and then the second-file.json will be read and applied over the top of the first. These files can be partial files with a subset of the fields from the out struct defined as &config in the example above if desired.

SetArgsMap

The arguments map is used by Gather to determine how command line switches can be applied to specific out struct fields.

options := settings.
  Options().
  SetArgsMap(map[string]string{
    "--switch-to-look-for": "CaseSensitive.Field.Where.Hiearchy.Is.Noted.By.Dot",
    "--logging-level":      "Logging.Level",
    "-l":                   "Logging.Level",
  })
settings.Gather(options, &config)

Any switches that are provided in the map that do not appear in the list of os.Args for the application are effectively ignored. If the desired outcome is to have an alias for a command line argument (i.e. --logging-level and -l both capable of overriding Logging.Level), each value can be independently added to the map. When processing arguments, --some-switch=value (notice the = character) is processed the same as --some-switch value so that the value will properly read and applied in either scenario.

SetBasePath

The base path for settings is the initial (yaml or json) file that is loaded to populate the out argument to the gather method. As with the command line override file and with the environment override file, this base settings file is not required to be a complete serialization of the out struct... it can be partially defined if desired. If a file is specified, and the file can't be found or read, the Gather method will return a file doesn't exist (i.e. os.ErrNotExist) or a SettingsFileReadError in the event there is some other read problem.

options := settings.
  Options().
  SetBasePath("./settings.yml")
settings.Gather(options, &config)
SetDefaultsMap

The defaults map is used by settings to apply default values to fields in the out struct. These defaults are applied immediately after the base settings (if provided) are applied.

options := settings.
  Options().
  SetArgsMap(map[string]interface{}{
    "CaseSensitive.Field.Where.Hiearchy.Is.Noted.By.Dot": true,
    "Data.Port":                                          27017,
    "Server.Address":                                     ":8080",
    "Logging.Level":                                      "trace",
    "Name":                                               "cool name",
  })
settings.Gather(options, &config)

The string value of the map is the field path where hierarchy / depth is noted by the . character.

SetEnvOverride and SetEnvSearchPaths and SetEnvSearchPattern

Environment override and search paths can be provided to the package to enable virtually named environment level overrides at a partial or complete configuration level.

options := settings.
  Options().
  SetEnvSearchPaths("./", "./settings"). // look for files in "./" and "./settings
  SetEnvOverride("GO_ENV", "GO_ENVIRONMENT")
settings.Gather(options, &config)

In the above example, if a value is set in the GO_ENV or GO_ENVIRONMENT variables for the application, the value will be used in a search for matching yaml or json files that exist in the paths provided as search paths (in the above example, ./ and ./settings). To illustrate:

GO_ENV=testing go run cmd/app.go

The GO_ENV value is testing. Combined with the code snippet above, the app would search for the following files:

  • ./testing.yml
  • ./testing.yaml
  • ./testing.json
  • ./settings/testing.yml
  • ./settings/testing.yaml
  • ./settings/testing.json

Upon finding a file that matches (the first match), that file is read and the fields defined therein are applied to the out struct.

If SetEnvSearchPattern is used to defined a file name pattern, in addition to the above steps, files are searched using the file name pattern provided...

options := settings.
  Options().
  SetEnvSearchPaths("./", "./settings"). // look for files in "./" and "./settings
  SetEnvSearchPattern("config.%s").
  SetEnvOverride("GO_ENV", "GO_ENVIRONMENT")
settings.Gather(options, &config)

Using the above example, when the following is used to start the application:

GO_ENV=testing go run cmd/app.go

The settings package will look for the following files:

  • ./config.testing.yml
  • ./config.testing.yaml
  • ./config.testing.json
  • ./settings/config.testing.yml
  • ./settings/config.testing.yaml
  • ./settings/config.testing.json

In this scenario, if both ./testing.yml and ./config.testing.yml are found, only the ./testing.yml will be loaded.

SetVar

Similar to SetVarsMap, this can be used to associate environment variables, individually, to fields for settings.

options := settings.Options().
  SetVar("FIELD_NAME", "Field.Name")
settings.Gather(options, &config)
SetVarsMap

Similar to the Args map, the Vars map can be used to override individual fields with values defined as environment variables.

Q & A

Why build this?

For our use case, we desired a simple and extensible configuration mechanism that spiritually adheres to 12-factor principles and facilitates the layered specification of nn settings / configuration. An initial base configuration file to be the start, with an ability to specify override files as a layer of additional settings (either full or partial) on top of the base file (the locations for which specified via command line arguments or environment variables). Finally, having an ability to override individual keys within the configuration with specific environment variables or command line arguments.

This settings library was built in the spirit of https://github.com/brozeph/settings-lib, which in many ways could be considered this package's Node.js flavored older sibling. This package is not a direct port of settings-lib, though, and makes use of Go specific idioms.

How is this different than https://github.com/spf13/viper?

Viper is an incredible and feature rich configuration utility that also aligns, philosophically, with 12-factor principles. Viper supports several features that this package does not:

  • loading configuration from external sources (i.e. Consul, etcd, and k/v stores, etc.)
  • reading configuration from more sources (i.e. HCL, INI, TOML, dotenv files, etc.)
  • saving configuration back out to a destination

Where Viper differs is in the order in which configuration is loaded. Additionally, to load additional full or partial files specified through command line arguments or environment variables, custom code is required.

Ultimately, Viper is a great choice for configuration as well. This package provides a subset of the functionality of Viper, and approaches the loading of configuration layers in a different order and with a completely different underlying approach.

Viper doesn't support case-senstive keys... does this library?

TL;DR: yes, but this isn't really applicable in this package.

The approach within Viper involves loading configuration from various sources and each source is a potential origin of the configuration value. It would be a non-trivial matter for Viper to support case-sensitve key lookup.

In this package, there is a single source of truth for the final configuration, which is the out interface{} struct provided to the Gather method. All other sources for configuration are mapped to the desired output struct.

Documentation

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

func Gather 

func Gather(opts ReadOptions, out interface{}) error

Gather compiles configuration from various sources and iteratively builds up the out object with the values that are retrieved successively from the following sources: 1. base settings file 2. defaults as configured in options (*diverges from github.com/brozeph/settings-lib) 3. override files (from command line) 4. override files (from environment) 5. command line arguments 6. environment variables

Types

type ReadOptions 

type ReadOptions struct {
	ArgsFileOverride []string
	ArgsMap          map[string]string
	BasePath         string
	DefaultsMap      map[string]interface{}
	EnvOverride      []string
	EnvSearchPaths   []string
	EnvSearchPattern string
	VarsMap          map[string]string
}

ReadOptions define additional optional instructions for the Settings package when reading and compiling layers of configuration settings from various sources

func Options 

func Options() ReadOptions

Options returns an empty ReadOptions for use with the Settings package

func (ReadOptions) EnvDefault added in v1.1.0 

func (ro ReadOptions) EnvDefault() ReadOptions

EnvDefault populates the ReadOptions struct with a a default environment override variable setting and a few default search paths

func (ReadOptions) SetArg added in v1.1.0 

func (ro ReadOptions) SetArg(arg string, fieldPath string) ReadOptions

SetArg can be used to explicitly map a command line argument to a field

func (ReadOptions) SetArgsFileOverride 

func (ro ReadOptions) SetArgsFileOverride(args ...string) ReadOptions

SetArgsFileOverride instructs the settings package on where to look for any potential override file locations that are provided as command line arguments

func (ReadOptions) SetArgsMap 

func (ro ReadOptions) SetArgsMap(argsMap map[string]string, rewrite ...bool) ReadOptions

SetArgsMap will either rewrite or, by default, augment the map that defines which configuration keys are related to the specified command line arguments

func (ReadOptions) SetBasePath 

func (ro ReadOptions) SetBasePath(path string) ReadOptions

SetBasePath can be used to define the path to the base settings file which is the first element loaded when the Settings package begins reading configuration

func (ReadOptions) SetDefaultsMap 

func (ro ReadOptions) SetDefaultsMap(defMap map[string]interface{}, rewrite ...bool) ReadOptions

SetDefaultsMap can be used to define default values for config elements in the event that the value is not provided in one of the layered mechanisms used to read settings

func (ReadOptions) SetEnvOverride 

func (ro ReadOptions) SetEnvOverride(vars ...string) ReadOptions

SetEnvOverride instructs the settings package on where to look for any potential override file locations that are provided as environment variables to the application

func (ReadOptions) SetEnvSearchPaths 

func (ro ReadOptions) SetEnvSearchPaths(paths ...string) ReadOptions

SetEnvSearchPaths can be used to instruct the Settings package on where it might find additional configuration files for use when loading additional layers of configuration

func (ReadOptions) SetEnvSearchPattern added in v1.2.0 

func (ro ReadOptions) SetEnvSearchPattern(pattern string) ReadOptions

func (ReadOptions) SetVar added in v1.1.0 

func (ro ReadOptions) SetVar(v string, fieldPath string) ReadOptions

SetVar can be used to explicitly map an environment variable to a field

func (ReadOptions) SetVarsMap 

func (ro ReadOptions) SetVarsMap(varsMap map[string]string, rewrite ...bool) ReadOptions

SetVarsMap will either rewrite or, by default, augment the map that associates environment variables to various configuration keys specified in the base

type SettingsError 

type SettingsError struct {
	Message string
}

SettingsError is any type of error that is raised specifically related to gathering and applying settings from each of the specified sources

func SettingsFieldDoesNotExist 

func SettingsFieldDoesNotExist(overrideType string, fieldName string) SettingsError

SettingsFieldDoesNotExist is an error when a field is specified via the DefaultsMap that does not exist in the out struct value that is provided to settings.Gather

func SettingsFieldSetError 

func SettingsFieldSetError(fieldName string, t reflect.Kind, m ...error) SettingsError

SettingsFieldSetError is raised when a field's value is not actually settable

func SettingsFieldTypeMismatch 

func SettingsFieldTypeMismatch(fieldName string, expectedType reflect.Kind, receivedType reflect.Kind) SettingsError

SettingsFieldTypeMismatch is raised in the event there is a mismatch between types when trying to override a specific value

func SettingsFileParseError 

func SettingsFileParseError(path string, desc string) SettingsError

SettingsFileParseError occurs when a specified settings file can't be properly unmarshalled

func SettingsFileReadError 

func SettingsFileReadError(path string, desc string) SettingsError

SettingsFileReadError occurs when a specified settings file is not readable

func SettingsFileTypeError 

func SettingsFileTypeError(path string, ext string) SettingsError

SettingsFileTypeError occurs when a format is requested that the settings package does not support

func SettingsOutCannotBeNil 

func SettingsOutCannotBeNil() SettingsError

SettingsOutCannotBeNil occurs when the out field in the settings struct is set to nil, intentionally or otherwise

func SettingsTypeDiscoveryError 

func SettingsTypeDiscoveryError(t reflect.Kind) SettingsError

SettingsTypeDiscoveryError occurs when the out value provided to settings.Gather is not a struct

func (SettingsError) Error 

func (e SettingsError) Error() string

Source Files

View all Source files

Directories

Path Synopsis
package main in the example can be used to demo the settings parsing from command line, environment variables, specified override files and environment overrides.
 package main in the example can be used to demo the settings parsing from command line, environment variables, specified override files and environment overrides.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.