conflux

README

conflux

conflux is a simple Go library to read configuration values from multiple sources such as YAML files, environment variables, and Bitwarden secrets.

Usage

// Define a struct with required fields
type proxmox struct {
	SSHPublicKeyPath     string `json:"ssh_public_key_path" required:"true"`
	NodeCIDRAddress      string `json:"node_cidr_address" required:"true"`
	GatewayAddress       string `json:"gateway_address" required:"true"`
	AdminPassword        string `json:"admin_password" required:"true" conflux:"proxmox_admin_password"`
}

// Create a config mux that reads from two YAML files, the environment, and Bitwarden
configMux := conflux.NewConfigMux(
  conflux.WithYAMLFileReader("config/all.yml", conflux.WithPath("config/other.yml")),
  conflux.WithEnvReader(),
  conflux.WithBitwardenSecretReader(),
)

// Read configuration values into a struct
var proxmoxConfig proxmox
diagnostics, err := conflux.Unmarshal(configMux, &proxmoxConfig)
if errors.Is(err, conflux.ErrInvalidFields) {
  return fmt.Errorf("invalid or missing config fields:\n%s", conflux.DiagnosticsToTable(diagnostics))
} else if err != nil {
  return fmt.Errorf("failed to unmarshal config: %w", err)
}

// at this point, `proxmoxConfig` is guaranteed to have all required fields set to some non-zero value

Why use this library instead of Viper, Koanf, etc?

Use this library if you want something that:

• Is light (~745 SLOC).
• Supports reading from Bitwarden Secrets.
• Has built-in validation. The required tag allows conflux to give you an exact report of the configurations that were found and missing. This report can be printed as a table for a user-friendly experience.
• Is easily extensible. You can easily your own Readers that read from any source you wish.
• Has flexible initialization logic. Readers can be initialized lazily. This allows us to initialize a Reader with a map of configs that have been read so far. The BitwardenSecretReader actually uses the configs found by YAMLFileReader and EnvReader to authenticate to Bitwarden.
• Has flexible validation logic. If your struct has some specific validation rules, conflux will run them if you define a receiver with the following signature: Validate(map[string]string) bool.
• Has flexible struct-filling logic. If your struct needs to fill-in additional fields after the required fields have been filled, conflux will fill those fields for you if you define a receiver with the following signature: FillInKeys() error.

Other features

• If you are unmarshalling into a struct, and one of the field names (FieldName) doesn't match the name of its corresponding config key (field_name), you can use the conflux tag. This will tell conflux to set FieldName to the value of the config key field_name. In the example above, a value for proxmox_admin_password will be used to set the field AdminPassword.
• First-class testing and mocking support. If you have a function with a *ConfigMux parameter, you can create a mock *ConfigMux and pass it in as an argument:

  mockFS := fstest.MapFS{
    "config/all.yml":     {Data: []byte("...")},
    "config/proxmox.yml": {Data: []byte("...")},
  }
  mockEnv := []string{"SSH_PORT=9999", "SSH_PUBLIC_KEY_PATH=~/.ssh/env.pub"}
  mockConfigMux := NewConfigMux(
    WithYAMLFileReader("config/all.yml", WithPath("config/proxmox.yml"), WithFileSystem(mockFS)),
    WithEnvReader(WithEnviron(mockEnv)),
  )
  

Reasons NOT to use this library

• You need something much more mature and battle-tested in production
• Your configs aren't just string to string key-value pairs. Some values are nested objects or arrays.
• You need out-of-the-box support for a bunch of config files like JSON/TOML, or data sources like S3/Vault.

Conflicts

• Sometimes there are two sources that define different values for the same configuration key. Conflux has the following rules for resolving conflicts:
  • When initializing ConfigMux, Readers should be passed as arguments from lowest to highest priority. For example, in the usage above, conflicting values in BitwardenSecretReader will override env values, which will in turn override file values.
  • When initializing YAMLFileReader, paths should be passed as arguments from lowest to highest priority. The path with the lowest priority is the one that is passed as the first argument. The path with the next-lowest priority is the one that is passed-in as the first functional option: WithPath("some-file-here.yml"), and so on.
  • If you pass-in a directory to YAMLFileReader, YAMLFileReader will read the files in that directory recursively in lexicographic order. That is to say that a file A that falls lexicographically before another file B, will have lower priority that B.

Name

I wanted to name this library Confluence, because one of its definitions is: "the flowing together of two or more streams". This word kind-of describes the logic this library has of merging multiple configuration sources into a single source of truth. Unfortunately, Confluence was already taken. So I went with another word that has the same definition, conflux.

Installation

go get github.com/dannyvelas/conflux

Testing

go test ./...

License

MIT License. See LICENSE for details.

Documentation

Overview

Package conflux provides a simple API for reading configs from the environment, yaml files, and bitwarden secrets

Index

• Constants
• Variables
• func DiagnosticsToTable(data map[string]string) string
• func NewBitwardenSecretReader(configMap map[string]string) *bitwardenSecretReader
• func NewEnvReader(opts ...func(*envReader)) envReader
• func NewYAMLFileReader(path string, opts ...func(*yamlFileReader)) *yamlFileReader
• func Unmarshal(r Reader, target any) (map[string]string, error)
• func WithBitwardenSecretReader() func(*ConfigMux)
• func WithCustomLazyReader(fn func(configMap map[string]string) Reader) func(*ConfigMux)
• func WithCustomReader(r Reader) func(*ConfigMux)
• func WithEnvReader(opts ...func(*envReader)) func(*ConfigMux)
• func WithEnviron(environ []string) func(*envReader)
• func WithFileSystem(fileSystem fs.FS) func(*yamlFileReader)
• func WithPath(path string) func(*yamlFileReader)
• func WithYAMLFileReader(path string, opts ...func(*yamlFileReader)) func(*ConfigMux)
• type ConfigMux
  • func NewConfigMux(opts ...func(*ConfigMux)) *ConfigMux
  • func (r *ConfigMux) Read() (ReadResult, error)
• type DiagnosticReadResult
  • func NewDiagnosticReadResult(configMap, diagnostics map[string]string) DiagnosticReadResult
  • func (r DiagnosticReadResult) GetConfigMap() map[string]string
• type ReadResult
• type Reader
• type SimpleReadResult
  • func NewSimpleReadResult(configMap map[string]string) SimpleReadResult
  • func (r SimpleReadResult) GetConfigMap() map[string]string

Constants

const (
	StatusMissing = "missing"
	StatusLoaded  = "loaded"
)

Variables

var ErrInvalidFields = errors.New("invalid or missing fields")

Functions

func DiagnosticsToTable

func DiagnosticsToTable(data map[string]string) string

DiagnosticsToTable takes a diagnostic map and returns it as a pretty-printed formatted table This is useful as a user-friendly report of missing and found configuration values

func NewBitwardenSecretReader

func NewBitwardenSecretReader(configMap map[string]string) *bitwardenSecretReader

NewBitwardenSecretReader creates a new Bitwarden secret reader using the provided config map to authenticate to Bitwarden.

func NewEnvReader

func NewEnvReader(opts ...func(*envReader)) envReader

NewEnvReader creates a new reader which gets key-value pairs from the environment

func NewYAMLFileReader

func NewYAMLFileReader(path string, opts ...func(*yamlFileReader)) *yamlFileReader

NewYAMLFileReader creates a new reader which gets key-value pairs from YML files from specified directories/files

func Unmarshal

func Unmarshal(r Reader, target any) (map[string]string, error)

Unmarshal reads key-value pairs from the provided Reader and unmarshals them into the target struct or map.

func WithBitwardenSecretReader

func WithBitwardenSecretReader() func(*ConfigMux)

WithBitwardenSecretReader adds a Bitwarden secret reader to the config mux

func WithCustomLazyReader

func WithCustomLazyReader(fn func(configMap map[string]string) Reader) func(*ConfigMux)

WithCustomLazyReader lets you add your own custom reader to the mux The difference between WithCustomReader and WithCustomLazyReader is: - WithCustomReader asks for an already-initialized reader - WithCustomLazyReader asks for a function to initialize a reader This function is useful if your reader needs to be initialized after some config values have already been read. For example, the BitwardenSecretReader would need to be initialized this way because it expects a map of configs as an argument. It uses this map to try to authenticate to Bitwarden

func WithCustomReader

func WithCustomReader(r Reader) func(*ConfigMux)

WithCustomReader lets you add your own custom reader to the mux your custom reader just needs to implement the "Reader" interface The difference between WithCustomReader and WithCustomLazyReader is: - WithCustomReader asks for an already-initialized reader - WithCustomLazyReader asks for a function to initialize a reader This function is useful if your reader can be initialized at the same time as the mux. WithCustomLazyReader is more powerful, but WithCustomReader is simpler to use and syntactically terse

func WithEnvReader

func WithEnvReader(opts ...func(*envReader)) func(*ConfigMux)

WithEnvReader adds an environment variable reader to the config mux

func WithEnviron

func WithEnviron(environ []string) func(*envReader)

WithEnviron allows specifying a custom environment slice for the envReader By default, it uses os.Environ()

func WithFileSystem

func WithFileSystem(fileSystem fs.FS) func(*yamlFileReader)

WithFileSystem allows specifying a custom file system for the yamlFileReader By default, it uses the OS file system, os.DirFS(".")

func WithPath

func WithPath(path string) func(*yamlFileReader)

WithPath allows specifying an additional path (file or directory) to read config from

func WithYAMLFileReader

func WithYAMLFileReader(path string, opts ...func(*yamlFileReader)) func(*ConfigMux)

WithYAMLFileReader adds a yaml file reader to the config mux

Types

type ConfigMux

type ConfigMux struct {
	// contains filtered or unexported fields
}

func NewConfigMux

func NewConfigMux(opts ...func(*ConfigMux)) *ConfigMux

NewConfigMux creates a new config mux which can read from multiple readers

func (*ConfigMux) Read

func (r *ConfigMux) Read() (ReadResult, error)

type DiagnosticReadResult

type DiagnosticReadResult struct {
	// contains filtered or unexported fields
}

DiagnosticReadResult is what you should return from a Read() function if your configuration source has diagnostics to report. For example, the BitwardenSecretReader uses this as a return value. If the BitwardenSecretReader realizes that it doesn't have enough configuration values to authenticate to Bitwarden, it fills the "diagnostics" map with information about what fields were missing.

func NewDiagnosticReadResult

func NewDiagnosticReadResult(configMap, diagnostics map[string]string) DiagnosticReadResult

NewDiagnosticReadResult creates a new DiagnosticReadResult

func (DiagnosticReadResult) GetConfigMap

func (r DiagnosticReadResult) GetConfigMap() map[string]string

type ReadResult

type ReadResult interface {
	GetConfigMap() map[string]string
	// contains filtered or unexported methods
}

ReadResult is the expected return value from the Read() function of a Reader You cannot define a struct outside of this package that implements this interface. You must use either NewSimpleReadResult or NewDiagnosticReadResult to initialize a value that implements ReadResult

type Reader

type Reader interface {
	Read() (ReadResult, error)
}

Reader is the interface that must be implemented if you want to define your own source of reading configuration data

type SimpleReadResult

type SimpleReadResult struct {
	// contains filtered or unexported fields
}

SimpleReadResult is what you should return from a Read() function if your configuration source doesn't have any diagnostics to report

func NewSimpleReadResult

func NewSimpleReadResult(configMap map[string]string) SimpleReadResult

NewSimpleReadResult creates a new SimpleReadResult

func (SimpleReadResult) GetConfigMap

func (r SimpleReadResult) GetConfigMap() map[string]string