templig

README

templig

templig (pronounced [ˈtɛmplɪç]) is a non-intrusive configuration library utilizing the text templating engine and the functions best known from helm charts, that originally stem from Masterminds/sprig. Its primary goal is to enable dynamic configuration files, that have access to the system environment to fill information using function like env and and read. To facilitate different environments, overlays can be defined, that amend a base configuration with environment specific attributes and changes. Configurations providing the methods for the Validator interface also have automated checking on load enabled.

Usage

Simple Case

Having a configuration file like the following:

id:   23
name: Interesting Name

The code to read that file would look like this:

package main

import (
	"fmt"

	"github.com/AlphaOne1/templig"
)

type Config struct {
	ID   int    `yaml:"id"`
	Name string `yaml:"name"`
}

func main() {
	c, confErr := templig.FromFile[Config]("my_config.yaml")

	fmt.Printf("read errors: %v", confErr)

	if confErr == nil {
		fmt.Printf("ID:   %v\n", c.Get().ID)
		fmt.Printf("Name: %v\n", c.Get().Name)
	}
}

The Get method gives a pointer to the internally held Config structure that the user supplied. The pointer is always non-nil, so additional nil-checks are not necessary. Running that program would give:

read errors: <nil>
ID:   23
Name: Interesting Name

Reading with Overlays

Having a base configuration file my_config.yaml like the following:

id:   23
name: Interesting DevName

and a file that contains specific configuration for e.g. the production environment my_prod_overlay.yaml:

name: Important ProdName

The code to read those files would look like this:

package main

import (
	"fmt"

	"github.com/AlphaOne1/templig"
)

type Config struct {
	ID   int    `yaml:"id"`
	Name string `yaml:"name"`
}

func main() {
	c, confErr := templig.FromFiles[Config]([]string{
		"my_config.yaml",
		"my_prod_overlay.yaml",
	})

	fmt.Printf("read errors: %v", confErr)

	if confErr == nil {
		fmt.Printf("ID:   %v\n", c.Get().ID)
		fmt.Printf("Name: %v\n", c.Get().Name)
	}
}

That way, the different configuration files are read in order, with he first one as the base. Every additional file gives changes to all the ones read before. In this example, changing the name. Running this program would give:

read errors: <nil>
ID:   23
Name: Important ProdName

As expected, the value of Name was replaced by the one provided in overlay configuration.

Template functionality

Overview

templig supports templating the configuration files. On top of the few templating functions provided by the Go text/template library, the functions of sprig, that are maybe best known for their use in helm charts. On top of that, the following functions are provided for convenience:

Function	Description	Example
arg	reads the value of the command line argument with the given name	Link
hasArg	true if an argument with the given name is present, false otherwise	Link
required	checks that its second argument is not zero length or nil	Link
read	reads the content of a file	Link

The expansion of the templated parts is done before overlaying takes place. Any errors of templating will thus be displayed in their respective source locations.

Reading environment

Having a templated configuration file like this one:

id:   23
name: Interesting Name
pass: {{ env "PASSWORD" | required "password required" | quote }}

or this one;

id:   23
name: Interesting Name
pass: {{ read "pass.txt" | required "password required" | quote }}

One can see the templating code between the double curly braces {{ and }}. The following programm is essentially the same as in the Simple Case. It just adds the pass field to the configuration:

package main

import (
	"fmt"
	"strings"

	"github.com/AlphaOne1/templig"
)

type Config struct {
	ID   int    `yaml:"id"`
	Name string `yaml:"name"`
	Pass string `yaml:"pass"`
}

func main() {
	c, confErr := templig.FromFile[Config]("my_config.yaml")

	fmt.Printf("read errors: %v", confErr)

	if confErr == nil {
		fmt.Printf("ID:   %v\n", c.Get().ID)
		fmt.Printf("Name: %v\n", c.Get().Name)
		fmt.Printf("Pass: %v\n", strings.Repeat("*", len(c.Get().Pass)))
	}
}

Validation

The templating facilities allow also for a wide range of tests, but depend on the configuration file read. As it is most likely user supplied, possible consistency checks are not reliable in the form of template code. For this purpose, templig also allows for the configuration structure to implement the Validator interface. Implementing types provide a function Validate that allows templig to check after the configuration was read, if its structure should be considered valid and report errors accordingly.

package main

import (
    "errors"
    "fmt"

	"github.com/AlphaOne1/templig"
)

type Config struct {
    ID   int    `yaml:"id"`
    Name string `yaml:"name"`
}

// Validate fulfills the Validator interface provided by templig.
// This method is called, if it is defined. It influences the outcome of the configuration reading.
func (c *Config) Validate() error {
    result := make([]error, 0)

	if len(c.Name) == 0 {
		result = append(result, errors.New("name is required"))
	}

	if c.ID < 0 {
		result = append(result, errors.New("id greater than zero is required"))
	}

	return errors.Join(result...)
}

func main() {
	c, confErr := templig.FromFile[Config]("my_config_good.yaml")

	if confErr == nil {
		fmt.Printf("ID:   %v\n", c.Get().ID)
		fmt.Printf("Name: %v\n", c.Get().Name)
	}
}

Validation functionality can be as simple as in this example. But as the complexity of the configuration grows, automated tools to generate the configuration structure and basic consistency checks could be employed. These use e.g. JSON Schema or its embedded form in OpenAPI 2 or 3.

A non-exhaustive list of these:

• https://github.com/omissis/go-jsonschema (JSON Schema)
• https://github.com/ogen-go/ogen (OpenAPI 3.x)
• https://github.com/go-swagger/go-swagger (OpenAPI 2.0 / Swagger 2.0)

Documentation

Index

• Constants
• Variables
• func HideSecrets(n *yaml.Node, hideStructure bool)
• func MergeYAMLNodes(a, b *yaml.Node) (*yaml.Node, error)
• type Config
  • func From[T any](r io.Reader) (*Config[T], error)
  • func FromFile[T any](path string) (*Config[T], error)
  • func FromFiles[T any](paths []string) (*Config[T], error)
  • func (c *Config[T]) Get() *T
  • func (c *Config[T]) To(w io.Writer) error
  • func (c *Config[T]) ToFile(path string) error
  • func (c *Config[T]) ToSecretsHidden(w io.Writer) error
  • func (c *Config[T]) ToSecretsHiddenStructured(w io.Writer) error
• type Validator

Constants

const SecretDefaultRE = "(key)|(secret)|(pass)|(password)|(cert)|(certificate)"

SecretDefaultRE is the default regular expression used to identify secret values automatically.

Variables

var SecretRE = regexp.MustCompile(SecretDefaultRE)

SecretRE is the regular expression used to identify secret values automatically. In case there are different properties to identify secrets, extend it.

Functions

func HideSecrets

func HideSecrets(n *yaml.Node, hideStructure bool)

HideSecrets hides secrets in the given YAML node structure. Secrets are identified using the SecretRE. Depending on the parameter `hideStructure` the structure of the secret is hidden too (`true`) or visible (`false`).

func MergeYAMLNodes

func MergeYAMLNodes(a, b *yaml.Node) (*yaml.Node, error)

MergeYAMLNodes merges the content of node `b` into node `a`. If `a` contains already an element with the same name and of the same kind as `b`, they are merged recursively.

Types

type Config

type Config[T any] struct {
	// contains filtered or unexported fields
}

Config is the generic structure holding the configuration information for the specified type.

func From

func From[T any](r io.Reader) (*Config[T], error)

From reads a configuration from the given io.Reader.

func FromFile

func FromFile[T any](path string) (*Config[T], error)

FromFile loads a configuration from a file with the given name.

func FromFiles

func FromFiles[T any](paths []string) (*Config[T], error)

FromFiles loads a series of configuration files. The first file is considered the base, all others are loaded on top of that one using the MergeYAMLNodes functionality.

func (*Config[T]) Get

func (c *Config[T]) Get() *T

Get gives a pointer to the deserialized configuration.

func (*Config[T]) To

func (c *Config[T]) To(w io.Writer) error

To writes a configuration to the given io.Writer.

func (*Config[T]) ToFile

func (c *Config[T]) ToFile(path string) error

ToFile saves a configuration to a file with the given name, replacing it in case.

func (*Config[T]) ToSecretsHidden

func (c *Config[T]) ToSecretsHidden(w io.Writer) error

ToSecretsHidden writes the configuration to the given io.Writer and hides secret values using the SecretRE. Strings are replaced with the number of * corresponding to their length. Substructures containing secrets, are replaced with a single '*'. The following example

id: id0
secrets:
  - secret0
  - secret1

thus will be replaced by

id: id0
secrets: *

func (*Config[T]) ToSecretsHiddenStructured

func (c *Config[T]) ToSecretsHiddenStructured(w io.Writer) error

ToSecretsHiddenStructured writes the configuration to the given io.Writer and hides secret values using the SecretRE. Strings are replaced with the number of * corresponding to their length. Substructures containing secrets, are replaced with a corresponding structure of '*'. The following example

id: id0
secrets:
  - secret0
  - secret1

thus will be replaced by

id: id0
secrets:
  - *******
  - *******

type Validator

type Validator interface {
	// Validate is used to validate a configuration.
	Validate() error
}

Validator is the interface to facility validity checks on configuration types.