structs

README

structs

structs gives you the tools to work with Go's struct types.

• structs.GetStructFields reads the entire nested struct field tree.
• structs.SetStructFields takes a map[string]any and populates the struct.
• structs.ValidateStructFields uses a rule map to validate your map[string]any against each field.
• structs.New a small abstraction to Validate and Set.

Install

go get github.com/toaweme/structs

Features

• Validate without mutating - Validate(inputs) runs each field's rules: and returns a map[field][]messages; an empty map means everything passed.
• Populate from a map[string]any - Set(inputs) applies default: values then matches each field by tag, coercing the value into the field's type.
• Tag priority - matches by the first tag a field carries (default ["arg", "short", "json", "yaml"]), overridable with structs.WithTags(...).
• Defaults - default:"..." seeds empty fields.
• Built-in rules - required and oneof:a,b,c, extend or replace them with structs.WithRules(...).
• Slice splitting - a scalar slice fed one string is split on the field's sep tag (default ,) and each element coerced; structured inputs pass through.
• Nested and embedded structs - reach nested fields by dotted path, nested map, or env tag; embedded structs promote their fields like Go does.

This package does not read the env or any other value source. That's your responsibility.

---

Quickstart

type ServerConfig struct {
    Host     string   `json:"host" yaml:"host" default:"0.0.0.0"`
    Port     int      `json:"port" yaml:"port" env:"PORT" default:"8080" rules:"required"`
    LogLevel string   `json:"log_level" yaml:"log_level" default:"info" rules:"oneof:debug,info,warn,error"`
    Tags     []string `json:"tags" yaml:"tags" sep:","`
    Database Database `json:"database" yaml:"database"`
}

type Database struct {
	DSN string `json:"dsn" yaml:"dsn" env:"DATABASE_DSN" rules:"required"`
}

cfg := &ServerConfig{}
structManager := structs.New(cfg, structs.WithTags("json", "yaml"))

// it's your responsibility to collect the values
// inputs := merge(env(), config())
inputs := map[string]any{
	"host":      "127.0.0.1",        // matched by the json/yaml "host" tag
	"PORT":      "9090",             // matched by the env tag, coerced to int
	"log_level": "debug",            // matched by the "log_level" json tag
	"tags":      "edge,beta,canary", // split on sep into []string
	"database":  map[string]any{     // nested sub-section, matched by dotted path
		"dsn": "postgres://localhost/app",
	},
}

if errs, err := structManager.Validate(inputs); err != nil {
	log.Fatal(err)
} else if len(errs) > 0 {
	log.Fatalf("config is invalid: %v", errs)
}

if err := structManager.Set(inputs); err != nil {
	log.Fatal(err)
}
// cfg.Port == 9090, cfg.Tags == ["edge","beta","canary"], cfg.Database.DSN set.

Runnable examples

See example_test.go for the full, runnable versions of everything mentioned above.

go test -run Example -v

Documentation

Overview

Example

Example wires the whole thing together the way a real program would: build a manager once, validate the merged inputs, then populate the struct. The inputs map is deliberately heterogeneous: most keys came from a decoded config file (including a nested "database" sub-map), while "PORT" arrived under its env tag. structs resolves each against the right field.

package main

import (
	"fmt"

	"github.com/toaweme/structs"
)

// ServerConfig is the kind of struct you already have lying around: one set of
// fields, annotated once, that doubles as your config-file schema and your
// environment-variable contract. structs reads those tags and populates the
// struct from whatever input map you hand it, regardless of which source each
// value came from.
type ServerConfig struct {
	Host string `json:"host" yaml:"host" default:"0.0.0.0"`

	Port int `json:"port" yaml:"port" env:"PORT" default:"8080" rules:"required"`

	LogLevel string `json:"log_level" yaml:"log_level" default:"info" rules:"oneof:debug,info,warn,error"`

	Tags []string `json:"tags" yaml:"tags" sep:","`

	Database Database `json:"database" yaml:"database"`
}

type Database struct {
	DSN string `json:"dsn" yaml:"dsn" env:"DATABASE_DSN" rules:"required"`
}

func main() {
	cfg := &ServerConfig{}

	manager := structs.New(cfg, structs.WithTags("json", "yaml"))

	// in a real app this map is the merge of a decoded config file and os.Environ.
	inputs := map[string]any{
		"host":      "127.0.0.1",        // the "host" json/yaml tag
		"PORT":      "9090",             // the env tag, coerced string -> int
		"log_level": "debug",            // the "log_level" json tag
		"tags":      "edge,beta,canary", // split on sep into a []string
		"database": map[string]any{ // a nested config sub-section
			"dsn": "postgres://localhost/app",
		},
	}

	errs, err := manager.Validate(inputs)
	if err != nil {
		panic(err)
	}
	if len(errs) > 0 {
		fmt.Println("config is invalid:", errs)
		return
	}

	if err := manager.Set(inputs); err != nil {
		panic(err)
	}

	fmt.Printf("listen   %s:%d\n", cfg.Host, cfg.Port)
	fmt.Printf("loglevel %s\n", cfg.LogLevel)
	fmt.Printf("tags     %v\n", cfg.Tags)
	fmt.Printf("database %s\n", cfg.Database.DSN)
}

Output:
listen   127.0.0.1:9090
loglevel debug
tags     [edge beta canary]
database postgres://localhost/app

Example (CliArgs)

Example_cliArgs shows that structs is agnostic about where the input map comes from. It does not parse argv itself; you bring your own flag parser (here the toy cliArgsToMap), and structs matches the resulting keys against the arg/short tags. A bool flag with no value becomes true, and a string argument is coerced into the field's type.

package main

import (
	"fmt"
	"strings"

	"github.com/toaweme/structs"
)

func main() {
	type Flags struct {
		Output  string `arg:"output" short:"o" default:"table" rules:"oneof:table,json,yaml"`
		Verbose bool   `arg:"verbose" short:"v"`
		Limit   int    `arg:"limit" default:"50"`
	}

	values := cliArgsToMap([]string{"--output", "json", "-v", "--limit", "10"})

	flags := &Flags{}
	manager := structs.New(flags, structs.WithTags("arg", "short"))

	if errs, err := manager.Validate(values); err != nil {
		panic(err)
	} else if len(errs) > 0 {
		fmt.Println("invalid flags:", errs)
		return
	}
	if err := manager.Set(values); err != nil {
		panic(err)
	}

	fmt.Printf("output=%s verbose=%t limit=%d\n", flags.Output, flags.Verbose, flags.Limit)
}

// cliArgsToMap is a stand-in for whatever flag parser you already use: it turns
// raw argv into a map keyed by flag name (the leading dashes stripped). A flag
// followed by a non-flag token takes that token as its value; otherwise it is
// treated as a present boolean. structs never sees argv, only this map.
func cliArgsToMap(args []string) map[string]any {
	out := make(map[string]any)
	for i := 0; i < len(args); i++ {
		arg := args[i]
		if !strings.HasPrefix(arg, "-") {
			continue
		}
		name := strings.TrimLeft(arg, "-")
		if i+1 < len(args) && !strings.HasPrefix(args[i+1], "-") {
			out[name] = args[i+1]
			i++
			continue
		}
		out[name] = true
	}
	return out
}

Output:
output=json verbose=true limit=10

Example (Defaults)

Example_defaults shows what happens when the caller provides almost nothing: default tags fill the gaps, so a near-empty input map still yields a fully populated, runnable config. The nested DSN here is addressed by its dotted path, the flat-key alternative to a nested sub-map.

package main

import (
	"fmt"

	"github.com/toaweme/structs"
)

// ServerConfig is the kind of struct you already have lying around: one set of
// fields, annotated once, that doubles as your config-file schema and your
// environment-variable contract. structs reads those tags and populates the
// struct from whatever input map you hand it, regardless of which source each
// value came from.
type ServerConfig struct {
	Host string `json:"host" yaml:"host" default:"0.0.0.0"`

	Port int `json:"port" yaml:"port" env:"PORT" default:"8080" rules:"required"`

	LogLevel string `json:"log_level" yaml:"log_level" default:"info" rules:"oneof:debug,info,warn,error"`

	Tags []string `json:"tags" yaml:"tags" sep:","`

	Database Database `json:"database" yaml:"database"`
}

type Database struct {
	DSN string `json:"dsn" yaml:"dsn" env:"DATABASE_DSN" rules:"required"`
}

func main() {
	cfg := &ServerConfig{}
	manager := structs.New(cfg, structs.WithTags("json", "yaml"))

	inputs := map[string]any{
		"database.dsn": "postgres://localhost/app",
	}

	if errs, err := manager.Validate(inputs); err != nil {
		panic(err)
	} else if len(errs) > 0 {
		fmt.Println("config is invalid:", errs)
		return
	}
	if err := manager.Set(inputs); err != nil {
		panic(err)
	}

	fmt.Printf("%s:%d level=%s dsn=%s\n", cfg.Host, cfg.Port, cfg.LogLevel, cfg.Database.DSN)
}

Output:
0.0.0.0:8080 level=info dsn=postgres://localhost/app

Example (EnvironmentVariables)

Example_environmentVariables shows the same struct populated entirely from environment-style keys. Fields with an env tag pick their value up by that name, the nested DSN included, and defaults fill in whatever the environment did not set.

package main

import (
	"fmt"

	"github.com/toaweme/structs"
)

// ServerConfig is the kind of struct you already have lying around: one set of
// fields, annotated once, that doubles as your config-file schema and your
// environment-variable contract. structs reads those tags and populates the
// struct from whatever input map you hand it, regardless of which source each
// value came from.
type ServerConfig struct {
	Host string `json:"host" yaml:"host" default:"0.0.0.0"`

	Port int `json:"port" yaml:"port" env:"PORT" default:"8080" rules:"required"`

	LogLevel string `json:"log_level" yaml:"log_level" default:"info" rules:"oneof:debug,info,warn,error"`

	Tags []string `json:"tags" yaml:"tags" sep:","`

	Database Database `json:"database" yaml:"database"`
}

type Database struct {
	DSN string `json:"dsn" yaml:"dsn" env:"DATABASE_DSN" rules:"required"`
}

func main() {
	cfg := &ServerConfig{}
	manager := structs.New(cfg, structs.WithTags("json", "yaml"))

	// these are the names you would read out of os.Environ.
	env := map[string]any{
		"PORT":         "3000",
		"DATABASE_DSN": "postgres://db:5432/prod",
	}

	if err := manager.Set(env); err != nil {
		panic(err)
	}

	fmt.Printf("host=%s port=%d level=%s dsn=%s\n",
		cfg.Host, cfg.Port, cfg.LogLevel, cfg.Database.DSN)
}

Output:
host=0.0.0.0 port=3000 level=info dsn=postgres://db:5432/prod

Example (Validation)

Example_validation shows the validation side on its own. Validate never mutates the struct; it reports which fields failed which rules, keyed by the field's resolved tag name. Here Name is missing (required) and Format is not in the oneof set, so both rules report.

package main

import (
	"fmt"
	"sort"

	"github.com/toaweme/structs"
)

func main() {
	type Args struct {
		Name   string `json:"name" rules:"required"`
		Format string `json:"format" rules:"oneof:json,yaml,toml"`
	}

	manager := structs.New(&Args{}, structs.WithTags("json"))

	errs, err := manager.Validate(map[string]any{
		"format": "xml", // not one of json,yaml,toml
		// name omitted, so the required rule fires
	})
	if err != nil {
		panic(err)
	}

	// errs is a map[string][]string; sort the keys for deterministic output.
	fields := make([]string, 0, len(errs))
	for field := range errs {
		fields = append(fields, field)
	}
	sort.Strings(fields)

	for _, field := range fields {
		fmt.Printf("%s: %v\n", field, errs[field])
	}
}

Output:
format: [must be one of: json, yaml, toml]
name: [required]

Index

• Variables
• func MapDefaultValues(fields []Field, values map[string]any, tagPriority ...string) map[string]any
• func SetField(field Field, settings Settings, inputs map[string]any) error
• func SetFields(fields []Field, settings Settings, inputs map[string]any) error
• func SetStructFields(structure any, settings Settings, inputs map[string]any) error
• func ValidateStructFields(ruleFuncs map[string]RuleFunc, structFields []Field, values map[string]any, ...) (map[string][]string, error)
• type Field
  • func GetStructFields(structure any, parent *Field, encodingTags []string) ([]Field, error)
  • func NewField(name string, dataType reflect.Kind, value reflect.Value, ...) Field
• type Option
  • func WithEncodingTags(tags ...string) Option
  • func WithRules(rules map[string]RuleFunc) Option
  • func WithTags(tags ...string) Option
  • func WithValidationMessageTag(tag string) Option
• type Rule
• type RuleFunc
• type Settings
• type Struct
  • func New(structure any, opts ...Option) *Struct
  • func (m *Struct) Set(inputs map[string]any) error
  • func (m *Struct) Validate(inputs map[string]any) (map[string][]string, error)

Examples

• Package
• Package (CliArgs)
• Package (Defaults)
• Package (EnvironmentVariables)
• Package (Validation)

Variables

var DefaultEncodingTags = []string{"json", "yaml", "toml", "xml"}

DefaultEncodingTags are the struct tags whose values follow the stdlib convention of a name followed by comma-separated options (e.g. `json:"name,omitempty"`). Only these tags have their comma-options stripped when parsed; freeform tags (help, default, rules, ...) keep their value verbatim. Override to change which tags are treated as encoding tags.

var DefaultRules = map[string]RuleFunc{
	"required": Required,
	"oneof":    OneOf,
}

DefaultRules is the built-in rule set, keyed by the name used in a `rules:` tag. Pass it to New/ValidateStructFields, or build your own map (optionally extending this one) to register custom RuleFuncs.

var DefaultTags = []string{"arg", "short", "json", "yaml"}

DefaultTags is the default tag priority order for input lookup and validation.

var ErrInputPointer = errors.New("structure should be a pointer")

ErrInputPointer is returned when the structure passed in is not a pointer.

var ErrInputPointerStruct = errors.New("structure should be a pointer to a struct")

ErrInputPointerStruct is returned when the structure passed in is a pointer but does not point to a struct.

Functions

func MapDefaultValues

func MapDefaultValues(fields []Field, values map[string]any, tagPriority ...string) map[string]any

MapDefaultValues returns a copy of values with each field's `default:` tag value filled in under the field's tag name, for fields that have no non-empty value yet. Defaults are keyed by the first matching tag in tagPriority.

func SetField

func SetField(field Field, settings Settings, inputs map[string]any) error

SetField sets a single field from inputs: it applies the field's default first, then looks up a value by env tag, exact field name, and tag priority (using the field's FQN for nested fields), honoring the override settings.

func SetFields

func SetFields(fields []Field, settings Settings, inputs map[string]any) error

SetFields sets each field in fields from inputs, recursing into nested structs. It is the recursive worker behind SetStructFields.

func SetStructFields

func SetStructFields(structure any, settings Settings, inputs map[string]any) error

SetStructFields sets the fields of a struct based on the inputs provided

func ValidateStructFields

func ValidateStructFields(ruleFuncs map[string]RuleFunc, structFields []Field, values map[string]any, validationMessageTag string, tagPriority ...string) (map[string][]string, error)

ValidateStructFields runs each field's rules (looked up in ruleFuncs) against values and returns field name to error messages. Field names are resolved by tagPriority, then overridden by the validationMessageTag value when a field carries one. An empty result means everything passed.

Types

type Field

type Field struct {
	// Name is the Go struct field name (e.g. "Field1").
	Name string
	// Type is the string form of Kind (e.g. "string", "struct").
	Type string
	// Tags holds the parsed struct tags as tag name to value (e.g. {"json": "field_1"}).
	Tags map[string]string
	// Default is the `default:` tag value, kept as a string because it comes
	// from the tag; it is converted to the field's type when the field is set.
	Default string
	// Kind is the field's reflect.Kind.
	Kind reflect.Kind
	// Value is the addressable reflect.Value backing the field, used to set it.
	Value reflect.Value
	// Rules are the validation rules parsed from the `rules:` tag.
	Rules []Rule
	// FQN is the fully-qualified view of a nested field: Name and Tags glued to
	// the parent's with "." (or "_" for the `env` tag). nil for top-level fields.
	FQN *Field
	// Parent points to the enclosing struct's field, nil for top-level fields.
	Parent *Field
	// Fields are the nested fields when Kind is reflect.Struct.
	Fields []Field
}

Field is the reflected description of one struct field, produced by GetStructFields. Nested struct fields are described recursively through Fields, and each nested field also carries an FQN giving its dotted path and glued tags relative to the root struct.

func GetStructFields

func GetStructFields(structure any, parent *Field, encodingTags []string) ([]Field, error)

GetStructFields reflects over structure (a pointer to a struct) and returns its fields as []Field, recursing into named nested structs and building each nested field's FQN (field.subfield.subsubfield). Embedded (anonymous) struct fields are promoted: their fields are returned inline at this level, with no wrapper field and no FQN, matching Go's own field promotion. encodingTags selects which tags get their comma options stripped (see DefaultEncodingTags). It returns ErrInputPointer or ErrInputPointerStruct when structure is not a pointer to a struct.

func NewField

func NewField(name string, dataType reflect.Kind, value reflect.Value, tags map[string]string, parentField *Field) Field

NewField builds a Field from a struct field's name, kind, value, and parsed tags, extracting the `default:` and `rules:` tags into Default and Rules.

type Option

type Option func(*Struct)

Option configures a Struct. See WithTags, WithEncodingTags, WithRules, WithValidationMessageTag.

func WithEncodingTags

func WithEncodingTags(tags ...string) Option

WithEncodingTags sets the tags whose values use comma-separated options (see DefaultEncodingTags). Pass none to disable comma stripping entirely.

func WithRules

func WithRules(rules map[string]RuleFunc) Option

WithRules sets the rule set used by Validate, keyed by the name used in a `rules:` tag. Defaults to DefaultRules; pass your own map to extend or replace it.

func WithTags

func WithTags(tags ...string) Option

WithTags sets the tag priority order used for input lookup and validation.

func WithValidationMessageTag

func WithValidationMessageTag(tag string) Option

WithValidationMessageTag sets the struct tag whose value is used as the field key in the map returned by Validate. When a field carries this tag, its value replaces the tag-priority-resolved name in the validation messages, letting you report errors under a stable, caller-facing name. Defaults to "rules".

type Rule

type Rule struct {
	// Name is the rule identifier, used to look up its RuleFunc in the rule set.
	Name string
	// Args are the colon-suffixed, comma-separated arguments, nil if none.
	Args []string
}

Rule is a single validation rule parsed from a `rules:` tag entry. For `rules:"oneof:json,yaml"` Name is "oneof" and Args is ["json", "yaml"].

type RuleFunc

type RuleFunc func(fieldName string, values map[string]any, defaultValue string, fieldValue reflect.Value, args []string) (map[string][]string, error)

RuleFunc validates one field against one rule. It receives the lookup key (fieldName, already resolved by tag priority), the full input values, the field's default, its reflect.Value, and the rule's args. It returns a map of field name to error messages (empty when valid). the returned error is for internal failures, not validation failures.

var OneOf RuleFunc = func(fieldName string, values map[string]any, defaultValue string, fieldValue reflect.Value, args []string) (map[string][]string, error) {
	value := ""
	if raw, ok := values[fieldName]; ok {
		if s, isStr := raw.(string); isStr {
			value = strings.TrimSpace(s)
		} else if raw != nil {
			value = fmt.Sprintf("%v", raw)
		}
	}
	if value == "" {
		value = defaultValue
	}
	if value == "" {

		return nil, nil
	}

	for _, allowed := range args {
		if value == allowed {

			return nil, nil
		}
	}

	return map[string][]string{
		fieldName: {"must be one of: " + strings.Join(args, ", ")},
	}, nil
}

OneOf restricts a field to one of the values listed in the rule args, e.g. `rules:"oneof:json,yaml,toml"`. An empty/absent value passes (pair with `required` to force presence); the default value, when set, is what gets checked if no input was provided.

var Required RuleFunc = func(fieldName string, values map[string]any, defaultValue string, fieldValue reflect.Value, args []string) (map[string][]string, error) {
	value, ok := values[fieldName]
	if !ok && defaultValue == "" {
		if fieldValue.IsValid() && !fieldValue.IsZero() {

			return nil, nil
		}
		errors := map[string][]string{
			fieldName: {"required"},
		}
		return errors, nil
	}

	if valStr, ok := value.(string); ok {
		value = strings.TrimSpace(valStr)
		if value == "" {
			value = defaultValue
		}
		if value == "" {
			errors := map[string][]string{
				fieldName: {"required"},
			}
			return errors, nil
		}
	}

	return nil, nil
}

Required fails when a field has no non-empty input and no default and the field's current value is zero. Whitespace-only string inputs count as empty.

type Settings

type Settings struct {
	// TagOrder is the tag priority used to match input keys to fields
	// the first tag in the list that a field carries wins.
	TagOrder []string
	// AllowEnvOverride toggles whether env vars can be overridden by tag inputs
	// env var handling takes priority over tag handling and is enabled by having a tag `env:"ENV_VAR"`
	// this toggles whether `env:"ENV_VAR"` can be overridden by `anything:"env_var"`
	AllowEnvOverride bool
	// AllowTagOverride toggles whether a later matching tag in TagOrder can
	// override a value already set by an earlier one.
	// if false, the first tag in TagOrder that matches wins and matching stops there.
	// if true, every matching tag is applied in order, so the last match wins.
	AllowTagOverride bool
	// EncodingTags are the tags whose values use comma-separated options (see
	// DefaultEncodingTags). Empty disables comma stripping.
	EncodingTags []string
}

Settings controls how SetStructFields resolves inputs onto struct fields.

type Struct

type Struct struct {
	// contains filtered or unexported fields
}

Struct holds the structure to be validated and the rules to validate it with

func New

func New(structure any, opts ...Option) *Struct

New binds a pointer to a struct into a reusable *Struct. structure must be a pointer to a struct. Options override the defaults: no tag priority, DefaultRules, DefaultEncodingTags, and the "rules" validation message tag.

func (*Struct) Set

func (m *Struct) Set(inputs map[string]any) error

Set populates the bound struct from inputs, resolving keys by tag priority and applying `default:` tag values to fields left zero.

func (*Struct) Validate

func (m *Struct) Validate(inputs map[string]any) (map[string][]string, error)

Validate runs the configured rules over inputs and returns the validation errors as a map of field name (resolved by tag priority, or the validation tag when present) to messages. An empty map means everything passed.