structs

package module
v0.2.0 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Jun 12, 2026 License: MIT Imports: 6 Imported by: 1

README

structs

Quality Go Reference GitHub Tag License

Use Go's structs to innovate

github.com/toaweme/structs gives you the tools to work with Go's structs, its fields, tags and values.

This module was originally built as a fun way to solve the CLI app arg parsing problem. I'm a big fan of simplicity and the stdlib while powerful, doesn't make CLI flag/arg parsing simple, there's a lot of boilerplate. structs abstracts the complicated bits and can magically set struct field values (however nested) from a simple map[string]any.

Module

  • structs.New a small abstraction to Validate and Set.
    • structs.WithTags a priority list of tags for Set (default: ["json", "yaml"]).
    • structs.WithEncodingTags a list of tags in which commas are treated as encoding configuration (e.g. json:"field,omitempty").
    • structs.WithRules extend or replace the built-in validation rules.
    • structs.WithValidationTag tag used to define the validation rules (default: rules)
  • structs.GetStructFields reads the entire nested struct field tree.
  • structs.SetStructFields takes a map[string]any and fills the struct fields.
  • structs.ValidateStructFields uses a rule map to validate your map[string]any against selected fields.

Overview

Struct embedding and nesting
Nesting (a named struct field)

Define your structs:

type Server struct {
	Database Database `json:"database" env:"DATABASE"`
}

type Database struct {
	URL string `json:"url" env:"URL"`
}

A nested field can be reached three ways, all equivalent:

// 1. dotted path: the field's tag glued to its parent's with "."
map[string]any{
	"database.url": "mysql://127.0.0.1:3306/beep",
}

// 2. nested map: a sub-section keyed by the parent's tag
map[string]any{
	"database": map[string]any{
		"url": "mysql://127.0.0.1:3306/beep",
	},
}

// 3. env tag: the env tags glued with "_"
map[string]any{
	"DATABASE_URL": "mysql://127.0.0.1:3306/beep",
}

Nesting goes arbitrarily deep (a.b.c, or maps within maps). This is how a decoded JSON/YAML config drops straight in.

Embedding (an anonymous struct field)

An untagged embedded struct has its fields promoted to the parent level, exactly as Go (and encoding/json) promote them: no wrapper, no prefix. The embedded type may be exported or unexported.

type Network struct {
	Host string `json:"host" env:"HOST"`
	Port int    `json:"port" env:"PORT"`
}

type Server struct {
	Network        // embedded: Host and Port are promoted
	Name string `json:"name"`
}

Set the promoted fields by their own tag or name, with no parent prefix:

map[string]any{
	"host": "127.0.0.1", // -> Server.Host
	"port": 8080,        // -> Server.Port
	"name": "edge",      // -> Server.Name
}

A tagged anonymous field is not promoted; it nests under its tag instead, just like encoding/json, so it behaves like the named nesting above.

Limitations
  • Nested maps must be map[string]any at every level (the form JSON/YAML decoders produce). A value whose concrete type is a typed map such as map[string]map[string]any is only descended into where its element type is map[string]any; a deeper typed-map intermediate is not traversed, so the leaf stays unset. Use the dotted path or a map[string]any sub-section.

Install

go get github.com/toaweme/structs

Features

  • Validate without mutating - check inputs against each field's rules and get back a map of field names with the validation messages

  • Populate from a single map - fill a struct from one map of values, matching each field and converting the value into the field's type.

  • Type coercion - string, int, float, bool, slice, map, and interface fields are all set from loosely typed inputs, so a port given as the string "9090" lands in an int field.

  • Tag priority - decide which struct tag names a field by giving an ordered list; the first tag a field carries wins. Defaults to json then yaml, and is overridable.

  • Defaults - a field left empty is seeded from its declared default value, and a default never overrides a value that is already present.

  • Built-in validation rules - required and one-of out of the box, with the ability to add your own named rules or replace the built-in set.

  • Slice splitting - a single string handed to a scalar slice field is split into elements (comma by default, or a custom separator per field) and each element is converted; already-structured inputs pass through untouched.

  • Nested structs - reach a field inside a nested struct by dotted path, by a nested map, or by an env-style key, to any depth.

  • Embedded structs - fields of an anonymous embedded struct are promoted and set directly, the way Go does it, whether the embedded type is exported or not.

This package does not read the env or any other value source. That's your responsibility.


Quickstart

type ServerConfig struct {
    Host     string   `json:"host" yaml:"host" default:"0.0.0.0"`
    Port     int      `json:"port" yaml:"port" env:"PORT" default:"8080" rules:"required"`
    LogLevel string   `json:"log_level" yaml:"log_level" default:"info" rules:"oneof:debug,info,warn,error"`
    Tags     []string `json:"tags" yaml:"tags" sep:","`
    Database Database `json:"database" yaml:"database"`
}

type Database struct {
	DSN string `json:"dsn" yaml:"dsn" env:"DATABASE_DSN" rules:"required"`
}

cfg := &ServerConfig{}
structManager := structs.New(cfg)

// it's your responsibility to collect the values
// inputs := merge(env(), config())
inputs := map[string]any{
	"host":      "127.0.0.1",        // matched by the json/yaml "host" tag
	"PORT":      "9090",             // matched by the env tag, coerced to int
	"log_level": "debug",            // matched by the "log_level" json tag
	"tags":      "edge,beta,canary", // split on sep into []string
	"database":  map[string]any{     // nested sub-section, matched by dotted path
		"dsn": "postgres://localhost/app",
	},
}

if errs, err := structManager.Validate(inputs); err != nil {
	log.Fatal(err)
} else if len(errs) > 0 {
	log.Fatalf("config is invalid: %v", errs)
}

if err := structManager.Set(inputs); err != nil {
	log.Fatal(err)
}
// cfg.Port == 9090, cfg.Tags == ["edge","beta","canary"], cfg.Database.DSN set.

Runnable examples

See example_test.go for the full, runnable versions of everything mentioned above.

go test -run Example -v

Documentation

Overview

Example

Example wires the whole thing together the way a real program would: build a manager once, validate the merged inputs, then populate the struct. The inputs map is deliberately heterogeneous: most keys came from a decoded config file (including a nested "database" sub-map), while "PORT" arrived under its env tag. structs resolves each against the right field.

package main

import (
	"fmt"

	"github.com/toaweme/structs"
)

// ServerConfig is the kind of struct you already have lying around: one set of
// fields, annotated once, that doubles as your config-file schema and your
// environment-variable contract. structs reads those tags and populates the
// struct from whatever input map you hand it, regardless of which source each
// value came from.
type ServerConfig struct {
	Host string `json:"host" yaml:"host" default:"0.0.0.0"`

	Port int `json:"port" yaml:"port" env:"PORT" default:"8080" rules:"required"`

	LogLevel string `json:"log_level" yaml:"log_level" default:"info" rules:"oneof:debug,info,warn,error"`

	Tags []string `json:"tags" yaml:"tags" sep:","`

	Database Database `json:"database" yaml:"database"`
}

type Database struct {
	DSN string `json:"dsn" yaml:"dsn" env:"DATABASE_DSN" rules:"required"`
}

func main() {
	cfg := &ServerConfig{}

	manager := structs.New(cfg)

	// in a real app this map is the merge of a decoded config file and os.Environ.
	inputs := map[string]any{
		"host":      "127.0.0.1",        // the "host" json/yaml tag
		"PORT":      "9090",             // the env tag, coerced string -> int
		"log_level": "debug",            // the "log_level" json tag
		"tags":      "edge,beta,canary", // split on sep into a []string
		"database": map[string]any{ // a nested config sub-section
			"dsn": "postgres://localhost/app",
		},
	}

	errs, err := manager.Validate(inputs)
	if err != nil {
		panic(err)
	}
	if len(errs) > 0 {
		fmt.Println("config is invalid:", errs)
		return
	}

	if err := manager.Set(inputs); err != nil {
		panic(err)
	}

	fmt.Printf("listen   %s:%d\n", cfg.Host, cfg.Port)
	fmt.Printf("loglevel %s\n", cfg.LogLevel)
	fmt.Printf("tags     %v\n", cfg.Tags)
	fmt.Printf("database %s\n", cfg.Database.DSN)
}
Output:
listen   127.0.0.1:9090
loglevel debug
tags     [edge beta canary]
database postgres://localhost/app
Example (CliArgs)

Example_cliArgs shows that structs is agnostic about where the input map comes from. It does not parse argv itself; you bring your own flag parser (here the toy cliArgsToMap), and structs matches the resulting keys against the arg/short tags. A bool flag with no value becomes true, and a string argument is coerced into the field's type.

package main

import (
	"fmt"
	"strings"

	"github.com/toaweme/structs"
)

func main() {
	type Flags struct {
		Output  string `arg:"output" short:"o" default:"table" rules:"oneof:table,json,yaml"`
		Verbose bool   `arg:"verbose" short:"v"`
		Limit   int    `arg:"limit" default:"50"`
	}

	values := cliArgsToMap([]string{"--output", "json", "-v", "--limit", "10"})

	flags := &Flags{}
	manager := structs.New(flags, structs.WithTags("arg", "short"))

	if errs, err := manager.Validate(values); err != nil {
		panic(err)
	} else if len(errs) > 0 {
		fmt.Println("invalid flags:", errs)
		return
	}
	if err := manager.Set(values); err != nil {
		panic(err)
	}

	fmt.Printf("output=%s verbose=%t limit=%d\n", flags.Output, flags.Verbose, flags.Limit)
}

// cliArgsToMap is a stand-in for whatever flag parser you already use: it turns
// raw argv into a map keyed by flag name (the leading dashes stripped). A flag
// followed by a non-flag token takes that token as its value; otherwise it is
// treated as a present boolean. structs never sees argv, only this map.
func cliArgsToMap(args []string) map[string]any {
	out := make(map[string]any)
	for i := 0; i < len(args); i++ {
		arg := args[i]
		if !strings.HasPrefix(arg, "-") {
			continue
		}
		name := strings.TrimLeft(arg, "-")
		if i+1 < len(args) && !strings.HasPrefix(args[i+1], "-") {
			out[name] = args[i+1]
			i++
			continue
		}
		out[name] = true
	}
	return out
}
Output:
output=json verbose=true limit=10
Example (Defaults)

Example_defaults shows what happens when the caller provides almost nothing: default tags fill the gaps, so a near-empty input map still yields a fully populated, runnable config. The nested DSN here is addressed by its dotted path, the flat-key alternative to a nested sub-map.

package main

import (
	"fmt"

	"github.com/toaweme/structs"
)

// ServerConfig is the kind of struct you already have lying around: one set of
// fields, annotated once, that doubles as your config-file schema and your
// environment-variable contract. structs reads those tags and populates the
// struct from whatever input map you hand it, regardless of which source each
// value came from.
type ServerConfig struct {
	Host string `json:"host" yaml:"host" default:"0.0.0.0"`

	Port int `json:"port" yaml:"port" env:"PORT" default:"8080" rules:"required"`

	LogLevel string `json:"log_level" yaml:"log_level" default:"info" rules:"oneof:debug,info,warn,error"`

	Tags []string `json:"tags" yaml:"tags" sep:","`

	Database Database `json:"database" yaml:"database"`
}

type Database struct {
	DSN string `json:"dsn" yaml:"dsn" env:"DATABASE_DSN" rules:"required"`
}

func main() {
	cfg := &ServerConfig{}
	manager := structs.New(cfg)

	inputs := map[string]any{
		"database.dsn": "postgres://localhost/app",
	}

	if errs, err := manager.Validate(inputs); err != nil {
		panic(err)
	} else if len(errs) > 0 {
		fmt.Println("config is invalid:", errs)
		return
	}
	if err := manager.Set(inputs); err != nil {
		panic(err)
	}

	fmt.Printf("%s:%d level=%s dsn=%s\n", cfg.Host, cfg.Port, cfg.LogLevel, cfg.Database.DSN)
}
Output:
0.0.0.0:8080 level=info dsn=postgres://localhost/app
Example (EnvironmentVariables)

Example_environmentVariables shows the same struct populated entirely from environment-style keys. Fields with an env tag pick their value up by that name, the nested DSN included, and defaults fill in whatever the environment did not set.

package main

import (
	"fmt"

	"github.com/toaweme/structs"
)

// ServerConfig is the kind of struct you already have lying around: one set of
// fields, annotated once, that doubles as your config-file schema and your
// environment-variable contract. structs reads those tags and populates the
// struct from whatever input map you hand it, regardless of which source each
// value came from.
type ServerConfig struct {
	Host string `json:"host" yaml:"host" default:"0.0.0.0"`

	Port int `json:"port" yaml:"port" env:"PORT" default:"8080" rules:"required"`

	LogLevel string `json:"log_level" yaml:"log_level" default:"info" rules:"oneof:debug,info,warn,error"`

	Tags []string `json:"tags" yaml:"tags" sep:","`

	Database Database `json:"database" yaml:"database"`
}

type Database struct {
	DSN string `json:"dsn" yaml:"dsn" env:"DATABASE_DSN" rules:"required"`
}

func main() {
	cfg := &ServerConfig{}
	manager := structs.New(cfg)

	// these are the names you would read out of os.Environ.
	env := map[string]any{
		"PORT":         "3000",
		"DATABASE_DSN": "postgres://db:5432/prod",
	}

	if err := manager.Set(env); err != nil {
		panic(err)
	}

	fmt.Printf("host=%s port=%d level=%s dsn=%s\n",
		cfg.Host, cfg.Port, cfg.LogLevel, cfg.Database.DSN)
}
Output:
host=0.0.0.0 port=3000 level=info dsn=postgres://db:5432/prod
Example (Validation)

Example_validation shows the validation side on its own. Validate never mutates the struct; it reports which fields failed which rules, keyed by the field's resolved tag name. Here Name is missing (required) and Format is not in the oneof set, so both rules report.

package main

import (
	"fmt"
	"sort"

	"github.com/toaweme/structs"
)

func main() {
	type Args struct {
		Name   string `json:"name" rules:"required"`
		Format string `json:"format" rules:"oneof:json,yaml,toml"`
	}

	manager := structs.New(&Args{}, structs.WithTags("json"))

	errs, err := manager.Validate(map[string]any{
		"format": "xml", // not one of json,yaml,toml
		// name omitted, so the required rule fires
	})
	if err != nil {
		panic(err)
	}

	// errs is a map[string][]string; sort the keys for deterministic output.
	fields := make([]string, 0, len(errs))
	for field := range errs {
		fields = append(fields, field)
	}
	sort.Strings(fields)

	for _, field := range fields {
		fmt.Printf("%s: %v\n", field, errs[field])
	}
}
Output:
format: [must be one of: json, yaml, toml]
name: [required]

Index

Examples

Constants

This section is empty.

Variables

View Source
var DefaultCLITags = []string{"arg", "short", "json", "yaml"}

DefaultCLITags is the default tag priority order for a cli app's command configuration structs

View Source
var DefaultEncodingTags = []string{"json", "yaml", "toml", "xml"}

DefaultEncodingTags are the struct tags whose values follow the stdlib convention of a name followed by comma-separated options (e.g. `json:"name,omitempty"`). Only these tags have their comma-options stripped when parsed; freeform tags (help, default, rules, ...) keep their value verbatim. Override to change which tags are treated as encoding tags.

View Source
var DefaultRules = map[string]RuleFunc{
	"required": Required,
	"oneof":    OneOf,
}

DefaultRules is the built-in rule set, keyed by the name used in a `rules:` tag. Pass it to New/ValidateStructFields, or build your own map (optionally extending this one) to register custom RuleFuncs.

View Source
var DefaultTags = []string{"json", "yaml"}

DefaultTags is the default tag priority order for input lookup and validation.

View Source
var ErrInputPointer = errors.New("structure should be a pointer")

ErrInputPointer is returned when the structure passed in is not a pointer.

View Source
var ErrInputPointerStruct = errors.New("structure should be a pointer to a struct")

ErrInputPointerStruct is returned when the structure passed in is a pointer but does not point to a struct.

Functions

func MapDefaultValues

func MapDefaultValues(fields []Field, values map[string]any, tagPriority ...string) map[string]any

MapDefaultValues returns a copy of values with each field's `default:` tag value filled in under the field's tag name, for fields that have no non-empty value yet. Defaults are keyed by the first matching tag in tagPriority.

func SetField

func SetField(field Field, settings Settings, inputs map[string]any) error

SetField sets a single field from inputs: it applies the field's default first, then looks up a value by env tag, exact field name, and tag priority (using the field's FQN for nested fields), honoring the override settings.

func SetFields

func SetFields(fields []Field, settings Settings, inputs map[string]any) error

SetFields sets each field in fields from inputs, recursing into nested structs. It is the recursive worker behind SetStructFields.

func SetStructFields

func SetStructFields(structure any, settings Settings, inputs map[string]any) error

SetStructFields sets the fields of a struct based on the inputs provided

func ValidateStructFields

func ValidateStructFields(ruleFuncs map[string]RuleFunc, structFields []Field, values map[string]any, validationTag string, tagPriority ...string) (map[string][]string, error)

ValidateStructFields runs each field's rules (looked up in ruleFuncs) against values and returns field name to error messages. Field names are resolved by tagPriority, then overridden by the validationTag value when a field carries one. An empty result means everything passed.

Types

type Field

type Field struct {
	// Name is the Go struct field name (e.g. "Field1").
	Name string
	// Type is the string form of Kind (e.g. "string", "struct").
	Type string
	// Tags holds the parsed struct tags as tag name to value (e.g. {"json": "field_1"}).
	Tags map[string]string
	// Default is the `default:` tag value, kept as a string because it comes
	// from the tag; it is converted to the field's type when the field is set.
	Default string
	// Kind is the field's reflect.Kind.
	Kind reflect.Kind
	// Value is the addressable reflect.Value backing the field, used to set it.
	Value reflect.Value
	// Rules are the validation rules parsed from the `rules:` tag.
	Rules []Rule
	// FQN is the fully-qualified view of a nested field: Name and Tags glued to
	// the parent's with "." (or "_" for the `env` tag). nil for top-level fields.
	FQN *Field
	// Parent points to the enclosing struct's field, nil for top-level fields.
	Parent *Field
	// Fields are the nested fields when Kind is reflect.Struct.
	Fields []Field
}

Field is the reflected description of one struct field, produced by GetStructFields. Nested struct fields are described recursively through Fields, and each nested field also carries an FQN giving its dotted path and glued tags relative to the root struct.

func GetStructFields

func GetStructFields(structure any, parent *Field, encodingTags []string) ([]Field, error)

GetStructFields reflects over structure (a pointer to a struct) and returns its fields as []Field, recursing into named nested structs and building each nested field's FQN (field.subfield.subsubfield). Embedded (anonymous) struct fields are promoted: their fields are returned inline at this level, with no wrapper field and no FQN, matching Go's own field promotion. encodingTags selects which tags get their comma options stripped (see DefaultEncodingTags). It returns ErrInputPointer or ErrInputPointerStruct when structure is not a pointer to a struct.

func NewField

func NewField(name string, dataType reflect.Kind, value reflect.Value, tags map[string]string, parentField *Field) Field

NewField builds a Field from a struct field's name, kind, value, and parsed tags, extracting the `default:` and `rules:` tags into Default and Rules.

type Option

type Option func(*Struct)

Option configures a Struct. See WithTags, WithEncodingTags, WithRules, WithValidationTag.

func WithEncodingTags

func WithEncodingTags(tags ...string) Option

WithEncodingTags sets the tags whose values use comma-separated options (see DefaultEncodingTags). Pass none to disable comma stripping entirely.

func WithRules

func WithRules(rules map[string]RuleFunc) Option

WithRules sets the rule set used by Validate, keyed by the name used in a `rules:` tag. Defaults to DefaultRules; pass your own map to extend or replace it.

func WithTags

func WithTags(tags ...string) Option

WithTags sets the tag priority order used for input lookup and validation.

func WithValidationTag added in v0.2.0

func WithValidationTag(tag string) Option

WithValidationTag sets the struct tag whose value is used as the field key in the map returned by Validate. When a field carries this tag, its value replaces the tag-priority-resolved name in the validation errors, letting you report errors under a stable, caller-facing name. Defaults to "rules".

type Rule

type Rule struct {
	// Name is the rule identifier, used to look up its RuleFunc in the rule set.
	Name string
	// Args are the colon-suffixed, comma-separated arguments, nil if none.
	Args []string
}

Rule is a single validation rule parsed from a `rules:` tag entry. For `rules:"oneof:json,yaml"` Name is "oneof" and Args is ["json", "yaml"].

type RuleFunc

type RuleFunc func(fieldName string, values map[string]any, defaultValue string, fieldValue reflect.Value, args []string) (map[string][]string, error)

RuleFunc validates one field against one rule. It receives the lookup key (fieldName, already resolved by tag priority), the full input values, the field's default, its reflect.Value, and the rule's args. It returns a map of field name to error messages (empty when valid). the returned error is for internal failures, not validation failures.

var OneOf RuleFunc = func(fieldName string, values map[string]any, defaultValue string, fieldValue reflect.Value, args []string) (map[string][]string, error) {
	value := ""
	if raw, ok := values[fieldName]; ok {
		if s, isStr := raw.(string); isStr {
			value = strings.TrimSpace(s)
		} else if raw != nil {
			value = fmt.Sprintf("%v", raw)
		}
	}
	if value == "" {
		value = defaultValue
	}
	if value == "" {

		return nil, nil
	}

	for _, allowed := range args {
		if value == allowed {

			return nil, nil
		}
	}

	return map[string][]string{
		fieldName: {"must be one of: " + strings.Join(args, ", ")},
	}, nil
}

OneOf restricts a field to one of the values listed in the rule args, e.g. `rules:"oneof:json,yaml,toml"`. An empty/absent value passes (pair with `required` to force presence); the default value, when set, is what gets checked if no input was provided.

var Required RuleFunc = func(fieldName string, values map[string]any, defaultValue string, fieldValue reflect.Value, args []string) (map[string][]string, error) {
	value, ok := values[fieldName]
	if !ok && defaultValue == "" {
		if fieldValue.IsValid() && !fieldValue.IsZero() {

			return nil, nil
		}
		errors := map[string][]string{
			fieldName: {"required"},
		}
		return errors, nil
	}

	if valStr, ok := value.(string); ok {
		value = strings.TrimSpace(valStr)
		if value == "" {
			value = defaultValue
		}
		if value == "" {
			errors := map[string][]string{
				fieldName: {"required"},
			}
			return errors, nil
		}
	}

	return nil, nil
}

Required fails when a field has no non-empty input and no default and the field's current value is zero. Whitespace-only string inputs count as empty.

type Settings

type Settings struct {
	// TagOrder is the tag priority used to match input keys to fields
	// the first tag in the list that a field carries wins.
	TagOrder []string
	// AllowEnvOverride toggles whether env vars can be overridden by tag inputs
	// env var handling takes priority over tag handling and is enabled by having a tag `env:"ENV_VAR"`
	// this toggles whether `env:"ENV_VAR"` can be overridden by `anything:"env_var"`
	AllowEnvOverride bool
	// AllowTagOverride toggles whether a later matching tag in TagOrder can
	// override a value already set by an earlier one.
	// if false, the first tag in TagOrder that matches wins and matching stops there.
	// if true, every matching tag is applied in order, so the last match wins.
	AllowTagOverride bool
	// EncodingTags are the tags whose values use comma-separated options (see
	// DefaultEncodingTags). Empty disables comma stripping.
	EncodingTags []string
}

Settings controls how SetStructFields resolves inputs onto struct fields.

type Struct

type Struct struct {
	// contains filtered or unexported fields
}

Struct holds the structure to be validated and the rules to validate it with

func New

func New(structure any, opts ...Option) *Struct

New binds a pointer to a struct into a reusable *Struct. structure must be a pointer to a struct. Options override the defaults: DefaultTags for tag priority, DefaultRules, DefaultEncodingTags, and the "rules" validation message tag.

func (*Struct) Set

func (m *Struct) Set(inputs map[string]any) error

Set populates the bound struct from inputs, resolving keys by tag priority and applying `default:` tag values to fields left zero.

func (*Struct) Validate

func (m *Struct) Validate(inputs map[string]any) (map[string][]string, error)

Validate runs the configured rules over inputs and returns the validation errors as a map of field name (resolved by tag priority, or the validation tag when present) to messages. An empty map means everything passed.

Directories

Path Synopsis
Package utils provides type-coercion helpers used to convert loosely-typed input values (from tags, env vars, decoded config) into concrete field types.
Package utils provides type-coercion helpers used to convert loosely-typed input values (from tags, env vars, decoded config) into concrete field types.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL