structor

README

structor

Uses EL in Go struct tags to compute struct fields.

In a nutshell

Basic idea is to use simple expression language (EL) within Go struct tags to compute struct fields based on other fields or provided additional context.

It can be viewed/used as an advanced struct builder or constructor. Hence the name. Another possible use case is a "struct walker" or "struct field visitor".

Implementation uses reflection and EL interpretation, so, it is relatively slow by design, and, because of that, should be used for infrequent tasks. For example, during program initialization. Or in cases when all other alternatives are equally slow.

Initial implementation uses "text/template" as EL, it get fields name, value, tags, full struct and extra context as a "dot" context and either use special "set" custom function to save new field's value as object or just stores result of the template evaluation as string into the field.

Map of the custom functions can be passed to EL engine for use in the expressions. "set" is a special custom EL function which stores argument into the currently processed field. Special case: if annotated field is struct and result can not be converted to it, then result stored into .Sub, and evaluation executed recursively on the inner struct.

Another special custom EL function is "eval". This function allows to invoke interpreter from within EL expression to evaluate another expression. Example usage is when expression of interest come from another field or extra context. For instance, it can come from configuration file.

Simple namespaces and "import" mechanism provided for custom functions - whole map of custom functions can be combined from several maps with (optional) custom prefixes, prepended to every function name within map. Maps for provided functions placed into separate Go packages and can be imported as needed. When used with Glide, they can be imported as subpackages, allowing to optimize dependencies.

It's simpler to illustrate on examples, see examples in tests (funcs_test.go, goel_funcs_test.go) and Godoc.

Possible use cases

• take a struct, filled up from configuration file using github.com/spf13/viper and compute additional fields:

  • replace referencies in settings to their values (for example base URL in othre URLs); also: templates in the settings themeselves, not in the tags of structure;
  • load short text files into string fields (with file names from config);
  • decode passwords;
  • extract data from environment variables, listed in config;
  • execute bash scripts to compute fields (iconv, openssl, etc);
  • parse complex types from string representation;

• use engines like regexp, xpath or goquery to extract pieces of data from text, xml, html etc. formats into fields (scraping data from html pages, text files or emails); for long and complex expressions it can be convenient to use multiline tags and process whole tag value as single expression;

• registering a custom EL "interpreter", which executes arbitrary custom logic in its Execute() method, it is possible to use structor as a "struct walker" or "field visitor", which will traverse structure fields and invoke custom function for every field, containing custom tag; it can be convenient with ability to use multiline tags;

Ideas of functions, available in expressions

• atoi
• base64/unbase64
• exec - invoke external process (shell, for instance)
• encrypt/decrypt
• env
• eval
• fields
• readFile - read file content to []byte or string
• goquery
• hex/unhex
• match
• math (basic arithmetics)
• replace
• rot13
• set
• split
• standard for package "text/template"
• trimSpace
• xpath
• ... (custom)

Other ideas

• go expressions (using "github.com/apaxa-go/eval")

Godoc

Godoc

Documentation

Overview

Package structor contains interface and default implementation of EL (expression language) evaluator which evaluates EL expressions within tags of the struct, using optional additional struct as an extra context.

Basic idea is to use simple expression language within Go struct tags to compute struct fields based on other fields or provided additional context.

Due usage of reflection and EL interpretation, this package is hardly suitable for tasks, requiring high performance, but rather intended to be used during application setup or in cases where high performance is not an ultimate goal.

See tests for examples of usage with xpath, regexp, goquery, encryption, reading from files, shell invocation, etc.

Example

Example is an example of structor's usage.

Whole struct tag string is used for EL expression. Interpreter based on "text/template" used to interpret it. Custom math and strings functions are used in expressions.

package main

import (
	"fmt"

	"github.com/nikolay-turpitko/structor"
	"github.com/nikolay-turpitko/structor/el"
	"github.com/nikolay-turpitko/structor/funcs/math"

	funcs_strings "github.com/nikolay-turpitko/structor/funcs/strings"
	"github.com/nikolay-turpitko/structor/funcs/use"
	"github.com/nikolay-turpitko/structor/scanner"
)

func main() {
	ev := structor.NewEvaluator(
		scanner.Default,
		structor.Interpreters{
			structor.WholeTag: &el.DefaultInterpreter{
				AutoEnclose: true,
				Funcs: use.Packages(
					use.Pkg{Funcs: math.Pkg},
					use.Pkg{Funcs: funcs_strings.Pkg},
				),
			},
		})
	type theStruct struct {
		A int    `set 40`
		B int    `set 2`
		C int    `add .Struct.A .Struct.B | set`
		D string `"structor" | upper`
	}
	v := &theStruct{}
	err := ev.Eval(v, nil)
	if err != nil {
		panic(err)
	}
	fmt.Println(v.C)
	fmt.Println(v.D)

}

Output:

42
STRUCTOR

Example (Complex)

Example_complex is a more complex example of structor's usage.

Two interpreters are registered to process correspondent struct's tags. Custom strings functions are registered with custom prefix and name translations. EL uses value from other tag via execution context. One interpreter calls another to evaluate expression, got from some tag. Custom quotation marks are used in tags, when it's convenient.

package main

import (
	"fmt"
	"strings"

	"github.com/apaxa-go/eval"

	"github.com/nikolay-turpitko/structor"
	"github.com/nikolay-turpitko/structor/el"

	goel "github.com/nikolay-turpitko/structor/el/go-el"

	funcs_strings "github.com/nikolay-turpitko/structor/funcs/strings"
	"github.com/nikolay-turpitko/structor/funcs/use"
	"github.com/nikolay-turpitko/structor/scanner"
)

func main() {
	ev := structor.NewEvaluator(
		scanner.Default,
		structor.Interpreters{
			"tmplEL": &el.DefaultInterpreter{
				AutoEnclose: true,
				Funcs:       use.Packages(use.Pkg{Prefix: "str", MapName: strings.Title, Funcs: funcs_strings.Pkg}),
			},
			"goEL": &goel.Interpreter{
				Args: eval.ArgsFromInterfaces(use.Packages(
					use.Pkg{Prefix: "strings.", MapName: strings.Title, Funcs: funcs_strings.Pkg},
				)),
			},
		})
	type theStruct struct {
		A string `tmplEL:".Tags.arg | strUpper" arg:"structor"`
		B string `goEL:'strings.Upper(ctx.Tags["arg"])' arg:"structor"`
		C int    `tmplEL:'.Tags.expr | eval "goEL" | set' expr:"40+2"`
		D int    `goEL:'eval("tmplEL", ctx.Tags["expr"])' expr:'"42" | strAtoi | set'`
	}
	v := &theStruct{}
	err := ev.Eval(v, nil)
	if err != nil {
		panic(err)
	}
	fmt.Println(v.A)
	fmt.Println(v.B)
	fmt.Println(v.C)
	fmt.Println(v.D)

}

Output:

STRUCTOR
STRUCTOR
42
42

Index

• Constants
• func AddressableCopy(s interface{}) interface{}
• func DeepCopy(s interface{}) interface{}
• type Evaluator
  • func NewDefaultEvaluator(funcs use.FuncMap) Evaluator
  • func NewEvaluator(scanner scanner.Scanner, interpreters Interpreters) Evaluator
  • func NewEvaluatorWithOptions(scanner scanner.Scanner, interpreters Interpreters, options Options) Evaluator
  • func NewNonmutatingEvaluator(scanner scanner.Scanner, interpreters Interpreters) Evaluator
• type Interpreters
• type Options

Examples

• Package
• Package (Complex)

Constants

const WholeTag = ""

WholeTag constant can be used as tag name in the Interpreters to indicate that whole tag value should be passed to the interpreter.

Registering interpreter to the whole tag value conflicts with any other usage of the struct's tag, but can be convenient to simplify complex EL expressions with quotes (regexp, for example).

WholeTag interpreter is probed after all other registered interpreters.

Functions

func AddressableCopy

func AddressableCopy(s interface{}) interface{}

AddressableCopy returns a pointer to the addressable copy of the struct.

func DeepCopy

func DeepCopy(s interface{}) interface{}

DeepCopy returns a pointer to the deep copy of the struct. This is utility function to create a copy of the struct so that it would be possible to use the copy with structor without corruption of the original struct. It should handle pointers, slices and maps so that full independent copy would be created. Copied struct should not have any references back to the original struct. BUG: Current implementation doesn't copy non-exported fields.

Types

type Evaluator

type Evaluator interface {
	Eval(s, extra interface{}) error
}

Evaluator is an interface of evaluator, which gets structure and extra context as input, iterates over `s`'s fields and evaluate expression tag on every field.

func NewDefaultEvaluator

func NewDefaultEvaluator(funcs use.FuncMap) Evaluator

NewDefaultEvaluator returns default Evaluator implementation. Default implementation uses tag "eval" for expressions and EL interpreter, based on `"text/template"`.

funcs - custom functions, available for interpreter;

func NewEvaluator

func NewEvaluator(
	scanner scanner.Scanner,
	interpreters Interpreters) Evaluator

NewEvaluator returns Evaluator with desired settings. It invokes NewEvaluatorWithOptions with default options.

func NewEvaluatorWithOptions

func NewEvaluatorWithOptions(
	scanner scanner.Scanner,
	interpreters Interpreters,
	options Options) Evaluator

NewEvaluatorWithOptions returns Evaluator with desired settings.

Only first tag with EL will be recognized and used (only one expression per struct field). Different fields of the same struct can be processed using different EL interpreters.

scanner - is a scanner implementation to be used to scan tags.
interpreters - is a map of registered tag names to EL interpreters.
options - is an Options structure.

func NewNonmutatingEvaluator

func NewNonmutatingEvaluator(
	scanner scanner.Scanner,
	interpreters Interpreters) Evaluator

NewNonmutatingEvaluator creates Evaluator implementation which does not change original structure (does not save evaluated results) itself. Though, interpreters can change structures' fields as a side effect. In this case Evaluator can be used as a visitor of fields with tags for which it have registered interpreters. It will invoke registered interpreter for field with corresponded tag. Interpreter can then manipulate it's own state or el.Context. For example, it can store processing results into context's Extra field.

See NewEvaluator() for additional information.

type Interpreters

type Interpreters map[string]el.Interpreter

Interpreters is a map of tag names to el.Interpreters. Used to register different interpreters for different tag names.

Only first tag name on the struct field is currently recognized and processed. So, only one EL expression per structure field, but different fields of the same structure can be processed by different interpreters.

type Options

type Options struct {

	// NonMutating creates non-mutating Evaluator, see NewNonmutatingEvaluator.
	NonMutating bool

	// EvalEmptyTags causes Evaluator to invoke Interpreter for fields with
	// empty tags.
	EvalEmptyTags bool
}

Options is an options to create Evaluator.