structtags

README

StructTags

structtags provides a way of parsing struct tag Go fields.

The goal is to provide some ways to parse struct tags:

• Some projects need a full parsing (key, values)
• Some others only need the key and the raw value.
• Other projects need to escape the comma.
• Etc.

This is the first version of the module, and I want to extend it based on feedback so that the API can evolve and break.

Usage

---
title: What is the best parsing function for your context?
---
flowchart TB
    A([Need to keep the keys in order?]) -- yes --> B([Need the values to be split on commas?])
    B -- yes --> G(ParseToSliceValues)
    B -- no --> F(ParseToSlice)
    B -- both --> L(ParseToStructured)
    A -- no --> C([Need the values to be split on commas?])
    C -- yes --> I(ParseToMapValues)
    C -- no --> H([Using duplicated keys?])
    H -- yes --> J(ParseToMapMultikeys)
    H -- no --> K(ParseToMap)
    click G "https://github.com/ldez/structtags?tab=readme-ov-file#structtagsparsetoslicevaluestag-options" "ParseToSliceValues"
    click F "https://github.com/ldez/structtags?tab=readme-ov-file#structtagsparsetoslicetag-options" "ParseToSlice"
    click L "https://github.com/ldez/structtags?tab=readme-ov-file#structtagsparsetostructuredtag-options" "ParseToStructured"
    click I "https://github.com/ldez/structtags?tab=readme-ov-file#structtagsparsetomapvaluestag-options" "ParseToMapValues"
    click J "https://github.com/ldez/structtags?tab=readme-ov-file#structtagsparsetomapmultikeystag" "ParseToMapMultikeys"
    click K "https://github.com/ldez/structtags?tab=readme-ov-file#structtagsparsetomaptag-options" "ParseToMap"

structtags.ParseToMap(tag, ...options)

Parses a struct tag to a map[string]string.

Example

Options:

• WithDuplicateKeysMode:
  • DuplicateKeysIgnore (default)
  • DuplicateKeysDeny

structtags.ParseToMapValues(tag, ...options)

Parses a struct tag to a map[string][]string.

The value is split on a comma.

Example

Options:

• WithEscapeComma: Comma escaped by backslash.
• WithDuplicateKeysMode:
  • DuplicateKeysIgnore (default)
  • DuplicateKeysDeny
  • DuplicateKeysAllow (non-conventional, so not recommended)

structtags.ParseToMapMultikeys(tag)

NOT RECOMMENDED. For non-conventional tags where the key is repeated.

Parses a struct tag to a map[string][]string.

Example

structtags.ParseToSlice(tag, ...options)

Parses a struct tag to a slice of type Tag struct { Key, Value string }.

Example

Options:

• WithDuplicateKeysMode:
  • DuplicateKeysIgnore (default)
  • DuplicateKeysDeny
  • DuplicateKeysAllow (non-conventional, so not recommended)

structtags.ParseToSliceValues(tag, ...options)

Parses a struct tag to a slice of type Tag struct { Key string, Value []string }.

The value is split on a comma.

Example

Options:

• WithEscapeComma: Comma escaped by backslash.
• WithDuplicateKeysMode:
  • DuplicateKeysIgnore (default)
  • DuplicateKeysDeny
  • DuplicateKeysAllow (non-conventional, so not recommended)

structtags.ParseToStructured(tag, ...options)

Parses a struct tag to a *structured.Tag.

The value is split on a comma.

The value is parsed lazily: only if you call Entry.Values()

Example

Options:

• WithEscapeComma: Comma escaped by backslash.
• WithDuplicateKeysMode:
  • DuplicateKeysIgnore (default)
  • DuplicateKeysDeny
  • DuplicateKeysAllow (non-conventional, so not recommended)

structtags.ParseToFatih(tag, escapeComma)

Parses a struct tag to a *structtag.Tags.

The value is split on a comma.

Option: comma escaped by backslash.

Custom Parser

The parser package provides the tooling to parse a struct tag and its associated value.

To implement a custom parser, you can implement the parser.Filler interface.

Why this library?

fatih/structtag is a great library, but it's not extensible, it was build around JSON tags, and it was designed to modify tags.

For example, the Tag struct inside fatih/structtag that represents a tag is:

type Tag struct {
    // Key is the tag key, such as json, xml, etc..
    // i.e: `json:"foo,omitempty". Here key is: "json"
    Key string
    
    // Name is a part of the value
    // i.e: `json:"foo,omitempty". Here name is: "foo"
    Name string
    
    // Options is a part of the value. It contains a slice of tag options i.e:
    // `json:"foo,omitempty". Here options is: ["omitempty"]
    Options []string
}

Name and Options are related to JSON tags (and other marshaling/unmarshalling libraries).

But the first element in a struct tag value is not necessarily a name.

Example:

type Foo struct {
  Field1 float64  `minimum:"10.5" example:"20.6" required:"true"`
  Field2 string   `jsonschema:"required"`
  Field3 string   `description:"This is a description"`
}

Also, most projects don't need to modify the struct tags, but only to read them.

There are some limitations with fatih/structtag when there is comma inside the value.

Example:

type Foo struct {
  Field1 string   `regexp:"[a-z\\,.]"`
}

ldez/structtags provides straightforward ways to parse, read, or modify struct tags, and a compatibility layer with fatih/structtag if you need it.

Instead of rewriting the wheel for each project, I also provided a package with the plumbing:

• parser.Tag(): extracted from reflect.StructTag for the base parsing.
• parser.Value(): to parse the value (support optional comma escaping).

Notes

The struct tag specifications say that struct tags can be any string.

The key/value syntax, the comma separator, and the space separator are conventions based on reflect.StructTag and json implementation.

reflect.StructTag behaves like the struct tags are map[string]string, but with one difference: The first key always wins if there are multiple keys with the same name.

We can say that the reflect.StructTag doesn't support multiple keys with the same name. But some rare projects/libraries use multiple keys with the same name.

Also, the specification doesn't talk about comma escaping inside the value.

Maybe the specification should clarify those points.

References

• https://go.dev/ref/spec#Struct_types
• https://go.dev/ref/spec#string_lit
• https://pkg.go.dev/reflect#StructTag
• https://github.com/golang/go/blob/411c250d64304033181c46413a6e9381e8fe9b82/src/reflect/type.go#L1030-L1108
• https://github.com/golang/tools/blob/master/go/analysis/passes/structtag/structtag.go
• https://github.com/fatih/structtag/blob/master/tags.go

Documentation

Index

• func ParseToFatih(tag string, escapeComma bool) (*structtag.Tags, error)
• func ParseToMap(tag string, options ...mapsraw.Option) (mapsraw.Tag, error)
• func ParseToMapMultikeys(tag string) (mapsmultikeys.Tag, error)
• func ParseToMapValues(tag string, options ...mapsvalues.Option) (mapsvalues.Tag, error)
• func ParseToSlice(tag string, options ...sliceraw.Option) (sliceraw.Tags, error)
• func ParseToSliceValues(tag string, options ...slicevalues.Option) (slicevalues.Tags, error)
• func ParseToStructured(tag string, options ...structured.Option) (*structured.Tag, error)

Examples

• ParseToMap
• ParseToMapMultikeys
• ParseToMapValues
• ParseToMapValues (Escaped_comma)
• ParseToSlice
• ParseToSliceValues
• ParseToSliceValues (Escaped_comma)
• ParseToStructured

Functions

func ParseToFatih

func ParseToFatih(tag string, escapeComma bool) (*structtag.Tags, error)

ParseToFatih parses a struct tag to a *structtag.Tags. The value is split on comma.

func ParseToMap

func ParseToMap(tag string, options ...mapsraw.Option) (mapsraw.Tag, error)

ParseToMap parses a struct tag to a `map[string]string`. Ignore duplicated keys by default.

Example

package main

import (
	"fmt"
	"reflect"

	"github.com/ldez/structtags"
)

func main() {
	type MyStruct struct {
		Field string `a:"1,2" b:"hello"`
	}

	// Gets the raw tag from the struct field.
	rawTag := reflect.TypeOf(MyStruct{}).Field(0).Tag

	data, err := structtags.ParseToMap(string(rawTag))
	if err != nil {
		panic(err)
	}

	// cast to map only to have a deterministic output for the example.
	fmt.Println(map[string]string(data))

}

Output:

map[a:1,2 b:hello]

func ParseToMapMultikeys

func ParseToMapMultikeys(tag string) (mapsmultikeys.Tag, error)

ParseToMapMultikeys parses a struct tag to a `map[string][]string`. For non-conventional tags where the key is repeated.

Example

package main

import (
	"fmt"
	"reflect"

	"github.com/ldez/structtags"
)

func main() {
	type MyStruct struct {
		Field string `a:"1,2" b:"hello" b:"world"`
	}

	// Gets the raw tag from the struct field.
	rawTag := reflect.TypeOf(MyStruct{}).Field(0).Tag

	data, err := structtags.ParseToMapMultikeys(string(rawTag))
	if err != nil {
		panic(err)
	}

	// cast to map only to have a deterministic output for the example.
	fmt.Println(map[string][]string(data))

}

Output:

map[a:[1,2] b:[hello world]]

func ParseToMapValues

func ParseToMapValues(tag string, options ...mapsvalues.Option) (mapsvalues.Tag, error)

ParseToMapValues parses a struct tag to a `map[string][]string`. The value is split on comma. Ignore duplicated keys by default.

Example

package main

import (
	"fmt"
	"reflect"

	"github.com/ldez/structtags"
)

func main() {
	type MyStruct struct {
		Field string `a:"1,2" b:"hello\\,world"`
	}

	// Gets the raw tag from the struct field.
	rawTag := reflect.TypeOf(MyStruct{}).Field(0).Tag

	data, err := structtags.ParseToMapValues(string(rawTag))
	if err != nil {
		panic(err)
	}

	// Cast to map only to display the raw map as output for the example.
	fmt.Println(map[string][]string(data))

}

Output:

map[a:[1 2] b:[hello\ world]]

Example (Escaped_comma)

package main

import (
	"fmt"
	"reflect"

	"github.com/ldez/structtags"
	mapsvalues "github.com/ldez/structtags/variant/maps/values"
)

func main() {
	type MyStruct struct {
		Field string `a:"1,2" b:"hello\\,world"`
	}

	// Gets the raw tag from the struct field.
	rawTag := reflect.TypeOf(MyStruct{}).Field(0).Tag

	data, err := structtags.ParseToMapValues(string(rawTag), mapsvalues.WithEscapeComma())
	if err != nil {
		panic(err)
	}

	// Cast to map only to display the raw map as output for the example.
	fmt.Println(map[string][]string(data))

}

Output:

map[a:[1 2] b:[hello\,world]]

func ParseToSlice

func ParseToSlice(tag string, options ...sliceraw.Option) (sliceraw.Tags, error)

ParseToSlice parses a struct tag to a slice of sliceraw.Tag. Ignore duplicated keys by default.

Example

package main

import (
	"fmt"
	"reflect"

	"github.com/ldez/structtags"
)

func main() {
	type MyStruct struct {
		Field string `a:"1,2" b:"hello"`
	}

	// Gets the raw tag from the struct field.
	rawTag := reflect.TypeOf(MyStruct{}).Field(0).Tag

	data, err := structtags.ParseToSlice(string(rawTag))
	if err != nil {
		panic(err)
	}

	for _, datum := range data {
		fmt.Println(datum)
	}

}

Output:

{a 1,2}
{b hello}

func ParseToSliceValues

func ParseToSliceValues(tag string, options ...slicevalues.Option) (slicevalues.Tags, error)

ParseToSliceValues parses a struct tag to a slice of slicevalues.Tag. The value is split on comma. Ignore duplicated keys by default.

Example

package main

import (
	"fmt"
	"reflect"

	"github.com/ldez/structtags"
)

func main() {
	type MyStruct struct {
		Field string `a:"1,2" b:"hello\\,world"`
	}

	// Gets the raw tag from the struct field.
	rawTag := reflect.TypeOf(MyStruct{}).Field(0).Tag

	data, err := structtags.ParseToSliceValues(string(rawTag))
	if err != nil {
		panic(err)
	}

	for _, datum := range data {
		fmt.Println(datum)
	}

}

Output:

{a [1 2]}
{b [hello\ world]}

Example (Escaped_comma)

package main

import (
	"fmt"
	"reflect"

	"github.com/ldez/structtags"

	slicevalues "github.com/ldez/structtags/variant/slices/values"
)

func main() {
	type MyStruct struct {
		Field string `a:"1,2" b:"hello\\,world"`
	}

	// Gets the raw tag from the struct field.
	rawTag := reflect.TypeOf(MyStruct{}).Field(0).Tag

	data, err := structtags.ParseToSliceValues(string(rawTag), slicevalues.WithEscapeComma())
	if err != nil {
		panic(err)
	}

	for _, datum := range data {
		fmt.Println(datum)
	}

}

Output:

{a [1 2]}
{b [hello\,world]}

func ParseToStructured

func ParseToStructured(tag string, options ...structured.Option) (*structured.Tag, error)

ParseToStructured parses a struct tag to a structured.Tag. Allows modifying the struct tags. The value is split on comma. Ignore duplicated keys by default.

Example

package main

import (
	"fmt"
	"reflect"

	"github.com/ldez/structtags"
	"github.com/ldez/structtags/variant/structured"
)

func main() {
	type MyStruct struct {
		Field string `b:"hello" a:"1,2" c:"world"`
	}

	// Gets the raw tag from the struct field.
	rawTag := reflect.TypeOf(MyStruct{}).Field(0).Tag

	tag, err := structtags.ParseToStructured(string(rawTag))
	if err != nil {
		panic(err)
	}

	// Iterates over all entries from the struct tag.
	for entry := range tag.Seq() {
		fmt.Printf("entry: %s\n", entry)
	}

	// get a single tag
	entryA := tag.Get("a")
	if entryA == nil {
		panic("no entry with key a")
	}

	fmt.Println("key `a`, entry:", entryA)
	fmt.Println("key `a`, entry key:", entryA.Key)
	fmt.Println("key `a`, entry value:", entryA.RawValue)

	// change existing tag
	values, err := entryA.Values()
	if err != nil {
		panic(err)
	}

	values = append(values, "test")

	entryA.RawValue = values.String()

	// Adds a new entry to the struct tag.
	err = tag.Add(&structured.Entry{
		Key:      "e",
		RawValue: "foo,bar",
	})
	if err != nil {
		panic(err)
	}

	fmt.Println("tag:", tag)

	// Sorts the entries.
	tag.Sort()

	fmt.Println("sorted tag:", tag)

}

Output:

entry: b="hello"
entry: a="1,2"
entry: c="world"
key `a`, entry: a="1,2"
key `a`, entry key: a
key `a`, entry value: 1,2
tag: b="hello" a="1,2,test" c="world" e="foo,bar"
sorted tag: a="1,2,test" b="hello" c="world" e="foo,bar"