v0.14.0 Latest Latest

This package is not in the latest version of its module.

Go to latest
Published: Jan 8, 2024 License: Apache-2.0 Imports: 11 Imported by: 109



Package markers contains utilities for defining and parsing "marker comments", also occasionally called tag comments (we use the term marker to avoid confusing with struct tags). Parsed result (output) values take the form of Go values, much like the "encoding/json" package.

Definitions and Parsing

Markers are defined as structured Definitions which can be used to consistently parse marker comments. A Definition contains an concrete output type for the marker, which can be a simple type (like string), a struct, or a wrapper type (useful for defining additional methods on marker types).

Markers take the general form




Arguments may be ints, bools, strings, and slices. Ints and bool take their standard form from Go. Strings may take any of their standard forms, or any sequence of unquoted characters up until a `,` or `;` is encountered. Lists take either of the following forms:


{val, val, val}

Note that the first form will not properly parse nested slices, but is generally convenient and is the form used in many existing markers.

Each of those argument types maps to the corresponding go type. Pointers mark optional fields (a struct tag, below, may also be used). The empty interface will match any type.

Struct fields may optionally be annotated with the `marker` struct tag. The first argument is a name override. If it's left blank (or the tag isn't present), the camelCase version of the name will be used. The only additional argument defined is `optional`, which marks a field as optional without using a pointer.

All parsed values are unmarshalled into the output type. If any non-optional fields aren't mentioned, an error will be raised unless `Strict` is set to false.

Registries and Lookup

Definitions can be added to registries to facilitate lookups. Each definition is marked as either describing a type, struct field, or package (unassociated). The same marker name may be registered multiple times, as long as each describes a different construct (type, field, or package). Definitions can then be looked up by passing unparsed markers.

Collection and Extraction

Markers can be collected from a loader.Package using a Collector. The Collector will read from a given Registry, collecting comments that look like markers and parsing them if they match some definition on the registry.

Markers are considered associated with a particular field or type if they exist in the Godoc, or the closest non-godoc comment. Any other markers not inside a some other block (e.g. a struct definition, interface definition, etc) are considered package level. Markers in a "closest non-Go comment block" may also be considered package level if registered as such and no identical type-level definition exists.

Like loader.Package, Collector's methods are idempotent and will not reperform work.


EachType function iterates over each type in a Package, providing conveniently structured type and field information with marker values associated.

PackageMarkers can be used to fetch just package-level markers.


Help can be defined for each marker using the DefinitionHelp struct. It's mostly intended to be generated off of godocs using cmd/helpgen, which takes the first line as summary (removing the type/field name), and considers the rest as details. It looks for the


marker to start generation.

If you can't use godoc-based generation for whatever reasons (e.g. primitive-typed markers), you can use the SimpleHelp and DeprecatedHelp helper functions to generate help structs.

Help is then registered into a registry as associated with the actual definition, and can then be later retrieved from the registry.



This section is empty.


This section is empty.


func EachType

func EachType(col *Collector, pkg *loader.Package, cb TypeCallback) error

EachType collects all markers, then calls the given callback for each type declaration in a package. Each individual spec is considered separate, so

type (
    Foo string
    Bar int
    Baz struct{}

yields three calls to the callback.

func RegisterAll

func RegisterAll(reg *Registry, defs ...*Definition) error

RegisterAll attempts to register all definitions against the given registry, stopping and returning if an error occurs.


type Argument

type Argument struct {
	// Type is the type of this argument For non-scalar types (map and slice),
	// further information is specified in ItemType.
	Type ArgumentType
	// Optional indicates if this argument is optional.
	Optional bool
	// Pointer indicates if this argument was a pointer (this is really only
	// needed for deserialization, and should alway imply optional)
	Pointer bool

	// ItemType is the type of the slice item for slices, and the value type
	// for maps.
	ItemType *Argument

Argument is the type of a marker argument.

func ArgumentFromType

func ArgumentFromType(rawType reflect.Type) (Argument, error)

ArgumentFromType constructs an Argument by examining the given raw reflect.Type. It can construct arguments from the Go types corresponding to any of the types listed in ArgumentType.

func (*Argument) Parse

func (a *Argument) Parse(scanner *sc.Scanner, raw string, out reflect.Value)

Parse attempts to consume the argument from the given scanner (based on the given raw input as well for collecting ranges of content), and places the output value in the given reflect.Value. Errors are reported via the given scanner.

func (Argument) String

func (a Argument) String() string

func (Argument) TypeString

func (a Argument) TypeString() string

TypeString returns a string roughly equivalent (but not identical) to the underlying Go type that this argument would parse to. It's mainly useful for user-friendly formatting of this argument (e.g. help strings).

type ArgumentType

type ArgumentType int

ArgumentType is the kind of a marker argument type. It's roughly analogous to a subset of reflect.Kind, with an extra "AnyType" to represent the empty interface.

const (
	// Invalid represents a type that can't be parsed, and should never be used.
	InvalidType ArgumentType = iota
	// IntType is an int
	// NumberType is a float64
	// StringType is a string
	// BoolType is a bool
	// AnyType is the empty interface, and matches the rest of the content
	// SliceType is any slice constructed of the ArgumentTypes
	// MapType is any map constructed of string keys, and ArgumentType values.
	// Keys are strings, and it's common to see AnyType (non-uniform) values.
	// RawType represents content that gets passed directly to the marker
	// without any parsing. It should *only* be used with anonymous markers.

type Collector

type Collector struct {
	// contains filtered or unexported fields

Collector collects and parses marker comments defined in the registry from package source code. If no registry is provided, an empty one will be initialized on the first call to MarkersInPackage.

func (*Collector) MarkersInPackage

func (c *Collector) MarkersInPackage(pkg *loader.Package) (map[ast.Node]MarkerValues, error)

MarkersInPackage computes the marker values by node for the given package. Results are cached by package ID, so this is safe to call repeatedly from different functions. Each file in the package is treated as a distinct node.

We consider a marker to be associated with a given AST node if either of the following are true:

- it's in the Godoc for that AST node

  • it's in the closest non-godoc comment group above that node, *and* that node is a type or field node, *and* [it's either registered as type-level *or* it's not registered as being package-level]

  • it's not in the Godoc of a node, doesn't meet the above criteria, and isn't in a struct definition (in which case it's package-level)

type Definition

type Definition struct {
	// Output is the deserialized Go type of the marker.
	Output reflect.Type
	// Name is the marker's name.
	Name string
	// Target indicates which kind of node this marker can be associated with.
	Target TargetType
	// Fields lists out the types of each field that this marker has, by
	// argument name as used in the marker (if the output type isn't a struct,
	// it'll have a single, blank field name).  This only lists exported fields,
	// (as per reflection rules).
	Fields map[string]Argument
	// FieldNames maps argument names (as used in the marker) to struct field name
	// in the output type.
	FieldNames map[string]string
	// Strict indicates that this definition should error out when parsing if
	// not all non-optional fields were seen.
	Strict bool

Definition is a parsed definition of a marker.

func MakeAnyTypeDefinition added in v0.2.2

func MakeAnyTypeDefinition(name string, target TargetType, output interface{}) (*Definition, error)

MakeAnyTypeDefinition constructs a definition for an output struct with a field named `Value` of type `interface{}`. The argument to the marker will be parsed as AnyType and assigned to the field named `Value`.

func MakeDefinition

func MakeDefinition(name string, target TargetType, output interface{}) (*Definition, error)

MakeDefinition constructs a definition from a name, type, and the output type. All such definitions are strict by default. If a struct is passed as the output type, its public fields will automatically be populated into Fields (and similar fields in Definition). Other values will have a single, empty-string-named Fields entry.

func Must

func Must(def *Definition, err error) *Definition

Must panics on errors creating definitions.

func (*Definition) AnonymousField

func (d *Definition) AnonymousField() bool

AnonymousField indicates that the definition has one field, (actually the original object), and thus the field doesn't get named as part of the name.

func (*Definition) Empty

func (d *Definition) Empty() bool

Empty indicates that this definition has no fields.

func (*Definition) Parse

func (d *Definition) Parse(rawMarker string) (interface{}, error)

Parse uses the type information in this Definition to parse the given raw marker in the form `+a:b:c=arg,d=arg` into an output object of the type specified in the definition.

type DefinitionHelp

type DefinitionHelp struct {
	// DetailedHelp contains the overall help for the marker.
	// Category describes what kind of marker this is.
	Category string
	// DeprecatedInFavorOf marks the marker as deprecated.
	// If non-nil & empty, it's assumed to just mean deprecated permanently.
	// If non-empty, it's assumed to be a marker name.
	DeprecatedInFavorOf *string

	// FieldHelp defines the per-field help for this marker, *in terms of the
	// go struct field names.  Use the FieldsHelp method to map this to
	// marker argument names.
	FieldHelp map[string]DetailedHelp

DefinitionHelp contains overall help for a marker Definition, as well as per-field help.

func DeprecatedHelp

func DeprecatedHelp(inFavorOf, category, summary string) *DefinitionHelp

DeprecatedHelp returns simple help (a la SimpleHelp), except marked as deprecated in favor of the given marker (or an empty string for just deprecated).

func SimpleHelp

func SimpleHelp(category, summary string) *DefinitionHelp

SimpleHelp returns help that just has marker-level summary information (e.g. for use with empty or primitive-typed markers, where Godoc-based generation isn't possible).

func (*DefinitionHelp) FieldsHelp

func (d *DefinitionHelp) FieldsHelp(def *Definition) map[string]DetailedHelp

FieldsHelp maps per-field help to the actual marker argument names from the given definition.

type DetailedHelp

type DetailedHelp struct {
	Summary string
	Details string

DetailedHelp contains brief help, as well as more details. For the "full" help, join the two together.

type FieldInfo

type FieldInfo struct {
	// Name is the name of the field (or "" for embedded fields)
	Name string
	// Doc is the Godoc of the field, pre-processed to remove markers and joine
	// single newlines together.
	Doc string
	// Tag struct tag associated with this field (or "" if non existed).
	Tag reflect.StructTag

	// Markers are all registered markers associated with this field.
	Markers MarkerValues

	// RawField is the raw, underlying field AST object that this field represents.
	RawField *ast.Field

FieldInfo contains marker values and commonly used information for a struct field.

type MarkerValues

type MarkerValues map[string][]interface{}

MarkerValues are all the values for some set of markers.

func PackageMarkers

func PackageMarkers(col *Collector, pkg *loader.Package) (MarkerValues, error)

PackageMarkers collects all the package-level marker values for the given package.

func (MarkerValues) Get

func (v MarkerValues) Get(name string) interface{}

Get fetches the first value that for the given marker, returning nil if no values are available.

type RawArguments

type RawArguments []byte

RawArguments is a special type that can be used for a marker to receive *all* raw, underparsed argument data for a marker. You probably want to use `interface{}` to match any type instead. Use *only* for legacy markers that don't follow Definition's normal parsing logic. It should *not* be used as a field in a marker struct.

type Registry

type Registry struct {
	// contains filtered or unexported fields

Registry keeps track of registered definitions, and allows for easy lookup. It's thread-safe, and the zero-value can be safely used.

func (*Registry) AddHelp

func (r *Registry) AddHelp(def *Definition, help *DefinitionHelp)

AddHelp stores the given help in the registry, marking it as associated with the given definition.

func (*Registry) AllDefinitions

func (r *Registry) AllDefinitions() []*Definition

AllDefinitions returns all marker definitions known to this registry.

func (*Registry) Define

func (r *Registry) Define(name string, target TargetType, obj interface{}) error

Define defines a new marker with the given name, target, and output type. It's a shortcut around

r.Register(MakeDefinition(name, target, obj))

func (*Registry) HelpFor

func (r *Registry) HelpFor(def *Definition) *DefinitionHelp

HelpFor fetches the help for a given definition, if present.

func (*Registry) Lookup

func (r *Registry) Lookup(name string, target TargetType) *Definition

Lookup fetches the definition corresponding to the given name and target type.

func (*Registry) Register

func (r *Registry) Register(def *Definition) error

Register registers the given marker definition with this registry for later lookup.

type ScannerError added in v0.2.2

type ScannerError struct {
	Msg string
	Pos sc.Position

func (*ScannerError) Error added in v0.2.2

func (e *ScannerError) Error() string

type TargetType

type TargetType int

TargetType describes which kind of node a given marker is associated with.

const (
	// DescribesPackage indicates that a marker is associated with a package.
	DescribesPackage TargetType = iota
	// DescribesType indicates that a marker is associated with a type declaration.
	// DescribesField indicates that a marker is associated with a struct field.

func (TargetType) String

func (t TargetType) String() string

type TypeCallback

type TypeCallback func(info *TypeInfo)

TypeCallback is a callback called for each type declaration in a package.

type TypeInfo

type TypeInfo struct {
	// Name is the name of the type.
	Name string
	// Doc is the Godoc of the type, pre-processed to remove markers and joine
	// single newlines together.
	Doc string

	// Markers are all registered markers associated with the type.
	Markers MarkerValues

	// Fields are all the fields associated with the type, if it's a struct.
	// (if not, Fields will be nil).
	Fields []FieldInfo

	// RawDecl contains the raw GenDecl that the type was declared as part of.
	RawDecl *ast.GenDecl
	// RawSpec contains the raw Spec that declared this type.
	RawSpec *ast.TypeSpec
	// RawFile contains the file in which this type was declared.
	RawFile *ast.File

TypeInfo contains marker values and commonly used information for a type declaration.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL