xmlstruct

README

go-xmlstruct

Generate Go structs from multiple XML documents.

What does go-xmlstruct do and why should I use it?

go-xmlstruct generates Go structs from XML documents. Alternatively put, go-xmlstruct infers XML schemas from one or more example XML documents. For example, given this XML document, go-xmlstruct generates this Go source code.

Compared to existing Go struct generators like zek, XMLGen, and chidley, go-xmlstruct:

• Takes multiple XML documents as input.
• Generates field types of bool, int, string, or time.Time as appropriate.
• Creates named types for all elements.
• Handles optional attributes and elements.
• Handles repeated attributes and elements.
• Ignores empty chardata.
• Provides a CLI for simple use.
• Can be used as a Go package for advanced use, including configurable field naming.

go-xmlstruct is useful for quick-and-dirty unmarshalling of arbitrary XML documents, especially when you have no schema or the schema is extremely complex and you want something that "just works" with the documents you have.

Note the bug in Go's encoding/xml:

Mapping between XML elements and data structures is inherently flawed: an XML element is an order-dependent collection of anonymous values, while a data structure is an order-independent collection of named values. See encoding/json for a textual representation more suitable to data structures.

Install

Install the goxmlstruct CLI with:

$ go install github.com/twpayne/go-xmlstruct/cmd/goxmlstruct@latest

Example

Feed goxmlstruct the simple XML document:

<parent>
  <child flag="true">
    text
  </child>
</parent>

by running:

$ echo '<parent><child flag="true">text</child></parent>' | goxmlstruct

This produces the output:

// Code generated by goxmlstruct. DO NOT EDIT.

package main

type Parent struct {
        Child struct {
                Flag     bool   `xml:"flag,attr"`
                CharData string `xml:",chardata"`
        } `xml:"child"`
}

This demonstrates:

• A Go struct is generated from the structure of the input XML document.
• Attributes, child elements, and chardata are all considered.
• Field names are generated automatically.
• Field types are detected automatically.

For a full list of options to the goxmlstruct CLI run:

$ goxmlstruct -help

You can run a more advanced example with:

$ git clone https://github.com/twpayne/go-xmlstruct.git
$ cd go-xmlstruct
$ goxmlstruct internal/tests/gpx/testdata/*.gpx

This demonstrates generating a Go struct from multiple complex XML documents.

For an example of configurable field naming and named types by using go-xmlstruct as a package, see internal/tests/rss/rss_test.go.

For an example of a complex schema, see internal/tests/aixm/aixm_test.go.

How does go-xmlstruct work?

Similar to go-jsonstruct, go-xmlstruct consists of two phases:

1. Firstly, go-xmlstruct explores all input XML documents to determine their structure. It gathers statistics on the types used for each attribute, chardata, and child element.
2. Secondly, go-xmlstruct generates a Go struct based on the observed structure using the gathered statistics to determine the type of each field.

License

MIT

Documentation

Overview

Package xmlstruct generates Go structs from multiple XML documents.

Index

• Constants
• Variables
• type ExportNameFunc
• type Generator
  • func NewGenerator(options ...GeneratorOption) *Generator
  • func (g *Generator) Generate() ([]byte, error)
  • func (g *Generator) ObserveFS(fsys fs.FS, root string, observeFunc func(string, fs.DirEntry, error) error) error
  • func (g *Generator) ObserveFile(name string) error
  • func (g *Generator) ObserveReader(r io.Reader) error
• type GeneratorOption
  • func WithAttrNameSuffix(attrSuffix string) GeneratorOption
  • func WithCharDataFieldName(charDataFieldName string) GeneratorOption
  • func WithCompactTypes(compactTypes bool) GeneratorOption
  • func WithElemNameSuffix(elemNameSuffix string) GeneratorOption
  • func WithEmptyElements(emptyElements bool) GeneratorOption
  • func WithExportNameFunc(exportNameFunc ExportNameFunc) GeneratorOption
  • func WithExportRenames(exportRenames map[string]string) GeneratorOption
  • func WithExportTypeNameFunc(exportTypeNameFunc ExportNameFunc) GeneratorOption
  • func WithFormatSource(formatSource bool) GeneratorOption
  • func WithHeader(header string) GeneratorOption
  • func WithImports(withImports bool) GeneratorOption
  • func WithIntType(intType string) GeneratorOption
  • func WithModifyDecoderFunc(modifyDecoderFunc ModifyDecoderFunc) GeneratorOption
  • func WithNameFunc(nameFunc NameFunc) GeneratorOption
  • func WithNamedRoot(namedRoot bool) GeneratorOption
  • func WithNamedTypes(namedTypes bool) GeneratorOption
  • func WithPackageName(packageName string) GeneratorOption
  • func WithPreserveOrder(preserveOrder bool) GeneratorOption
  • func WithTimeLayout(timeLayout string) GeneratorOption
  • func WithTopLevelAttributes(topLevelAttributes bool) GeneratorOption
  • func WithUsePointersForOptionalFields(usePointersForOptionalFields bool) GeneratorOption
  • func WithUseRawToken(useRawToken bool) GeneratorOption
• type ModifyDecoderFunc
• type NameFunc

Constants

const (
	DefaultAttrNameSuffix               = ""
	DefaultCharDataFieldName            = "CharData"
	DefaultElemNameSuffix               = ""
	DefaultFormatSource                 = true
	DefaultHeader                       = "// Code generated by goxmlstruct. DO NOT EDIT."
	DefaultTopLevelAttributes           = false
	DefaultImports                      = true
	DefaultIntType                      = "int"
	DefaultNamedRoot                    = false
	DefaultNamedTypes                   = false
	DefaultCompactTypes                 = false
	DefaultPackageName                  = "main"
	DefaultPreserveOrder                = false
	DefaultTimeLayout                   = "2006-01-02T15:04:05Z"
	DefaultUsePointersForOptionalFields = true
	DefaultUseRawToken                  = false
	DefaultEmptyElements                = true
)

Variables

var (
	SkipDir = fs.SkipDir //nolint:errname
	//lint:ignore ST1012 SkipFile is not an error
	SkipFile = errors.New("skip file") //nolint:errname,revive,staticcheck
)

var (
	// TitleFirstRuneExportNameFunc returns name.Local with the initial rune
	// capitalized.
	TitleFirstRuneExportNameFunc = func(name xml.Name) string {
		runes := []rune(name.Local)
		runes[0] = unicode.ToUpper(runes[0])
		return string(runes)
	}

	// DefaultExportNameFunc returns name.Local with kebab- and snakecase words
	// converted to UpperCamelCase and any Id suffix converted to ID.
	DefaultExportNameFunc = func(name xml.Name) string {
		localName := kebabOrSnakeCaseWordBoundaryRx.ReplaceAllStringFunc(name.Local, func(s string) string {
			return strings.ToUpper(s[len(s)-1:])
		})
		localName = nonIdentifierRuneRx.ReplaceAllLiteralString(localName, "_")
		runes := []rune(localName)
		runes[0] = unicode.ToUpper(runes[0])
		if len(runes) > 1 && runes[len(runes)-2] == 'I' && runes[len(runes)-1] == 'd' {
			runes[len(runes)-1] = 'D'
		}
		return string(runes)
	}

	// DefaultUnexportNameFunc returns name.Local with kebab- and snakecase words
	// converted to lowerCamelCase
	// Any ID prefix is converted to id, and any Id suffix converted to ID.
	DefaultUnexportNameFunc = func(name xml.Name) string {
		localName := kebabOrSnakeCaseWordBoundaryRx.ReplaceAllStringFunc(name.Local, func(s string) string {
			return strings.ToUpper(s[len(s)-1:])
		})
		localName = nonIdentifierRuneRx.ReplaceAllLiteralString(localName, "_")
		runes := []rune(localName)
		runes[0] = unicode.ToLower(runes[0])
		if len(runes) > 1 {
			if runes[len(runes)-2] == 'I' && runes[len(runes)-1] == 'd' {
				runes[len(runes)-1] = 'D'
			}
			if runes[0] == 'i' && runes[1] == 'D' {
				runes[1] = 'd'
			}
		}
		return string(runes)
	}
)

var (
	// IgnoreNamespaceNameFunc returns name with name.Space cleared. The same
	// local name in different namespaces will be treated as identical names.
	IgnoreNamespaceNameFunc = func(name xml.Name) xml.Name {
		return xml.Name{
			Local: name.Local,
		}
	}

	// The IdentityNameFunc returns name unchanged. The same local name in
	// different namespaces will be treated as distinct names.
	IdentityNameFunc = func(name xml.Name) xml.Name {
		return name
	}

	DefaultNameFunc = IgnoreNamespaceNameFunc
)

Types

type ExportNameFunc

type ExportNameFunc func(xml.Name) string

An ExportNameFunc returns the exported Go identifier for the given xml.Name.

type Generator

type Generator struct {
	// contains filtered or unexported fields
}

A Generator observes XML documents and generates Go structs into which the XML documents can be unmarshalled.

func NewGenerator

func NewGenerator(options ...GeneratorOption) *Generator

NewGenerator returns a new Generator with the given options.

func (*Generator) Generate

func (g *Generator) Generate() ([]byte, error)

Generate returns the generated Go source for all the XML documents observed so far.

func (*Generator) ObserveFS

func (g *Generator) ObserveFS(fsys fs.FS, root string, observeFunc func(string, fs.DirEntry, error) error) error

ObserveFS observes all files in fs.

observeFunc is called before each entry. If observeFunc returns fs.SkipDir or SkipFile then the entry is skipped. If observeFunc returns any other non-nil error then ObserveFS terminates with the returned error. If observe func returns nil and the entry is a regular file or symlink then it is observed, otherwise the entry is ignored.

func (*Generator) ObserveFile

func (g *Generator) ObserveFile(name string) error

ObserveFile observes an XML document in the given file.

func (*Generator) ObserveReader

func (g *Generator) ObserveReader(r io.Reader) error

ObserveReader observes an XML document from r.

type GeneratorOption

type GeneratorOption func(*Generator)

A GeneratorOption sets an option on a Generator.

func WithAttrNameSuffix

func WithAttrNameSuffix(attrSuffix string) GeneratorOption

WithAttrNameSuffix sets the attribute suffix.

func WithCharDataFieldName

func WithCharDataFieldName(charDataFieldName string) GeneratorOption

WithCharDataFieldName sets the char data field name.

func WithCompactTypes

func WithCompactTypes(compactTypes bool) GeneratorOption

WithCompactTypes sets whether to generate compact types.

func WithElemNameSuffix

func WithElemNameSuffix(elemNameSuffix string) GeneratorOption

WithElemNameSuffix sets the element name suffix.

func WithEmptyElements

func WithEmptyElements(emptyElements bool) GeneratorOption

WithEmptyElements sets whether to use type struct{} or string for empty xml elements.

func WithExportNameFunc

func WithExportNameFunc(exportNameFunc ExportNameFunc) GeneratorOption

WithExportNameFunc sets the export name function for the generated Go source. It overrides WithExportRenames.

func WithExportRenames

func WithExportRenames(exportRenames map[string]string) GeneratorOption

WithExportRenames sets the export renames. It is overridden by WithExportRenameFunc.

func WithExportTypeNameFunc

func WithExportTypeNameFunc(exportTypeNameFunc ExportNameFunc) GeneratorOption

WithExportTypeNameFunc sets the export name function for the generated Go source Types. This is useful when unexported types are desired.

func WithFormatSource

func WithFormatSource(formatSource bool) GeneratorOption

WithFormatSource sets whether to format the generated Go source.

func WithHeader

func WithHeader(header string) GeneratorOption

WithHeader sets the header of the generated Go source.

func WithImports

func WithImports(withImports bool) GeneratorOption

WithImports sets whether to include an import statement in the generated code.

func WithIntType

func WithIntType(intType string) GeneratorOption

WithIntType sets the int type in the generated Go source.

func WithModifyDecoderFunc

func WithModifyDecoderFunc(modifyDecoderFunc ModifyDecoderFunc) GeneratorOption

WithModifyDecoderFunc sets the function that will modify the encoding/xml.Decoder used.

func WithNameFunc

func WithNameFunc(nameFunc NameFunc) GeneratorOption

WithNameFunc sets the name function.

func WithNamedRoot

func WithNamedRoot(namedRoot bool) GeneratorOption

WithNamedRoot sets whether to generate an XMLName field for the root element.

func WithNamedTypes

func WithNamedTypes(namedTypes bool) GeneratorOption

WithNamedTypes sets whether all to generate named types for all elements.

func WithPackageName

func WithPackageName(packageName string) GeneratorOption

WithPackageName sets the package name of the generated Go source.

func WithPreserveOrder

func WithPreserveOrder(preserveOrder bool) GeneratorOption

WithPreserveOrder sets whether to preserve the order of types and fields.

func WithTimeLayout

func WithTimeLayout(timeLayout string) GeneratorOption

WithTimeLayout sets the time layout used to identify times in the observed XML documents. Use an empty string to disable identifying times.

func WithTopLevelAttributes

func WithTopLevelAttributes(topLevelAttributes bool) GeneratorOption

WithTopLevelAttributes sets whether to include top level attributes.

func WithUsePointersForOptionalFields

func WithUsePointersForOptionalFields(usePointersForOptionalFields bool) GeneratorOption

WithUsePointersForOptionFields sets whether to use pointers for optional fields in the generated Go source.

func WithUseRawToken

func WithUseRawToken(useRawToken bool) GeneratorOption

WithUseRawToken sets whether to use encoding/xml.Decoder.Token or encoding/xml.Decoder.RawToken.

type ModifyDecoderFunc

type ModifyDecoderFunc func(*xml.Decoder)

A ModifyDecoderFunc makes arbitrary changes to an encoding/xml.Decoder before it is used.

type NameFunc

type NameFunc func(xml.Name) xml.Name

A NameFunc modifies xml.Names observed in the XML documents.