toml

package module
Version: v2.0.2 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Jun 8, 2022 License: MIT Imports: 18 Imported by: 106

README

go-toml v2

Go library for the TOML format.

This library supports TOML v1.0.0.

🐞 Bug Reports

💬 Anything else

Documentation

Full API, examples, and implementation notes are available in the Go documentation.

Go Reference

Import

import "github.com/pelletier/go-toml/v2"

See Modules.

Features

Stdlib behavior

As much as possible, this library is designed to behave similarly as the standard library's encoding/json.

Performance

While go-toml favors usability, it is written with performance in mind. Most operations should not be shockingly slow. See benchmarks.

Strict mode

Decoder can be set to "strict mode", which makes it error when some parts of the TOML document was not present in the target structure. This is a great way to check for typos. See example in the documentation.

Contextualized errors

When most decoding errors occur, go-toml returns DecodeError), which contains a human readable contextualized version of the error. For example:

2| key1 = "value1"
3| key2 = "missing2"
 | ~~~~ missing field
4| key3 = "missing3"
5| key4 = "value4"
Local date and time support

TOML supports native local date/times. It allows to represent a given date, time, or date-time without relation to a timezone or offset. To support this use-case, go-toml provides LocalDate, LocalTime, and LocalDateTime. Those types can be transformed to and from time.Time, making them convenient yet unambiguous structures for their respective TOML representation.

Getting started

Given the following struct, let's see how to read it and write it as TOML:

type MyConfig struct {
      Version int
      Name    string
      Tags    []string
}
Unmarshaling

Unmarshal reads a TOML document and fills a Go structure with its content. For example:

doc := `
version = 2
name = "go-toml"
tags = ["go", "toml"]
`

var cfg MyConfig
err := toml.Unmarshal([]byte(doc), &cfg)
if err != nil {
      panic(err)
}
fmt.Println("version:", cfg.Version)
fmt.Println("name:", cfg.Name)
fmt.Println("tags:", cfg.Tags)

// Output:
// version: 2
// name: go-toml
// tags: [go toml]
Marshaling

Marshal is the opposite of Unmarshal: it represents a Go structure as a TOML document:

cfg := MyConfig{
      Version: 2,
      Name:    "go-toml",
      Tags:    []string{"go", "toml"},
}

b, err := toml.Marshal(cfg)
if err != nil {
      panic(err)
}
fmt.Println(string(b))

// Output:
// Version = 2
// Name = 'go-toml'
// Tags = ['go', 'toml']

Benchmarks

Execution time speedup compared to other Go TOML libraries:

Benchmarkgo-toml v1BurntSushi/toml
Marshal/HugoFrontMatter-21.9x1.9x
Marshal/ReferenceFile/map-21.7x1.8x
Marshal/ReferenceFile/struct-22.2x2.5x
Unmarshal/HugoFrontMatter-22.9x2.9x
Unmarshal/ReferenceFile/map-22.6x2.9x
Unmarshal/ReferenceFile/struct-24.4x5.3x
See more

The table above has the results of the most common use-cases. The table below contains the results of all benchmarks, including unrealistic ones. It is provided for completeness.

Benchmarkgo-toml v1BurntSushi/toml
Marshal/SimpleDocument/map-21.8x2.9x
Marshal/SimpleDocument/struct-22.7x4.2x
Unmarshal/SimpleDocument/map-24.5x3.1x
Unmarshal/SimpleDocument/struct-26.2x3.9x
UnmarshalDataset/example-23.1x3.5x
UnmarshalDataset/code-22.3x3.1x
UnmarshalDataset/twitter-22.5x2.6x
UnmarshalDataset/citm_catalog-22.1x2.2x
UnmarshalDataset/canada-21.6x1.3x
UnmarshalDataset/config-24.3x3.2x
[Geo mean]2.7x2.8x

This table can be generated with ./ci.sh benchmark -a -html.

Modules

go-toml uses Go's standard modules system.

Installation instructions:

  • Go ≥ 1.16: Nothing to do. Use the import in your code. The go command deals with it automatically.
  • Go ≥ 1.13: GO111MODULE=on go get github.com/pelletier/go-toml/v2.

In case of trouble: Go Modules FAQ.

Tools

Go-toml provides three handy command line tools:

  • tomljson: Reads a TOML file and outputs its JSON representation.

    $ go install github.com/pelletier/go-toml/v2/cmd/tomljson@latest
    $ tomljson --help
    
  • jsontoml: Reads a JSON file and outputs a TOML representation.

    $ go install github.com/pelletier/go-toml/v2/cmd/jsontoml@latest
    $ jsontoml --help
    
  • tomll: Lints and reformats a TOML file.

    $ go install github.com/pelletier/go-toml/v2/cmd/tomll@latest
    $ tomll --help
    
Docker image

Those tools are also available as a Docker image. For example, to use tomljson:

docker run -i ghcr.io/pelletier/go-toml:v2 tomljson < example.toml

Multiple versions are availble on ghcr.io.

Migrating from v1

This section describes the differences between v1 and v2, with some pointers on how to get the original behavior when possible.

Decoding / Unmarshal
Automatic field name guessing

When unmarshaling to a struct, if a key in the TOML document does not exactly match the name of a struct field or any of the toml-tagged field, v1 tries multiple variations of the key (code).

V2 instead does a case-insensitive matching, like encoding/json.

This could impact you if you are relying on casing to differentiate two fields, and one of them is a not using the toml struct tag. The recommended solution is to be specific about tag names for those fields using the toml struct tag.

Ignore preexisting value in interface

When decoding into a non-nil interface{}, go-toml v1 uses the type of the element in the interface to decode the object. For example:

type inner struct {
  B interface{}
}
type doc struct {
  A interface{}
}

d := doc{
  A: inner{
    B: "Before",
  },
}

data := `
[A]
B = "After"
`

toml.Unmarshal([]byte(data), &d)
fmt.Printf("toml v1: %#v\n", d)

// toml v1: main.doc{A:main.inner{B:"After"}}

In this case, field A is of type interface{}, containing a inner struct. V1 sees that type and uses it when decoding the object.

When decoding an object into an interface{}, V2 instead disregards whatever value the interface{} may contain and replaces it with a map[string]interface{}. With the same data structure as above, here is what the result looks like:

toml.Unmarshal([]byte(data), &d)
fmt.Printf("toml v2: %#v\n", d)

// toml v2: main.doc{A:map[string]interface {}{"B":"After"}}

This is to match encoding/json's behavior. There is no way to make the v2 decoder behave like v1.

Values out of array bounds ignored

When decoding into an array, v1 returns an error when the number of elements contained in the doc is superior to the capacity of the array. For example:

type doc struct {
  A [2]string
}
d := doc{}
err := toml.Unmarshal([]byte(`A = ["one", "two", "many"]`), &d)
fmt.Println(err)

// (1, 1): unmarshal: TOML array length (3) exceeds destination array length (2)

In the same situation, v2 ignores the last value:

err := toml.Unmarshal([]byte(`A = ["one", "two", "many"]`), &d)
fmt.Println("err:", err, "d:", d)
// err: <nil> d: {[one two]}

This is to match encoding/json's behavior. There is no way to make the v2 decoder behave like v1.

Support for toml.Unmarshaler has been dropped

This method was not widely used, poorly defined, and added a lot of complexity. A similar effect can be achieved by implementing the encoding.TextUnmarshaler interface and use strings.

Support for default struct tag has been dropped

This feature adds complexity and a poorly defined API for an effect that can be accomplished outside of the library.

It does not seem like other format parsers in Go support that feature (the project referenced in the original ticket #202 has not been updated since 2017). Given that go-toml v2 should not touch values not in the document, the same effect can be achieved by pre-filling the struct with defaults (libraries like go-defaults can help). Also, string representation is not well defined for all types: it creates issues like #278.

The recommended replacement is pre-filling the struct before unmarshaling.

toml.Tree replacement

This structure was the initial attempt at providing a document model for go-toml. It allows manipulating the structure of any document, encoding and decoding from their TOML representation. While a more robust feature was initially planned in go-toml v2, this has been ultimately removed from scope of this library, with no plan to add it back at the moment. The closest equivalent at the moment would be to unmarshal into an interface{} and use type assertions and/or reflection to manipulate the arbitrary structure. However this would fall short of providing all of the TOML features such as adding comments and be specific about whitespace.

toml.Position are not retrievable anymore

The API for retrieving the position (line, column) of a specific TOML element do not exist anymore. This was done to minimize the amount of concepts introduced by the library (query path), and avoid the performance hit related to storing positions in the absence of a document model, for a feature that seemed to have little use. Errors however have gained more detailed position information. Position retrieval seems better fitted for a document model, which has been removed from the scope of go-toml v2 at the moment.

Encoding / Marshal
Default struct fields order

V1 emits struct fields order alphabetically by default. V2 struct fields are emitted in order they are defined. For example:

type S struct {
	B string
	A string
}

data := S{
	B: "B",
	A: "A",
}

b, _ := tomlv1.Marshal(data)
fmt.Println("v1:\n" + string(b))

b, _ = tomlv2.Marshal(data)
fmt.Println("v2:\n" + string(b))

// Output:
// v1:
// A = "A"
// B = "B"

// v2:
// B = 'B'
// A = 'A'

There is no way to make v2 encoder behave like v1. A workaround could be to manually sort the fields alphabetically in the struct definition, or generate struct types using reflect.StructOf.

No indentation by default

V1 automatically indents content of tables by default. V2 does not. However the same behavior can be obtained using Encoder.SetIndentTables. For example:

data := map[string]interface{}{
	"table": map[string]string{
		"key": "value",
	},
}

b, _ := tomlv1.Marshal(data)
fmt.Println("v1:\n" + string(b))

b, _ = tomlv2.Marshal(data)
fmt.Println("v2:\n" + string(b))

buf := bytes.Buffer{}
enc := tomlv2.NewEncoder(&buf)
enc.SetIndentTables(true)
enc.Encode(data)
fmt.Println("v2 Encoder:\n" + string(buf.Bytes()))

// Output:
// v1:
//
// [table]
//   key = "value"
//
// v2:
// [table]
// key = 'value'
//
//
// v2 Encoder:
// [table]
//   key = 'value'
Keys and strings are single quoted

V1 always uses double quotes (") around strings and keys that cannot be represented bare (unquoted). V2 uses single quotes instead by default ('), unless a character cannot be represented, then falls back to double quotes. As a result of this change, Encoder.QuoteMapKeys has been removed, as it is not useful anymore.

There is no way to make v2 encoder behave like v1.

TextMarshaler emits as a string, not TOML

Types that implement encoding.TextMarshaler can emit arbitrary TOML in v1. The encoder would append the result to the output directly. In v2 the result is wrapped in a string. As a result, this interface cannot be implemented by the root object.

There is no way to make v2 encoder behave like v1.

Encoder.CompactComments has been removed

Emitting compact comments is now the default behavior of go-toml. This option is not necessary anymore.

Struct tags have been merged

V1 used to provide multiple struct tags: comment, commented, multiline, toml, and omitempty. To behave more like the standard library, v2 has merged toml, multiline, and omitempty. For example:

type doc struct {
	// v1
	F string `toml:"field" multiline:"true" omitempty:"true"`
	// v2
	F string `toml:"field,multiline,omitempty"`
}

Has a result, the Encoder.SetTag* methods have been removed, as there is just one tag now.

commented tag has been removed

There is no replacement for the commented tag. This feature would be better suited in a proper document model for go-toml v2, which has been cut from scope at the moment.

Encoder.ArraysWithOneElementPerLine has been renamed

The new name is Encoder.SetArraysMultiline. The behavior should be the same.

Encoder.Indentation has been renamed

The new name is Encoder.SetIndentSymbol. The behavior should be the same.

Embedded structs behave like stdlib

V1 defaults to merging embedded struct fields into the embedding struct. This behavior was unexpected because it does not follow the standard library. To avoid breaking backward compatibility, the Encoder.PromoteAnonymous method was added to make the encoder behave correctly. Given backward compatibility is not a problem anymore, v2 does the right thing by default: it follows the behavior of encoding/json. Encoder.PromoteAnonymous has been removed.

query

go-toml v1 provided the go-toml/query package. It allowed to run JSONPath-style queries on TOML files. This feature is not available in v2. For a replacement, check out dasel.

This package has been removed because it was essentially not supported anymore (last commit May 2020), increased the complexity of the code base, and more complete solutions exist out there.

Versioning

Go-toml follows Semantic Versioning. The supported version of TOML is indicated at the beginning of this document. The last two major versions of Go are supported (see Go Release Policy).

License

The MIT License (MIT). Read LICENSE.

Documentation

Overview

Package toml is a library to read and write TOML documents.

Index

Examples

Constants

This section is empty.

Variables

This section is empty.

Functions

func Marshal

func Marshal(v interface{}) ([]byte, error)

Marshal serializes a Go value as a TOML document.

It is a shortcut for Encoder.Encode() with the default options.

Example
type MyConfig struct {
	Version int
	Name    string
	Tags    []string
}

cfg := MyConfig{
	Version: 2,
	Name:    "go-toml",
	Tags:    []string{"go", "toml"},
}

b, err := toml.Marshal(cfg)
if err != nil {
	panic(err)
}
fmt.Println(string(b))
Output:

Version = 2
Name = 'go-toml'
Tags = ['go', 'toml']

func Unmarshal

func Unmarshal(data []byte, v interface{}) error

Unmarshal deserializes a TOML document into a Go value.

It is a shortcut for Decoder.Decode() with the default options.

Example
type MyConfig struct {
	Version int
	Name    string
	Tags    []string
}

doc := `
	version = 2
	name = "go-toml"
	tags = ["go", "toml"]
	`

var cfg MyConfig
err := toml.Unmarshal([]byte(doc), &cfg)
if err != nil {
	panic(err)
}
fmt.Println("version:", cfg.Version)
fmt.Println("name:", cfg.Name)
fmt.Println("tags:", cfg.Tags)
Output:

version: 2
name: go-toml
tags: [go toml]

Types

type DecodeError

type DecodeError struct {
	// contains filtered or unexported fields
}

DecodeError represents an error encountered during the parsing or decoding of a TOML document.

In addition to the error message, it contains the position in the document where it happened, as well as a human-readable representation that shows where the error occurred in the document.

Example
doc := `name = 123__456`

s := map[string]interface{}{}
err := Unmarshal([]byte(doc), &s)

fmt.Println(err)

var derr *DecodeError
if errors.As(err, &derr) {
	fmt.Println(derr.String())
	row, col := derr.Position()
	fmt.Println("error occurred at row", row, "column", col)
}
Output:

toml: number must have at least one digit between underscores
1| name = 123__456
 |           ~~ number must have at least one digit between underscores
error occurred at row 1 column 11

func (*DecodeError) Error

func (e *DecodeError) Error() string

Error returns the error message contained in the DecodeError.

func (*DecodeError) Key

func (e *DecodeError) Key() Key

Key that was being processed when the error occurred. The key is present only if this DecodeError is part of a StrictMissingError.

func (*DecodeError) Position

func (e *DecodeError) Position() (row int, column int)

Position returns the (line, column) pair indicating where the error occurred in the document. Positions are 1-indexed.

func (*DecodeError) String

func (e *DecodeError) String() string

String returns the human-readable contextualized error. This string is multi-line.

type Decoder

type Decoder struct {
	// contains filtered or unexported fields
}

Decoder reads and decode a TOML document from an input stream.

func NewDecoder

func NewDecoder(r io.Reader) *Decoder

NewDecoder creates a new Decoder that will read from r.

func (*Decoder) Decode

func (d *Decoder) Decode(v interface{}) error

Decode the whole content of r into v.

By default, values in the document that don't exist in the target Go value are ignored. See Decoder.DisallowUnknownFields() to change this behavior.

When a TOML local date, time, or date-time is decoded into a time.Time, its value is represented in time.Local timezone. Otherwise the approriate Local* structure is used. For time values, precision up to the nanosecond is supported by truncating extra digits.

Empty tables decoded in an interface{} create an empty initialized map[string]interface{}.

Types implementing the encoding.TextUnmarshaler interface are decoded from a TOML string.

When decoding a number, go-toml will return an error if the number is out of bounds for the target type (which includes negative numbers when decoding into an unsigned int).

If an error occurs while decoding the content of the document, this function returns a toml.DecodeError, providing context about the issue. When using strict mode and a field is missing, a `toml.StrictMissingError` is returned. In any other case, this function returns a standard Go error.

Type mapping

List of supported TOML types and their associated accepted Go types:

String           -> string
Integer          -> uint*, int*, depending on size
Float            -> float*, depending on size
Boolean          -> bool
Offset Date-Time -> time.Time
Local Date-time  -> LocalDateTime, time.Time
Local Date       -> LocalDate, time.Time
Local Time       -> LocalTime, time.Time
Array            -> slice and array, depending on elements types
Table            -> map and struct
Inline Table     -> same as Table
Array of Tables  -> same as Array and Table

func (*Decoder) DisallowUnknownFields

func (d *Decoder) DisallowUnknownFields() *Decoder

DisallowUnknownFields causes the Decoder to return an error when the destination is a struct and the input contains a key that does not match a non-ignored field.

In that case, the Decoder returns a StrictMissingError that can be used to retrieve the individual errors as well as generate a human readable description of the missing fields.

Example
type S struct {
	Key1 string
	Key3 string
}
doc := `
key1 = "value1"
key2 = "value2"
key3 = "value3"
`
r := strings.NewReader(doc)
d := toml.NewDecoder(r)
d.DisallowUnknownFields()
s := S{}
err := d.Decode(&s)

fmt.Println(err.Error())

var details *toml.StrictMissingError
if !errors.As(err, &details) {
	panic(fmt.Sprintf("err should have been a *toml.StrictMissingError, but got %s (%T)", err, err))
}

fmt.Println(details.String())
Output:

strict mode: fields in the document are missing in the target struct
2| key1 = "value1"
3| key2 = "value2"
 | ~~~~ missing field
4| key3 = "value3"

type Encoder

type Encoder struct {
	// contains filtered or unexported fields
}

Encoder writes a TOML document to an output stream.

func NewEncoder

func NewEncoder(w io.Writer) *Encoder

NewEncoder returns a new Encoder that writes to w.

func (*Encoder) Encode

func (enc *Encoder) Encode(v interface{}) error

Encode writes a TOML representation of v to the stream.

If v cannot be represented to TOML it returns an error.

Encoding rules

A top level slice containing only maps or structs is encoded as [[table array]].

All slices not matching rule 1 are encoded as [array]. As a result, any map or struct they contain is encoded as an {inline table}.

Nil interfaces and nil pointers are not supported.

Keys in key-values always have one part.

Intermediate tables are always printed.

By default, strings are encoded as literal string, unless they contain either a newline character or a single quote. In that case they are emitted as quoted strings.

Unsigned integers larger than math.MaxInt64 cannot be encoded. Doing so results in an error. This rule exists because the TOML specification only requires parsers to support at least the 64 bits integer range. Allowing larger numbers would create non-standard TOML documents, which may not be readable (at best) by other implementations. To encode such numbers, a solution is a custom type that implements encoding.TextMarshaler.

When encoding structs, fields are encoded in order of definition, with their exact name.

Struct tags

The encoding of each public struct field can be customized by the format string in the "toml" key of the struct field's tag. This follows encoding/json's convention. The format string starts with the name of the field, optionally followed by a comma-separated list of options. The name may be empty in order to provide options without overriding the default name.

The "multiline" option emits strings as quoted multi-line TOML strings. It has no effect on fields that would not be encoded as strings.

The "inline" option turns fields that would be emitted as tables into inline tables instead. It has no effect on other fields.

The "omitempty" option prevents empty values or groups from being emitted.

In addition to the "toml" tag struct tag, a "comment" tag can be used to emit a TOML comment before the value being annotated. Comments are ignored inside inline tables. For array tables, the comment is only present before the first element of the array.

func (*Encoder) SetArraysMultiline

func (enc *Encoder) SetArraysMultiline(multiline bool) *Encoder

SetArraysMultiline forces the encoder to emit all arrays with one element per line.

This behavior can be controlled on an individual struct field basis with the multiline tag:

MyField `multiline:"true"`

func (*Encoder) SetIndentSymbol

func (enc *Encoder) SetIndentSymbol(s string) *Encoder

SetIndentSymbol defines the string that should be used for indentation. The provided string is repeated for each indentation level. Defaults to two spaces.

func (*Encoder) SetIndentTables

func (enc *Encoder) SetIndentTables(indent bool) *Encoder

SetIndentTables forces the encoder to intent tables and array tables.

func (*Encoder) SetTablesInline

func (enc *Encoder) SetTablesInline(inline bool) *Encoder

SetTablesInline forces the encoder to emit all tables inline.

This behavior can be controlled on an individual struct field basis with the inline tag:

MyField `inline:"true"`

type Key

type Key []string

type LocalDate

type LocalDate struct {
	Year  int
	Month int
	Day   int
}

LocalDate represents a calendar day in no specific timezone.

func (LocalDate) AsTime

func (d LocalDate) AsTime(zone *time.Location) time.Time

AsTime converts d into a specific time instance at midnight in zone.

func (LocalDate) MarshalText

func (d LocalDate) MarshalText() ([]byte, error)

MarshalText returns RFC 3339 representation of d.

func (LocalDate) String

func (d LocalDate) String() string

String returns RFC 3339 representation of d.

func (*LocalDate) UnmarshalText

func (d *LocalDate) UnmarshalText(b []byte) error

UnmarshalText parses b using RFC 3339 to fill d.

type LocalDateTime

type LocalDateTime struct {
	LocalDate
	LocalTime
}

LocalDateTime represents a time of a specific day in no specific timezone.

func (LocalDateTime) AsTime

func (d LocalDateTime) AsTime(zone *time.Location) time.Time

AsTime converts d into a specific time instance in zone.

func (LocalDateTime) MarshalText

func (d LocalDateTime) MarshalText() ([]byte, error)

MarshalText returns RFC 3339 representation of d.

func (LocalDateTime) String

func (d LocalDateTime) String() string

String returns RFC 3339 representation of d.

func (*LocalDateTime) UnmarshalText

func (d *LocalDateTime) UnmarshalText(data []byte) error

UnmarshalText parses b using RFC 3339 to fill d.

type LocalTime

type LocalTime struct {
	Hour       int // Hour of the day: [0; 24[
	Minute     int // Minute of the hour: [0; 60[
	Second     int // Second of the minute: [0; 60[
	Nanosecond int // Nanoseconds within the second:  [0, 1000000000[
	Precision  int // Number of digits to display for Nanosecond.
}

LocalTime represents a time of day of no specific day in no specific timezone.

func (LocalTime) MarshalText

func (d LocalTime) MarshalText() ([]byte, error)

MarshalText returns RFC 3339 representation of d.

func (LocalTime) String

func (d LocalTime) String() string

String returns RFC 3339 representation of d. If d.Nanosecond and d.Precision are zero, the time won't have a nanosecond component. If d.Nanosecond > 0 but d.Precision = 0, then the minimum number of digits for nanoseconds is provided.

func (*LocalTime) UnmarshalText

func (d *LocalTime) UnmarshalText(b []byte) error

UnmarshalText parses b using RFC 3339 to fill d.

type StrictMissingError

type StrictMissingError struct {
	// One error per field that could not be found.
	Errors []DecodeError
}

StrictMissingError occurs in a TOML document that does not have a corresponding field in the target value. It contains all the missing fields in Errors.

Emitted by Decoder when DisallowUnknownFields() was called.

func (*StrictMissingError) Error

func (s *StrictMissingError) Error() string

Error returns the canonical string for this error.

func (*StrictMissingError) String

func (s *StrictMissingError) String() string

String returns a human readable description of all errors.

Directories

Path Synopsis
cmd
jsontoml
Package jsontoml is a program that converts JSON to TOML.
Package jsontoml is a program that converts JSON to TOML.
tomljson
Package tomljson is a program that converts TOML to JSON.
Package tomljson is a program that converts TOML to JSON.
tomll
Package tomll is a linter program for TOML.
Package tomll is a linter program for TOML.
tomltestgen
tomltestgen retrieves a given version of the language-agnostic TOML test suite in https://github.com/BurntSushi/toml-test and generates go-toml unit tests.
tomltestgen retrieves a given version of the language-agnostic TOML test suite in https://github.com/BurntSushi/toml-test and generates go-toml unit tests.
internal
ast
cli
testsuite
Package testsuite provides helper functions for interoperating with the language-agnostic TOML test suite at github.com/BurntSushi/toml-test.
Package testsuite provides helper functions for interoperating with the language-agnostic TOML test suite at github.com/BurntSushi/toml-test.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL