toml

package module
Version: v0.4.1 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Aug 5, 2021 License: MIT Imports: 19 Imported by: 8,294

README

TOML parser and encoder for Go with reflection

TOML stands for Tom's Obvious, Minimal Language. This Go package provides a reflection interface similar to Go's standard library json and xml packages. This package also supports the encoding.TextUnmarshaler and encoding.TextMarshaler interfaces so that you can define custom data representations. (There is an example of this below.)

Compatible with TOML version v1.0.0.

Documentation: https://godocs.io/github.com/BurntSushi/toml

See the releases page for a changelog; this information is also in the git tag annotations (e.g. git show v0.4.0).

This library requires Go 1.13 or newer; install it with:

$ go get github.com/BurntSushi/toml

It also comes with a TOML validator CLI tool:

$ go get github.com/BurntSushi/toml/cmd/tomlv
$ tomlv some-toml-file.toml

Testing

This package passes all tests in toml-test for both the decoder and the encoder.

Examples

This package works similarly to how the Go standard library handles XML and JSON. Namely, data is loaded into Go values via reflection.

For the simplest example, consider some TOML file as just a list of keys and values:

Age = 25
Cats = [ "Cauchy", "Plato" ]
Pi = 3.14
Perfection = [ 6, 28, 496, 8128 ]
DOB = 1987-07-05T05:45:00Z

Which could be defined in Go as:

type Config struct {
	Age        int
	Cats       []string
	Pi         float64
	Perfection []int
	DOB        time.Time // requires `import time`
}

And then decoded with:

var conf Config
if _, err := toml.Decode(tomlData, &conf); err != nil {
  // handle error
}

You can also use struct tags if your struct field name doesn't map to a TOML key value directly:

some_key_NAME = "wat"
type TOML struct {
  ObscureKey string `toml:"some_key_NAME"`
}

Beware that like other most other decoders only exported fields are considered when encoding and decoding; private fields are silently ignored.

Using the encoding.TextUnmarshaler interface

Here's an example that automatically parses duration strings into time.Duration values:

[[song]]
name = "Thunder Road"
duration = "4m49s"

[[song]]
name = "Stairway to Heaven"
duration = "8m03s"

Which can be decoded with:

type song struct {
	Name     string
	Duration duration
}
type songs struct {
	Song []song
}
var favorites songs
if _, err := toml.Decode(blob, &favorites); err != nil {
	log.Fatal(err)
}

for _, s := range favorites.Song {
	fmt.Printf("%s (%s)\n", s.Name, s.Duration)
}

And you'll also need a duration type that satisfies the encoding.TextUnmarshaler interface:

type duration struct {
	time.Duration
}

func (d *duration) UnmarshalText(text []byte) error {
	var err error
	d.Duration, err = time.ParseDuration(string(text))
	return err
}

To target TOML specifically you can implement UnmarshalTOML TOML interface in a similar way.

More complex usage

Here's an example of how to load the example from the official spec page:

# This is a TOML document. Boom.

title = "TOML Example"

[owner]
name = "Tom Preston-Werner"
organization = "GitHub"
bio = "GitHub Cofounder & CEO\nLikes tater tots and beer."
dob = 1979-05-27T07:32:00Z # First class dates? Why not?

[database]
server = "192.168.1.1"
ports = [ 8001, 8001, 8002 ]
connection_max = 5000
enabled = true

[servers]

  # You can indent as you please. Tabs or spaces. TOML don't care.
  [servers.alpha]
  ip = "10.0.0.1"
  dc = "eqdc10"

  [servers.beta]
  ip = "10.0.0.2"
  dc = "eqdc10"

[clients]
data = [ ["gamma", "delta"], [1, 2] ] # just an update to make sure parsers support it

# Line breaks are OK when inside arrays
hosts = [
  "alpha",
  "omega"
]

And the corresponding Go types are:

type tomlConfig struct {
	Title   string
	Owner   ownerInfo
	DB      database `toml:"database"`
	Servers map[string]server
	Clients clients
}

type ownerInfo struct {
	Name string
	Org  string `toml:"organization"`
	Bio  string
	DOB  time.Time
}

type database struct {
	Server  string
	Ports   []int
	ConnMax int `toml:"connection_max"`
	Enabled bool
}

type server struct {
	IP string
	DC string
}

type clients struct {
	Data  [][]interface{}
	Hosts []string
}

Note that a case insensitive match will be tried if an exact match can't be found.

A working example of the above can be found in _examples/example.{go,toml}.

Documentation

Overview

Package toml implements decoding and encoding of TOML files.

This package supports TOML v1.0.0, as listed on https://toml.io

There is also support for delaying decoding with the Primitive type, and querying the set of keys in a TOML document with the MetaData type.

The github.com/BurntSushi/toml/cmd/tomlv package implements a TOML validator, and can be used to verify if TOML document is valid. It can also be used to print the type of each key.

Example (StrictDecoding)

Example StrictDecoding shows how to detect whether there are keys in the TOML document that weren't decoded into the value given. This is useful for returning an error to the user if they've included extraneous fields in their configuration.

package main

import (
	"fmt"
	"log"

	"github.com/BurntSushi/toml"
)

func main() {
	var blob = `
key1 = "value1"
key2 = "value2"
key3 = "value3"
`
	type config struct {
		Key1 string
		Key3 string
	}

	var conf config
	md, err := toml.Decode(blob, &conf)
	if err != nil {
		log.Fatal(err)
	}
	fmt.Printf("Undecoded keys: %q\n", md.Undecoded())
}
Output:

Undecoded keys: ["key2"]
Example (UnmarshalTOML)

Example UnmarshalTOML shows how to implement a struct type that knows how to unmarshal itself. The struct must take full responsibility for mapping the values passed into the struct. The method may be used with interfaces in a struct in cases where the actual type is not known until the data is examined.

package main

import (
	"fmt"
	"log"

	"github.com/BurntSushi/toml"
)

type order struct {
	parts parts
}

type parts []part

type part interface {
	Name() string
}

type valve struct {
	Type   string
	ID     string
	Size   float32
	Rating int
}

func (v *valve) Name() string {
	return fmt.Sprintf("VALVE: %s", v.ID)
}

type pipe struct {
	Type     string
	ID       string
	Length   float32
	Diameter int
}

func (p *pipe) Name() string {
	return fmt.Sprintf("PIPE: %s", p.ID)
}

type cable struct {
	Type   string
	ID     string
	Length int
	Rating float32
}

func (c *cable) Name() string {
	return fmt.Sprintf("CABLE: %s", c.ID)
}

func (o *order) UnmarshalTOML(data interface{}) error {

	d, _ := data.(map[string]interface{})
	parts, _ := d["parts"].([]map[string]interface{})

	for _, p := range parts {

		typ, _ := p["type"].(string)
		id, _ := p["id"].(string)

		switch p["type"] {
		case "valve":

			size := float32(p["size"].(float64))
			rating := int(p["rating"].(int64))

			valve := &valve{
				Type:   typ,
				ID:     id,
				Size:   size,
				Rating: rating,
			}

			o.parts = append(o.parts, valve)

		case "pipe":

			length := float32(p["length"].(float64))
			diameter := int(p["diameter"].(int64))

			pipe := &pipe{
				Type:     typ,
				ID:       id,
				Length:   length,
				Diameter: diameter,
			}

			o.parts = append(o.parts, pipe)

		case "cable":

			length := int(p["length"].(int64))
			rating := float32(p["rating"].(float64))

			cable := &cable{
				Type:   typ,
				ID:     id,
				Length: length,
				Rating: rating,
			}

			o.parts = append(o.parts, cable)

		}
	}

	return nil
}

func main() {

	var blob = `
[[parts]]
type = "valve"
id = "valve-1"
size = 1.2
rating = 4

[[parts]]
type = "valve"
id = "valve-2"
size = 2.1
rating = 5

[[parts]]
type = "pipe"
id = "pipe-1"
length = 2.1
diameter = 12

[[parts]]
type = "cable"
id = "cable-1"
length = 12
rating = 3.1
`
	o := &order{}
	err := toml.Unmarshal([]byte(blob), o)
	if err != nil {
		log.Fatal(err)
	}

	fmt.Println(len(o.parts))

	for _, part := range o.parts {
		fmt.Println(part.Name())
	}

	// Code to implement UmarshalJSON.

	// type order struct {
	// 	// NOTE `order.parts` is a private slice of type `part` which is an
	// 	// interface and may only be loaded from toml using the
	// 	// UnmarshalTOML() method of the Umarshaler interface.
	// 	parts parts
	// }

	// func (o *order) UnmarshalTOML(data interface{}) error {

	// 	// NOTE the example below contains detailed type casting to show how
	// 	// the 'data' is retrieved. In operational use, a type cast wrapper
	// 	// may be preferred e.g.
	// 	//
	// 	// func AsMap(v interface{}) (map[string]interface{}, error) {
	// 	// 		return v.(map[string]interface{})
	// 	// }
	// 	//
	// 	// resulting in:
	// 	// d, _ := AsMap(data)
	// 	//

	// 	d, _ := data.(map[string]interface{})
	// 	parts, _ := d["parts"].([]map[string]interface{})

	// 	for _, p := range parts {

	// 		typ, _ := p["type"].(string)
	// 		id, _ := p["id"].(string)

	// 		// detect the type of part and handle each case
	// 		switch p["type"] {
	// 		case "valve":

	// 			size := float32(p["size"].(float64))
	// 			rating := int(p["rating"].(int64))

	// 			valve := &valve{
	// 				Type:   typ,
	// 				ID:     id,
	// 				Size:   size,
	// 				Rating: rating,
	// 			}

	// 			o.parts = append(o.parts, valve)

	// 		case "pipe":

	// 			length := float32(p["length"].(float64))
	// 			diameter := int(p["diameter"].(int64))

	// 			pipe := &pipe{
	// 				Type:     typ,
	// 				ID:       id,
	// 				Length:   length,
	// 				Diameter: diameter,
	// 			}

	// 			o.parts = append(o.parts, pipe)

	// 		case "cable":

	// 			length := int(p["length"].(int64))
	// 			rating := float32(p["rating"].(float64))

	// 			cable := &cable{
	// 				Type:   typ,
	// 				ID:     id,
	// 				Length: length,
	// 				Rating: rating,
	// 			}

	// 			o.parts = append(o.parts, cable)

	// 		}
	// 	}

	// 	return nil
	// }

	// type parts []part

	// type part interface {
	// 	Name() string
	// }

	// type valve struct {
	// 	Type   string
	// 	ID     string
	// 	Size   float32
	// 	Rating int
	// }

	// func (v *valve) Name() string {
	// 	return fmt.Sprintf("VALVE: %s", v.ID)
	// }

	// type pipe struct {
	// 	Type     string
	// 	ID       string
	// 	Length   float32
	// 	Diameter int
	// }

	// func (p *pipe) Name() string {
	// 	return fmt.Sprintf("PIPE: %s", p.ID)
	// }

	// type cable struct {
	// 	Type   string
	// 	ID     string
	// 	Length int
	// 	Rating float32
	// }

	// func (c *cable) Name() string {
	// 	return fmt.Sprintf("CABLE: %s", c.ID)
	// }

}
Output:

4
VALVE: valve-1
VALVE: valve-2
PIPE: pipe-1
CABLE: cable-1
Example (Unmarshaler)

Example Unmarshaler shows how to decode TOML strings into your own custom data type.

package main

import (
	"fmt"
	"log"
	"time"

	"github.com/BurntSushi/toml"
)

type duration struct {
	time.Duration
}

func (d *duration) UnmarshalText(text []byte) error {
	var err error
	d.Duration, err = time.ParseDuration(string(text))
	return err
}

func main() {
	blob := `
[[song]]
name = "Thunder Road"
duration = "4m49s"

[[song]]
name = "Stairway to Heaven"
duration = "8m03s"
`
	type song struct {
		Name     string
		Duration duration
	}
	type songs struct {
		Song []song
	}
	var favorites songs
	if _, err := toml.Decode(blob, &favorites); err != nil {
		log.Fatal(err)
	}

	// Code to implement the TextUnmarshaler interface for `duration`:
	//
	// type duration struct {
	// 	time.Duration
	// }
	//
	// func (d *duration) UnmarshalText(text []byte) error {
	// 	var err error
	// 	d.Duration, err = time.ParseDuration(string(text))
	// 	return err
	// }

	for _, s := range favorites.Song {
		fmt.Printf("%s (%s)\n", s.Name, s.Duration)
	}
}
Output:

Thunder Road (4m49s)
Stairway to Heaven (8m3s)

Index

Examples

Constants

This section is empty.

Variables

This section is empty.

Functions

func PrimitiveDecode

func PrimitiveDecode(primValue Primitive, v interface{}) error

DEPRECATED!

Use MetaData.PrimitiveDecode instead.

func Unmarshal

func Unmarshal(p []byte, v interface{}) error

Unmarshal decodes the contents of `p` in TOML format into a pointer `v`.

Types

type Decoder added in v0.4.1

type Decoder struct {
	// contains filtered or unexported fields
}

Decoder decodes TOML data.

TOML tables correspond to Go structs or maps (dealer's choice – they can be used interchangeably).

TOML table arrays correspond to either a slice of structs or a slice of maps.

TOML datetimes correspond to Go time.Time values. Local datetimes are parsed in the local timezone.

All other TOML types (float, string, int, bool and array) correspond to the obvious Go types.

An exception to the above rules is if a type implements the TextUnmarshaler interface, in which case any primitive TOML value (floats, strings, integers, booleans, datetimes) will be converted to a []byte and given to the value's UnmarshalText method. See the Unmarshaler example for a demonstration with time duration strings.

Key mapping

TOML keys can map to either keys in a Go map or field names in a Go struct. The special `toml` struct tag can be used to map TOML keys to struct fields that don't match the key name exactly (see the example). A case insensitive match to struct names will be tried if an exact match can't be found.

The mapping between TOML values and Go values is loose. That is, there may exist TOML values that cannot be placed into your representation, and there may be parts of your representation that do not correspond to TOML values. This loose mapping can be made stricter by using the IsDefined and/or Undecoded methods on the MetaData returned.

This decoder does not handle cyclic types. Decode will not terminate if a cyclic type is passed.

func NewDecoder added in v0.4.1

func NewDecoder(r io.Reader) *Decoder

NewDecoder creates a new Decoder.

func (*Decoder) Decode added in v0.4.1

func (dec *Decoder) Decode(v interface{}) (MetaData, error)

Decode TOML data in to the pointer `v`.

type Encoder

type Encoder struct {
	// The string to use for a single indentation level. The default is two
	// spaces.
	Indent string
	// contains filtered or unexported fields
}

Encoder encodes a Go to a TOML document.

The mapping between Go values and TOML values should be precisely the same as for the Decode* functions. Similarly, the TextMarshaler interface is supported by encoding the resulting bytes as strings. If you want to write arbitrary binary data then you will need to use something like base64 since TOML does not have any binary types.

When encoding TOML hashes (Go maps or structs), keys without any sub-hashes are encoded first.

Go maps will be sorted alphabetically by key for deterministic output.

Encoding Go values without a corresponding TOML representation will return an error. Examples of this includes maps with non-string keys, slices with nil elements, embedded non-struct types, and nested slices containing maps or structs. (e.g. [][]map[string]string is not allowed but []map[string]string is okay, as is []map[string][]string).

NOTE: Only exported keys are encoded due to the use of reflection. Unexported keys are silently discarded.

func NewEncoder

func NewEncoder(w io.Writer) *Encoder

NewEncoder create a new Encoder.

func (*Encoder) Encode

func (enc *Encoder) Encode(v interface{}) error

Encode writes a TOML representation of the Go value to the Encoder's writer.

An error is returned if the value given cannot be encoded to a valid TOML document.

Example
package main

import (
	"bytes"
	"fmt"
	"log"
	"time"

	"github.com/BurntSushi/toml"
)

func main() {
	date, _ := time.Parse(time.RFC822, "14 Mar 10 18:00 UTC")
	var config = map[string]interface{}{
		"date":   date,
		"counts": []int{1, 1, 2, 3, 5, 8},
		"hash": map[string]string{
			"key1": "val1",
			"key2": "val2",
		},
	}
	buf := new(bytes.Buffer)
	if err := toml.NewEncoder(buf).Encode(config); err != nil {
		log.Fatal(err)
	}
	fmt.Println(buf.String())

}
Output:

counts = [1, 1, 2, 3, 5, 8]
date = 2010-03-14T18:00:00Z

[hash]
  key1 = "val1"
  key2 = "val2"

type Key

type Key []string

Key represents any TOML key, including key groups. Use (MetaData).Keys to get values of this type.

func (Key) String

func (k Key) String() string

type MetaData

type MetaData struct {
	// contains filtered or unexported fields
}

MetaData allows access to meta information about TOML data that may not be inferable via reflection. In particular, whether a key has been defined and the TOML type of a key.

func Decode

func Decode(data string, v interface{}) (MetaData, error)

Decode the TOML data in to the pointer v.

See the documentation on Decoder for a description of the decoding process.

Example
package main

import (
	"fmt"
	"log"
	"time"

	"github.com/BurntSushi/toml"
)

func main() {
	var tomlBlob = `
# Some comments.
[alpha]
ip = "10.0.0.1"

	[alpha.config]
	Ports = [ 8001, 8002 ]
	Location = "Toronto"
	Created = 1987-07-05T05:45:00Z

[beta]
ip = "10.0.0.2"

	[beta.config]
	Ports = [ 9001, 9002 ]
	Location = "New Jersey"
	Created = 1887-01-05T05:55:00Z
`

	type serverConfig struct {
		Ports    []int
		Location string
		Created  time.Time
	}

	type server struct {
		IP     string       `toml:"ip,omitempty"`
		Config serverConfig `toml:"config"`
	}

	type servers map[string]server

	var config servers
	if _, err := toml.Decode(tomlBlob, &config); err != nil {
		log.Fatal(err)
	}

	for _, name := range []string{"alpha", "beta"} {
		s := config[name]
		fmt.Printf("Server: %s (ip: %s) in %s created on %s\n",
			name, s.IP, s.Config.Location,
			s.Config.Created.Format("2006-01-02"))
		fmt.Printf("Ports: %v\n", s.Config.Ports)
	}

}
Output:

Server: alpha (ip: 10.0.0.1) in Toronto created on 1987-07-05
Ports: [8001 8002]
Server: beta (ip: 10.0.0.2) in New Jersey created on 1887-01-05
Ports: [9001 9002]

func DecodeFS added in v0.4.1

func DecodeFS(fsys fs.FS, path string, v interface{}) (MetaData, error)

DecodeFS is just like Decode, except it will automatically read the contents of the file at `path` from a fs.FS instance.

func DecodeFile

func DecodeFile(path string, v interface{}) (MetaData, error)

DecodeFile is just like Decode, except it will automatically read the contents of the file at path and decode it for you.

func DecodeReader

func DecodeReader(r io.Reader, v interface{}) (MetaData, error)

DEPRECATED!

Use NewDecoder(reader).Decode(&v) instead.

func (*MetaData) IsDefined

func (md *MetaData) IsDefined(key ...string) bool

IsDefined reports if the key exists in the TOML data.

The key should be specified hierarchically, for example to access the TOML key "a.b.c" you would use:

IsDefined("a", "b", "c")

IsDefined will return false if an empty key given. Keys are case sensitive.

func (*MetaData) Keys

func (md *MetaData) Keys() []Key

Keys returns a slice of every key in the TOML data, including key groups.

Each key is itself a slice, where the first element is the top of the hierarchy and the last is the most specific. The list will have the same order as the keys appeared in the TOML data.

All keys returned are non-empty.

func (*MetaData) PrimitiveDecode

func (md *MetaData) PrimitiveDecode(primValue Primitive, v interface{}) error

PrimitiveDecode is just like the other `Decode*` functions, except it decodes a TOML value that has already been parsed. Valid primitive values can *only* be obtained from values filled by the decoder functions, including this method. (i.e., `v` may contain more `Primitive` values.)

Meta data for primitive values is included in the meta data returned by the `Decode*` functions with one exception: keys returned by the Undecoded method will only reflect keys that were decoded. Namely, any keys hidden behind a Primitive will be considered undecoded. Executing this method will update the undecoded keys in the meta data. (See the example.)

Example
package main

import (
	"fmt"
	"log"

	"github.com/BurntSushi/toml"
)

func main() {
	var md toml.MetaData
	var err error

	var tomlBlob = `
ranking = ["Springsteen", "J Geils"]

[bands.Springsteen]
started = 1973
albums = ["Greetings", "WIESS", "Born to Run", "Darkness"]

[bands."J Geils"]
started = 1970
albums = ["The J. Geils Band", "Full House", "Blow Your Face Out"]
`

	type band struct {
		Started int
		Albums  []string
	}
	type classics struct {
		Ranking []string
		Bands   map[string]toml.Primitive
	}

	// Do the initial decode. Reflection is delayed on Primitive values.
	var music classics
	if md, err = toml.Decode(tomlBlob, &music); err != nil {
		log.Fatal(err)
	}

	// MetaData still includes information on Primitive values.
	fmt.Printf("Is `bands.Springsteen` defined? %v\n",
		md.IsDefined("bands", "Springsteen"))

	// Decode primitive data into Go values.
	for _, artist := range music.Ranking {
		// A band is a primitive value, so we need to decode it to get a
		// real `band` value.
		primValue := music.Bands[artist]

		var aBand band
		if err = md.PrimitiveDecode(primValue, &aBand); err != nil {
			log.Fatal(err)
		}
		fmt.Printf("%s started in %d.\n", artist, aBand.Started)
	}
	// Check to see if there were any fields left undecoded.
	// Note that this won't be empty before decoding the Primitive value!
	fmt.Printf("Undecoded: %q\n", md.Undecoded())

}
Output:

Is `bands.Springsteen` defined? true
Springsteen started in 1973.
J Geils started in 1970.
Undecoded: []

func (*MetaData) Type

func (md *MetaData) Type(key ...string) string

Type returns a string representation of the type of the key specified.

Type will return the empty string if given an empty key or a key that does not exist. Keys are case sensitive.

func (*MetaData) Undecoded

func (md *MetaData) Undecoded() []Key

Undecoded returns all keys that have not been decoded in the order in which they appear in the original TOML document.

This includes keys that haven't been decoded because of a Primitive value. Once the Primitive value is decoded, the keys will be considered decoded.

Also note that decoding into an empty interface will result in no decoding, and so no keys will be considered decoded.

In this sense, the Undecoded keys correspond to keys in the TOML document that do not have a concrete type in your representation.

type ParseError added in v0.4.1

type ParseError struct {
	Message string
	Line    int
	LastKey string
}

ParseError is used when a file can't be parsed: for example invalid integer literals, duplicate keys, etc.

func (ParseError) Error added in v0.4.1

func (pe ParseError) Error() string

type Primitive

type Primitive struct {
	// contains filtered or unexported fields
}

Primitive is a TOML value that hasn't been decoded into a Go value.

This type can be used for any value, which will cause decoding to be delayed. You can use the PrimitiveDecode() function to "manually" decode these values.

NOTE: The underlying representation of a `Primitive` value is subject to change. Do not rely on it.

NOTE: Primitive values are still parsed, so using them will only avoid the overhead of reflection. They can be useful when you don't know the exact type of TOML data until runtime.

type TextMarshaler

type TextMarshaler encoding.TextMarshaler

DEPRECATED!

Use the identical encoding.TextMarshaler instead. It is defined here to support Go 1.1 and older.

type TextUnmarshaler

type TextUnmarshaler encoding.TextUnmarshaler

DEPRECATED!

Use the identical encoding.TextUnmarshaler instead. It is defined here to support Go 1.1 and older.

type Unmarshaler

type Unmarshaler interface {
	UnmarshalTOML(interface{}) error
}

Unmarshaler is the interface implemented by objects that can unmarshal a TOML description of themselves.

Directories

Path Synopsis
cmd
toml-test-decoder
Command toml-test-decoder satisfies the toml-test interface for testing TOML decoders.
Command toml-test-decoder satisfies the toml-test interface for testing TOML decoders.
toml-test-encoder
Command toml-test-encoder satisfies the toml-test interface for testing TOML encoders.
Command toml-test-encoder satisfies the toml-test interface for testing TOML encoders.
tomlv
Command tomlv validates TOML documents and prints each key's type.
Command tomlv validates TOML documents and prints each key's type.
tag

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL