toml

README

go-toml

Go library for the TOML format.

This library supports TOML version v0.4.0

Features

Go-toml provides the following features for using data parsed from TOML documents:

• Load TOML documents from files and string data
• Easily navigate TOML structure using TomlTree
• Line & column position data for all parsed elements
• Query support similar to JSON-Path
• Syntax errors contain line and column numbers

Go-toml is designed to help cover use-cases not covered by reflection-based TOML parsing:

• Semantic evaluation of parsed TOML
• Informing a user of mistakes in the source document, after it has been parsed
• Programatic handling of default values on a case-by-case basis
• Using a TOML document as a flexible data-store

Import

import "github.com/pelletier/go-toml"

Usage

Example

Say you have a TOML file that looks like this:

[postgres]
user = "pelletier"
password = "mypassword"

Read the username and password like this:

import (
    "fmt"
    "github.com/pelletier/go-toml"
)

config, err := toml.LoadFile("config.toml")
if err != nil {
    fmt.Println("Error ", err.Error())
} else {
    // retrieve data directly
    user := config.Get("postgres.user").(string)
    password := config.Get("postgres.password").(string)

    // or using an intermediate object
    configTree := config.Get("postgres").(*toml.TomlTree)
    user = configTree.Get("user").(string)
    password = configTree.Get("password").(string)
    fmt.Println("User is ", user, ". Password is ", password)

    // show where elements are in the file
    fmt.Println("User position: %v", configTree.GetPosition("user"))
    fmt.Println("Password position: %v", configTree.GetPosition("password"))

    // use a query to gather elements without walking the tree
    results, _ := config.Query("$..[user,password]")
    for ii, item := range results.Values() {
      fmt.Println("Query result %d: %v", ii, item)
    }
}

Documentation

The documentation and additional examples are available at godoc.org.

Tools

Go-toml provides two handy command line tools:

• tomll: Reads TOML files and lint them.

  go install github.com/pelletier/go-toml/cmd/tomll
  tomll --help
  

• tomljson: Reads a TOML file and outputs its JSON representation.

  go install github.com/pelletier/go-toml/cmd/tomljson
  tomljson --help
  

Contribute

Feel free to report bugs and patches using GitHub's pull requests system on pelletier/go-toml. Any feedback would be much appreciated!

Run tests

You have to make sure two kind of tests run:

1. The Go unit tests
2. The TOML examples base

You can run both of them using ./test.sh.

License

The MIT License (MIT). Read LICENSE.

Documentation

Overview

Package toml is a TOML markup language parser.

This version supports the specification as described in https://github.com/toml-lang/toml/blob/master/versions/en/toml-v0.4.0.md

TOML Parsing

TOML data may be parsed in two ways: by file, or by string.

// load TOML data by filename
tree, err := toml.LoadFile("filename.toml")

// load TOML data stored in a string
tree, err := toml.Load(stringContainingTomlData)

Either way, the result is a TomlTree object that can be used to navigate the structure and data within the original document.

Getting data from the TomlTree

After parsing TOML data with Load() or LoadFile(), use the Has() and Get() methods on the returned TomlTree, to find your way through the document data.

if tree.Has("foo") {
  fmt.Println("foo is:", tree.Get("foo"))
}

Working with Paths

Go-toml has support for basic dot-separated key paths on the Has(), Get(), Set() and GetDefault() methods. These are the same kind of key paths used within the TOML specification for struct tames.

// looks for a key named 'baz', within struct 'bar', within struct 'foo'
tree.Has("foo.bar.baz")

// returns the key at this path, if it is there
tree.Get("foo.bar.baz")

TOML allows keys to contain '.', which can cause this syntax to be problematic for some documents. In such cases, use the GetPath(), HasPath(), and SetPath(), methods to explicitly define the path. This form is also faster, since it avoids having to parse the passed key for '.' delimiters.

// looks for a key named 'baz', within struct 'bar', within struct 'foo'
tree.HasPath([]string{"foo","bar","baz"})

// returns the key at this path, if it is there
tree.GetPath([]string{"foo","bar","baz"})

Note that this is distinct from the heavyweight query syntax supported by TomlTree.Query() and the Query() struct (see below).

Position Support

Each element within the TomlTree is stored with position metadata, which is invaluable for providing semantic feedback to a user. This helps in situations where the TOML file parses correctly, but contains data that is not correct for the application. In such cases, an error message can be generated that indicates the problem line and column number in the source TOML document.

// load TOML data
tree, _ := toml.Load("filename.toml")

// get an entry and report an error if it's the wrong type
element := tree.Get("foo")
if value, ok := element.(int64); !ok {
    return fmt.Errorf("%v: Element 'foo' must be an integer", tree.GetPosition("foo"))
}

// report an error if an expected element is missing
if !tree.Has("bar") {
   return fmt.Errorf("%v: Expected 'bar' element", tree.GetPosition(""))
}

Query Support

The TOML query path implementation is based loosely on the JSONPath specification: http://goessner.net/articles/JsonPath/

The idea behind a query path is to allow quick access to any element, or set of elements within TOML document, with a single expression.

result, err := tree.Query("$.foo.bar.baz")

This is roughly equivalent to:

next := tree.Get("foo")
if next != nil {
  next = next.Get("bar")
  if next != nil {
    next = next.Get("baz")
  }
}
result := next

err is nil if any parsing exception occurs.

If no node in the tree matches the query, result will simply contain an empty list of items.

As illustrated above, the query path is much more efficient, especially since the structure of the TOML file can vary. Rather than making assumptions about a document's structure, a query allows the programmer to make structured requests into the document, and get zero or more values as a result.

The syntax of a query begins with a root token, followed by any number sub-expressions:

$
                 Root of the TOML tree.  This must always come first.
.name
                 Selects child of this node, where 'name' is a TOML key
                 name.
['name']
                 Selects child of this node, where 'name' is a string
                 containing a TOML key name.
[index]
                 Selcts child array element at 'index'.
..expr
                 Recursively selects all children, filtered by an a union,
                 index, or slice expression.
..*
                 Recursive selection of all nodes at this point in the
                 tree.
.*
                 Selects all children of the current node.
[expr,expr]
                 Union operator - a logical 'or' grouping of two or more
                 sub-expressions: index, key name, or filter.
[start:end:step]
                 Slice operator - selects array elements from start to
                 end-1, at the given step.  All three arguments are
                 optional.
[?(filter)]
                 Named filter expression - the function 'filter' is
                 used to filter children at this node.

Query Indexes And Slices

Index expressions perform no bounds checking, and will contribute no values to the result set if the provided index or index range is invalid. Negative indexes represent values from the end of the array, counting backwards.

// select the last index of the array named 'foo'
tree.Query("$.foo[-1]")

Slice expressions are supported, by using ':' to separate a start/end index pair.

// select up to the first five elements in the array
tree.Query("$.foo[0:5]")

Slice expressions also allow negative indexes for the start and stop arguments.

// select all array elements.
tree.Query("$.foo[0:-1]")

Slice expressions may have an optional stride/step parameter:

// select every other element
tree.Query("$.foo[0:-1:2]")

Slice start and end parameters are also optional:

// these are all equivalent and select all the values in the array
tree.Query("$.foo[:]")
tree.Query("$.foo[0:]")
tree.Query("$.foo[:-1]")
tree.Query("$.foo[0:-1:]")
tree.Query("$.foo[::1]")
tree.Query("$.foo[0::1]")
tree.Query("$.foo[:-1:1]")
tree.Query("$.foo[0:-1:1]")

Query Filters

Query filters are used within a Union [,] or single Filter [] expression. A filter only allows nodes that qualify through to the next expression, and/or into the result set.

// returns children of foo that are permitted by the 'bar' filter.
tree.Query("$.foo[?(bar)]")

There are several filters provided with the library:

tree
       Allows nodes of type TomlTree.
int
       Allows nodes of type int64.
float
       Allows nodes of type float64.
string
       Allows nodes of type string.
time
       Allows nodes of type time.Time.
bool
       Allows nodes of type bool.

Query Results

An executed query returns a QueryResult object. This contains the nodes in the TOML tree that qualify the query expression. Position information is also available for each value in the set.

// display the results of a query
results := tree.Query("$.foo.bar.baz")
for idx, value := results.Values() {
    fmt.Println("%v: %v", results.Positions()[idx], value)
}

Compiled Queries

Queries may be executed directly on a TomlTree object, or compiled ahead of time and executed discretely. The former is more convienent, but has the penalty of having to recompile the query expression each time.

// basic query
results := tree.Query("$.foo.bar.baz")

// compiled query
query := toml.CompileQuery("$.foo.bar.baz")
results := query.Execute(tree)

// run the compiled query again on a different tree
moreResults := query.Execute(anotherTree)

User Defined Query Filters

Filter expressions may also be user defined by using the SetFilter() function on the Query object. The function must return true/false, which signifies if the passed node is kept or discarded, respectively.

// create a query that references a user-defined filter
query, _ := CompileQuery("$[?(bazOnly)]")

// define the filter, and assign it to the query
query.SetFilter("bazOnly", func(node interface{}) bool{
    if tree, ok := node.(*TomlTree); ok {
        return tree.Has("baz")
    }
    return false  // reject all other node types
})

// run the query
query.Execute(tree)

Example (ComprehensiveExample)

config, err := LoadFile("config.toml")

if err != nil {
	fmt.Println("Error ", err.Error())
} else {
	// retrieve data directly
	user := config.Get("postgres.user").(string)
	password := config.Get("postgres.password").(string)

	// or using an intermediate object
	configTree := config.Get("postgres").(*TomlTree)
	user = configTree.Get("user").(string)
	password = configTree.Get("password").(string)
	fmt.Println("User is ", user, ". Password is ", password)

	// show where elements are in the file
	fmt.Printf("User position: %v\n", configTree.GetPosition("user"))
	fmt.Printf("Password position: %v\n", configTree.GetPosition("password"))

	// use a query to gather elements without walking the tree
	results, _ := config.Query("$..[user,password]")
	for ii, item := range results.Values() {
		fmt.Printf("Query result %d: %v\n", ii, item)
	}
}

Index

• type NodeFilterFn
• type Position
  • func (p Position) Invalid() bool
  • func (p Position) String() string
• type Query
  • func CompileQuery(path string) (*Query, error)
  • func (q *Query) Execute(tree *TomlTree) *QueryResult
  • func (q *Query) SetFilter(name string, fn NodeFilterFn)
• type QueryResult
  • func (r QueryResult) Positions() []Position
  • func (r QueryResult) Values() []interface{}
• type TomlTree
  • func Load(content string) (tree *TomlTree, err error)
  • func LoadFile(path string) (tree *TomlTree, err error)
  • func LoadReader(reader io.Reader) (tree *TomlTree, err error)
  • func TreeFromMap(m map[string]interface{}) *TomlTree
  • func (t *TomlTree) Get(key string) interface{}
  • func (t *TomlTree) GetDefault(key string, def interface{}) interface{}
  • func (t *TomlTree) GetPath(keys []string) interface{}
  • func (t *TomlTree) GetPosition(key string) Position
  • func (t *TomlTree) GetPositionPath(keys []string) Position
  • func (t *TomlTree) Has(key string) bool
  • func (t *TomlTree) HasPath(keys []string) bool
  • func (t *TomlTree) Keys() []string
  • func (t *TomlTree) Query(query string) (*QueryResult, error)
  • func (t *TomlTree) Set(key string, value interface{})
  • func (t *TomlTree) SetPath(keys []string, value interface{})
  • func (t *TomlTree) String() string
  • func (t *TomlTree) ToMap() map[string]interface{}
  • func (t *TomlTree) ToTomlString() (string, error)
  • func (t *TomlTree) WriteTo(w io.Writer) (int64, error)

Examples

• Package (ComprehensiveExample)
• NodeFilterFn (FilterExample)
• Query (QueryExample)

Types

type NodeFilterFn

type NodeFilterFn func(node interface{}) bool

NodeFilterFn represents a user-defined filter function, for use with Query.SetFilter().

The return value of the function must indicate if 'node' is to be included at this stage of the TOML path. Returning true will include the node, and returning false will exclude it.

NOTE: Care should be taken to write script callbacks such that they are safe to use from multiple goroutines.

Example (FilterExample)

tree, _ := Load(`
      [struct_one]
      foo = "foo"
      bar = "bar"

      [struct_two]
      baz = "baz"
      gorf = "gorf"
    `)

// create a query that references a user-defined-filter
query, _ := CompileQuery("$[?(bazOnly)]")

// define the filter, and assign it to the query
query.SetFilter("bazOnly", func(node interface{}) bool {
	if tree, ok := node.(*TomlTree); ok {
		return tree.Has("baz")
	}
	return false // reject all other node types
})

// results contain only the 'struct_two' TomlTree
query.Execute(tree)

type Position

type Position struct {
	Line int // line within the document
	Col  int // column within the line
}

Position of a document element within a TOML document.

Line and Col are both 1-indexed positions for the element's line number and column number, respectively. Values of zero or less will cause Invalid(), to return true.

func (Position) Invalid

func (p Position) Invalid() bool

Invalid returns whether or not the position is valid (i.e. with negative or null values)

func (Position) String

func (p Position) String() string

String representation of the position. Displays 1-indexed line and column numbers.

type Query

type Query struct {
	// contains filtered or unexported fields
}

A Query is the representation of a compiled TOML path. A Query is safe for concurrent use by multiple goroutines.

Example (QueryExample)

config, _ := Load(`
      [[book]]
      title = "The Stand"
      author = "Stephen King"
      [[book]]
      title = "For Whom the Bell Tolls"
      author = "Ernest Hemmingway"
      [[book]]
      title = "Neuromancer"
      author = "William Gibson"
    `)

// find and print all the authors in the document
authors, _ := config.Query("$.book.author")
for _, name := range authors.Values() {
	fmt.Println(name)
}

func CompileQuery

func CompileQuery(path string) (*Query, error)

CompileQuery compiles a TOML path expression. The returned Query can be used to match elements within a TomlTree and its descendants.

func (*Query) Execute

func (q *Query) Execute(tree *TomlTree) *QueryResult

Execute executes a query against a TomlTree, and returns the result of the query.

func (*Query) SetFilter

func (q *Query) SetFilter(name string, fn NodeFilterFn)

SetFilter sets a user-defined filter function. These may be used inside "?(..)" query expressions to filter TOML document elements within a query.

type QueryResult

type QueryResult struct {
	// contains filtered or unexported fields
}

QueryResult is the result of Executing a Query.

func (QueryResult) Positions

func (r QueryResult) Positions() []Position

Positions is a set of positions for values within a QueryResult. Each index in Positions() corresponds to the entry in Value() of the same index.

func (QueryResult) Values

func (r QueryResult) Values() []interface{}

Values is a set of values within a QueryResult. The order of values is not guaranteed to be in document order, and may be different each time a query is executed.

type TomlTree

type TomlTree struct {
	// contains filtered or unexported fields
}

TomlTree is the result of the parsing of a TOML file.

func Load

func Load(content string) (tree *TomlTree, err error)

Load creates a TomlTree from a string.

func LoadFile

func LoadFile(path string) (tree *TomlTree, err error)

LoadFile creates a TomlTree from a file.

func LoadReader

func LoadReader(reader io.Reader) (tree *TomlTree, err error)

LoadReader creates a TomlTree from any io.Reader.

func TreeFromMap

func TreeFromMap(m map[string]interface{}) *TomlTree

TreeFromMap initializes a new TomlTree object using the given map.

func (*TomlTree) Get

func (t *TomlTree) Get(key string) interface{}

Get the value at key in the TomlTree. Key is a dot-separated path (e.g. a.b.c). Returns nil if the path does not exist in the tree. If keys is of length zero, the current tree is returned.

func (*TomlTree) GetDefault

func (t *TomlTree) GetDefault(key string, def interface{}) interface{}

GetDefault works like Get but with a default value

func (*TomlTree) GetPath

func (t *TomlTree) GetPath(keys []string) interface{}

GetPath returns the element in the tree indicated by 'keys'. If keys is of length zero, the current tree is returned.

func (*TomlTree) GetPosition

func (t *TomlTree) GetPosition(key string) Position

GetPosition returns the position of the given key.

func (*TomlTree) GetPositionPath

func (t *TomlTree) GetPositionPath(keys []string) Position

GetPositionPath returns the element in the tree indicated by 'keys'. If keys is of length zero, the current tree is returned.

func (*TomlTree) Has

func (t *TomlTree) Has(key string) bool

Has returns a boolean indicating if the given key exists.

func (*TomlTree) HasPath

func (t *TomlTree) HasPath(keys []string) bool

HasPath returns true if the given path of keys exists, false otherwise.

func (*TomlTree) Keys

func (t *TomlTree) Keys() []string

Keys returns the keys of the toplevel tree. Warning: this is a costly operation.

func (*TomlTree) Query

func (t *TomlTree) Query(query string) (*QueryResult, error)

Query compiles and executes a query on a tree and returns the query result.

func (*TomlTree) Set

func (t *TomlTree) Set(key string, value interface{})

Set an element in the tree. Key is a dot-separated path (e.g. a.b.c). Creates all necessary intermediates trees, if needed.

func (*TomlTree) SetPath

func (t *TomlTree) SetPath(keys []string, value interface{})

SetPath sets an element in the tree. Keys is an array of path elements (e.g. {"a","b","c"}). Creates all necessary intermediates trees, if needed.

func (*TomlTree) String

func (t *TomlTree) String() string

String generates a human-readable representation of the current tree. Alias of ToString. Present to implement the fmt.Stringer interface.

func (*TomlTree) ToMap

func (t *TomlTree) ToMap() map[string]interface{}

ToMap recursively generates a representation of the tree using Go built-in structures. The following types are used: * uint64 * int64 * bool * string * time.Time * map[string]interface{} (where interface{} is any of this list) * []interface{} (where interface{} is any of this list)

func (*TomlTree) ToTomlString

func (t *TomlTree) ToTomlString() (string, error)

ToTomlString generates a human-readable representation of the current tree. Output spans multiple lines, and is suitable for ingest by a TOML parser. If the conversion cannot be performed, ToString returns a non-nil error.

func (*TomlTree) WriteTo

func (t *TomlTree) WriteTo(w io.Writer) (int64, error)

WriteTo encode the TomlTree as Toml and writes it to the writer w. Returns the number of bytes written in case of success, or an error if anything happened.