tomlsawyer

package module
v0.0.0-...-8bed390 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Mar 29, 2026 License: MIT Imports: 13 Imported by: 0

README

tomlsawyer - Comment-Preserving TOML Library for Go

Go Reference

A Go library for parsing, modifying, and serializing TOML documents while preserving comments, declaration order, and structural style choices. It wraps the excellent creachadair/tomledit library to provide a stable, user-friendly API.

Features

  • Comment Preservation: All comments (block, inline, and trailing) are preserved during read-modify-write operations
  • Format Preservation: Declaration order, comment placement, and structural style choices are preserved. Whitespace within inline tables may be normalized by the formatter
  • Order Preservation: Declaration order of keys and sections is preserved
  • Quote Style Preservation: String quote styles (single ', double ", multiline """ or ''') are preserved when modifying values
  • Nestedness Style Preservation: Dotted keys (server.host = "...") vs section style ([server] + host = "...") are preserved
  • Simple API: Easy-to-use interface with Get, Set, Delete, Has, and Keys methods
  • Dotted Path Support: Access nested values using dotted paths (e.g., server.database.host)
  • Quoted Key Support: Full support for TOML quoted keys with special characters (e.g., aliases.".", section."key with spaces")
  • Path Validation: Validates paths according to TOML specification, rejecting invalid paths like aliases. (trailing dot)
  • Broad TOML v1.0.0 Support: Supports most TOML features including arrays, inline tables, and multiline strings (see Limitations for details)
  • Extensively Tested: Comprehensive test suite covering all edge cases

Installation

go get github.com/neongreen/tomlsawyer

Quick Start

package main

import (
	"fmt"
	"log"

	toml "github.com/neongreen/tomlsawyer"
)

func main() {
	// Parse a TOML document
	input := `# Server configuration
[server]
host = "localhost"  # The host to bind to
port = 8080         # The port to listen on

[database]
url = "postgres://localhost/db"
`

	doc, err := toml.ParseString(input)
	if err != nil {
		log.Fatal(err)
	}

	// Read values
	host, _ := doc.Get("server.host")
	fmt.Println("Host:", host) // Output: Host: localhost

	// Modify values (comments are preserved!)
	doc.Set("server.port", 9090)
	doc.Set("server.debug", true)

	// Delete values
	doc.Delete("database.url")

	// Check if a value exists
	if doc.Has("server.host") {
		fmt.Println("Host is configured")
	}

	// Write back to TOML (with all comments preserved)
	fmt.Println(doc.String())
}

Path Syntax

All methods that accept a path parameter use TOML dotted-key syntax to address nested values:

  • Simple keys: "name", "version"
  • Nested keys: "server.host", "database.connection.timeout"
  • Quoted keys for special characters: aliases.".", section."key with spaces"
  • Mixed: config."my.setting".value

The path is parsed using TOML's own key grammar, so any valid TOML key works as a path segment when quoted. Invalid paths like "foo." (trailing dot) or ".foo" (leading dot) are rejected.

Important: path segments and raw key names are different things. A path like aliases."." has two segments (aliases and .). The Keys() method returns raw key names — so it would return "." as a string, not "aliases.\".\"".

API Documentation

Parsing
Parse(input []byte) (*Document, error)

Parses a TOML document from a byte slice.

data, _ := os.ReadFile("config.toml")
doc, err := toml.Parse(data)
ParseString(input string) (*Document, error)

Parses a TOML document from a string.

doc, err := toml.ParseString(`
name = "myapp"
version = 1
`)
Reading Values
Get(path string) (interface{}, error)

Retrieves a value at the given dotted path. Returns nil if the path doesn't exist.

value, err := doc.Get("server.port")
if err != nil {
    log.Fatal(err)
}
fmt.Printf("Port: %v\n", value)

Supported return types:

  • string - for string values
  • int64 - for integer values
  • float64 - for floating-point values
  • bool - for boolean values
  • []interface{} - for arrays
  • map[string]interface{} - for inline tables
Has(path string) bool

Returns true if the given path exists in the document.

if doc.Has("server.port") {
    fmt.Println("Port is configured")
}
Keys(path string) ([]string, error)

Returns the child keys of the table at the given path. Use TopLevelKeys() to get top-level keys.

doc, _ := toml.ParseString(`
[servers]
[servers.alpha]
ip = "10.0.0.1"
[servers.beta]
ip = "10.0.0.2"
`)

keys, _ := doc.Keys("servers")
fmt.Println(keys) // Output: [alpha beta]
Modifying Values
Set(path string, value interface{}) error

Sets a value at the given dotted path, creating intermediate sections if necessary. If the key already exists, its comments are preserved.

Supported value types:

  • string, int, int64, float64, bool
  • []interface{}, []string, []int
  • map[string]interface{}
// Set a simple value
doc.Set("server.port", 8080)

// Set a nested value (creates [server] section if needed)
doc.Set("server.tls.enabled", true)

// Set an array
doc.Set("server.hosts", []string{"localhost", "example.com"})

// Set an inline table
doc.Set("person", map[string]interface{}{
    "name": "Alice",
    "age":  30,
})
Delete(path string) error

Removes a key at the given dotted path.

doc.Delete("server.debug")
Serialization
String() string

Serializes the document back to TOML format as a string, preserving all comments and formatting.

output := doc.String()
fmt.Println(output)
Bytes() []byte

Serializes the document back to TOML format as a byte slice.

data := doc.Bytes()
os.WriteFile("config.toml", data, 0644)

Examples

Example 1: Configuration File Management
package main

import (
	"fmt"
	"log"
	"os"

	toml "github.com/neongreen/tomlsawyer"
)

func main() {
	// Read existing config
	data, err := os.ReadFile("app.toml")
	if err != nil {
		log.Fatal(err)
	}

	doc, err := toml.Parse(data)
	if err != nil {
		log.Fatal(err)
	}

	// Update configuration
	doc.Set("app.version", "2.0.0")
	doc.Set("app.debug", false)
	doc.Set("server.max_connections", 1000)

	// Save back to file (comments preserved!)
	err = os.WriteFile("app.toml", doc.Bytes(), 0644)
	if err != nil {
		log.Fatal(err)
	}

	fmt.Println("Configuration updated successfully")
}
Example 2: Comment and Style Preservation
package main

import (
	"fmt"

	toml "github.com/neongreen/tomlsawyer"
)

func main() {
	input := `# Application settings
app_name = "myapp"    # The name of the application
version = 1           # Current version
license = 'MIT'       # Single-quoted string

# Database configuration
[database]
# Connection settings
host = "localhost"
port = 5432
`

	doc, _ := toml.ParseString(input)

	// Modify values - comments AND quote styles are preserved!
	doc.Set("version", 2)           // Comment preserved
	doc.Set("license", "Apache-2.0") // Single quotes preserved
	doc.Set("database.port", 5433)  // Comment preserved

	fmt.Println(doc.String())
	// Output:
	// # Application settings
	// app_name = "myapp"    # The name of the application
	// version = 2           # Current version
	// license = 'Apache-2.0'       # Single-quoted string (note: STILL single-quoted!)
	//
	// # Database configuration
	// [database]
	// # Connection settings
	// host = "localhost"
	// port = 5433
}
Example 3: Working with Complex Structures
package main

import (
	"fmt"

	toml "github.com/neongreen/tomlsawyer"
)

func main() {
	doc, _ := toml.ParseString("")

	// Create a complex nested structure
	doc.Set("servers.alpha.ip", "10.0.0.1")
	doc.Set("servers.alpha.dc", "eqdc10")
	doc.Set("servers.beta.ip", "10.0.0.2")
	doc.Set("servers.beta.dc", "eqdc10")

	// Set arrays
	doc.Set("database.ports", []int{8001, 8002, 8003})

	// Set inline tables
	doc.Set("owner", map[string]interface{}{
		"name":  "Tom",
		"email": "tom@example.com",
	})

	fmt.Println(doc.String())
}
Example 4: Style Preservation (Dotted Keys vs Sections)
package main

import (
	"fmt"

	toml "github.com/neongreen/tomlsawyer"
)

func main() {
	// Example with dotted key style
	input1 := `
server.host = "localhost"
server.port = 8080
`
	doc1, _ := toml.ParseString(input1)
	doc1.Set("server.host", "0.0.0.0")  // Stays as dotted key!
	fmt.Println(doc1.String())
	// Output:
	// server.host = "0.0.0.0"
	// server.port = 8080

	// Example with section style
	input2 := `
[server]
host = "localhost"
port = 8080
`
	doc2, _ := toml.ParseString(input2)
	doc2.Set("server.host", "0.0.0.0")  // Stays in [server] section!
	fmt.Println(doc2.String())
	// Output:
	// [server]
	// host = "0.0.0.0"
	// port = 8080
}
Example 5: Migrating Configuration Values
package main

import (
	"fmt"

	toml "github.com/neongreen/tomlsawyer"
)

func main() {
	input := `
[old_section]
setting = "value"

[server]
host = "localhost"
`

	doc, _ := toml.ParseString(input)

	// Migrate old setting to new location
	if oldValue, _ := doc.Get("old_section.setting"); oldValue != nil {
		doc.Set("server.setting", oldValue)
		doc.Delete("old_section.setting")
	}

	fmt.Println(doc.String())
}
Example 6: Working with Quoted Keys

TOML allows special characters in keys when they are quoted. This is useful for tools like jj (Jujutsu VCS) that use single-character aliases.

package main

import (
	"fmt"

	toml "github.com/neongreen/tomlsawyer"
)

func main() {
	// Example: jj-style aliases with quoted single-character keys
	input := `
[aliases]
"." = "status"
".." = "show @-"
"..." = "show @--"
l = "log"
`

	doc, _ := toml.ParseString(input)

	// Get quoted keys - use backslash escaping or raw strings
	status, _ := doc.Get(`aliases."."`)
	fmt.Println("Alias '.':", status) // Output: Alias '.': status

	showParent, _ := doc.Get(`aliases.".."`)
	fmt.Println("Alias '..':", showParent) // Output: Alias '..': show @-

	// Set new quoted keys
	doc.Set(`aliases."!!!!"`, "diff")

	// Regular keys work as usual
	log, _ := doc.Get("aliases.l")
	fmt.Println("Alias 'l':", log) // Output: Alias 'l': log

	// Invalid paths are rejected
	_, err := doc.Get("aliases.") // trailing dot
	fmt.Println("Invalid path error:", err != nil) // Output: Invalid path error: true

	fmt.Println(doc.String())
	// Output:
	// [aliases]
	// "." = "status"
	// ".." = "show @-"
	// "..." = "show @--"
	// l = "log"
	// "!!!!" = "diff"
}

Path Validation:

The library uses TOML-compliant path parsing that:

  • Accepts: aliases.".", section."key with spaces", config."key-with-dashes"
  • Rejects: aliases. (trailing dot), .aliases (leading dot), aliases..key (double dot)
Accessing Arrays
doc, _ := toml.ParseString(`
tags = ["go", "toml", "parser"]
`)

tags, _ := doc.Get("tags")
if arr, ok := tags.([]interface{}); ok {
	for i, tag := range arr {
		fmt.Printf("Tag %d: %v\n", i, tag)
	}
}
Accessing Inline Tables
doc, _ := toml.ParseString(`
person = { name = "Alice", age = 30 }
`)

person, _ := doc.Get("person")
if table, ok := person.(map[string]interface{}); ok {
	fmt.Println("Name:", table["name"])
	fmt.Println("Age:", table["age"])
}
Type Assertions

When retrieving values, you'll need to assert the type:

// String
if str, ok := value.(string); ok {
	fmt.Println(str)
}

// Integer (always returned as int64)
if num, ok := value.(int64); ok {
	fmt.Println(num)
}

// Float
if f, ok := value.(float64); ok {
	fmt.Println(f)
}

// Boolean
if b, ok := value.(bool); ok {
	fmt.Println(b)
}

Testing

The library includes a comprehensive test suite covering:

  • Parsing various TOML formats
  • Getting and setting values
  • Deleting values
  • Comment preservation (block, inline, trailing)
  • Quote style preservation (single, double, multiline)
  • Key order preservation
  • Nestedness style preservation (dotted keys vs sections)
  • Round-trip parsing and serialization
  • Arrays and inline tables
  • Array of tables ([[name]] syntax)
  • Array of inline tables with formatting preservation
  • Quoted keys ("key with spaces", [foo."bar:baz".qux])
  • Malformed TOML detection and error handling
  • Edge cases and special syntax
  • Unicode support
  • Large and complex documents
  • Multiple modifications and deletions
  • Concurrent access

Run tests with:

go test -v ./...

Comparison with Other Libraries

Feature tomlsawyer go-toml/v2 BurntSushi/toml
Comment preservation Yes No No
Format preservation Yes* No No
Order preservation Yes No No
Quote style preservation Yes No No
Nestedness preservation Yes No No
Marshal/Unmarshal No Yes Yes
Struct tags No Yes Yes
Use case Config editing Data serialization Data serialization

* Whitespace within inline tables may be normalized by the formatter.

When to use tomlsawyer:

  • You need to edit TOML files while preserving comments
  • You're building a configuration management tool
  • You need to maintain human-readable formatting (quote styles, key order, nestedness style)
  • You want to programmatically update config files without losing documentation
  • You need to respect the original author's formatting choices

When to use go-toml/v2 or BurntSushi/toml:

  • You just need to deserialize TOML into Go structs
  • Comments and formatting don't matter
  • You need the fastest parsing performance

Implementation Details

tomlsawyer is built on top of creachadair/tomledit, which provides the low-level AST-based parsing and formatting capabilities. tomlsawyer adds:

  • A higher-level, more ergonomic API
  • Automatic comment preservation when updating values
  • Simplified path-based access to nested values
  • Type conversion helpers

Limitations

  • The library works at the AST level, not the semantic level. It preserves the syntactic structure but doesn't validate semantic constraints (like duplicate keys).
  • When creating new values programmatically, they're added without comments unless you modify the underlying AST.
  • Marshal/unmarshal to Go structs is not supported (use go-toml/v2 or BurntSushi/toml for that).
  • Arrays of tables ([[name]]) are preserved during round-trip, but individual entries are not addressable via the Get/Set API.
  • ApplyMap/ReplaceMap don't support arrays of tables.

License

This project is licensed under the MIT License.

Contributing

Contributions are welcome! Please feel free to submit issues or pull requests.

Acknowledgments

Documentation

Overview

Package tomlsawyer provides comment-preserving TOML parsing and serialization.

This package wraps github.com/creachadair/tomledit to provide a stable, user-friendly API for reading, modifying, and writing TOML files while preserving comments, declaration order, and structural style choices (dotted keys vs sections, quote styles). Whitespace layout may be normalized.

Path Syntax

All methods that accept a path parameter use TOML dotted-key syntax: simple keys like "name", nested keys like "server.host", and quoted keys for special characters like aliases."." or section."key with spaces". Paths are parsed using TOML's own key grammar, so any valid TOML key works as a path segment when quoted. Invalid paths like "foo." (trailing dot) or ".foo" (leading dot) are rejected.

Path segments and raw key names are different things. A path like aliases."." has two segments (aliases and .). The Document.Keys method returns raw key names, not paths.

Example usage:

doc, err := tomlsawyer.Parse([]byte(`
  # This is a comment
  [server]
  host = "localhost"
  port = 8080
`))
if err != nil {
    log.Fatal(err)
}

// Modify a value
doc.Set("server.port", 9090)

// Use quoted keys for special characters
doc.Set(`aliases."."`, "status")

// Write back with comments preserved
output := doc.String()

Index

Examples

Constants

This section is empty.

Variables

View Source
var ErrNotValue = errors.New("path refers to a table section, not a value")

ErrNotValue is returned by Get when the path refers to a table section rather than a key-value entry.

Functions

func FormatValueToString

func FormatValueToString(v any) (string, error)

FormatValueToString converts a Go value into a TOML value string. This is useful for displaying values in a TOML-compatible format.

func WriteFile

func WriteFile(path string, values map[string]any) error

WriteFile writes the provided values to the given TOML file while preserving all existing comments and formatting. Missing parent directories are created automatically.

Types

type Document

type Document struct {
	// contains filtered or unexported fields
}

Document represents a parsed TOML document with preserved structure, including comments, formatting, and declaration order.

func Parse

func Parse(input []byte) (*Document, error)

Parse parses a TOML document from bytes and returns a Document that preserves all structural information including comments.

func ParseString

func ParseString(input string) (*Document, error)

ParseString is a convenience wrapper around Parse that accepts a string.

func (*Document) ApplyMap

func (d *Document) ApplyMap(values map[string]any) error

ApplyMap merges the provided nested map into the document, preserving any existing keys that are not present in values. This allows partial updates without affecting unmanaged configuration.

Example
package main

import (
	"fmt"

	"github.com/neongreen/tomlsawyer"
)

func main() {
	doc, _ := tomlsawyer.ParseString(`name = "myapp"
version = "1.0"
`)
	doc.ApplyMap(map[string]any{
		"version": "2.0",
		"author":  "Alice",
	})
	fmt.Print(doc.String())
}
Output:
name = "myapp"
version = "2.0"
author = "Alice"

func (*Document) Bytes

func (d *Document) Bytes() []byte

Bytes serializes the document back to TOML format as a byte slice.

func (*Document) Delete

func (d *Document) Delete(path string) error

Delete removes a key at the given path. Returns nil if the path doesn't exist.

Path syntax is the same as Document.Get.

func (*Document) Get

func (d *Document) Get(path string) (any, bool, error)

Get retrieves a value at the given path. Returns (value, true, nil) when the path exists and is a key-value entry, (nil, false, nil) when the path doesn't exist, and (nil, false, ErrNotValue) when the path refers to a table section.

The path uses TOML dotted-key syntax: segments are separated by dots. Use quoted segments for keys containing special characters:

doc.Get("server.host")       // key "host" under section "server"
doc.Get(`aliases."."`)       // key "." under section "aliases"
Example
package main

import (
	"fmt"

	"github.com/neongreen/tomlsawyer"
)

func main() {
	doc, _ := tomlsawyer.ParseString(`
[server]
host = "localhost"
port = 8080
`)
	host, _, _ := doc.Get("server.host")
	fmt.Println(host)
}
Output:
localhost

func (*Document) Has

func (d *Document) Has(path string) (bool, error)

Has reports whether the given path exists in the document. This includes key-value entries, table sections, and dotted key prefixes. Parse errors (e.g. invalid paths like "foo.") are returned as the second value.

Path syntax is the same as Document.Get.

Example
package main

import (
	"fmt"

	"github.com/neongreen/tomlsawyer"
)

func main() {
	doc, _ := tomlsawyer.ParseString(`
[server]
host = "localhost"
`)
	h1, _ := doc.Has("server")
	h2, _ := doc.Has("server.host")
	h3, _ := doc.Has("nonexistent")
	fmt.Println(h1)
	fmt.Println(h2)
	fmt.Println(h3)
}
Output:
true
true
false

func (*Document) Keys

func (d *Document) Keys(path string) ([]string, error)

Keys returns the child key names under the given path. For a table section like [foo], it returns the keys defined in that section. For an inline table value, it returns the keys of that inline table. Returns nil if the path doesn't exist or has no children.

Keys are returned in document order. For inline tables, this is the order they appear in the source. For Go maps passed to Set or FormatValueToString, keys are sorted alphabetically since Go maps have no inherent order.

The returned names are raw key names, not paths. For example, Keys("aliases") might return [".", "..", "l"] where "." is a literal key name containing a dot.

Path syntax is the same as Document.Get.

Example
package main

import (
	"fmt"

	"github.com/neongreen/tomlsawyer"
)

func main() {
	doc, _ := tomlsawyer.ParseString(`
[users]
alice = { role = "admin" }
bob = { role = "user" }
`)
	keys, _ := doc.Keys("users")
	fmt.Println(keys)
}
Output:
[alice bob]
Example (QuotedKeys)
package main

import (
	"fmt"

	"github.com/neongreen/tomlsawyer"
)

func main() {
	doc, _ := tomlsawyer.ParseString(`
[aliases]
"." = "status"
".." = "show @-"
l = "log"
`)
	keys, _ := doc.Keys("aliases")
	for _, k := range keys {
		val, _, _ := doc.Get(fmt.Sprintf(`aliases."%s"`, k))
		fmt.Printf("%s = %s\n", k, val)
	}
}
Output:
. = status
.. = show @-
l = log

func (*Document) Move

func (d *Document) Move(oldPath, newPath string) error

Move renames or moves a section or key, like Unix mv. Both oldPath and newPath use the same path syntax as Document.Get.

For sections, child sections are also renamed (e.g. moving "foo.bar" to "foo.qux" also renames "foo.bar.baz" to "foo.qux.baz"). Comments, formatting, and position are preserved.

For key-value entries within the same section (e.g. "server.host" to "server.addr"), the key is renamed in place. For cross-section moves (e.g. "server.host" to "app.host"), the key is removed from the source section and inserted into the destination, creating it if needed. Block comments and inline comments on the key are preserved.

Example
package main

import (
	"fmt"

	"github.com/neongreen/tomlsawyer"
)

func main() {
	doc, _ := tomlsawyer.ParseString(`
[server]
host = "localhost"
port = 8080
`)
	doc.Move("server", "app.server")
	fmt.Print(doc.String())
}
Output:
[app.server]
host = "localhost"
port = 8080
Example (CrossSection)
package main

import (
	"fmt"

	"github.com/neongreen/tomlsawyer"
)

func main() {
	doc, _ := tomlsawyer.ParseString(`
[old]
timeout = 30

[new]
host = "localhost"
`)
	doc.Move("old.timeout", "new.timeout")
	doc.Prune() // remove empty [old] section
	fmt.Print(doc.String())
}
Output:
[new]
host = "localhost"
timeout = 30

func (*Document) Prune

func (d *Document) Prune()

Prune removes empty sections from the document. A section is considered empty if it contains no key-value items (comments-only sections are also removed). This is useful after Delete or Move operations that may leave behind stale section headers.

func (*Document) ReplaceMap

func (d *Document) ReplaceMap(values map[string]any) error

ReplaceMap updates the document with the provided nested map and removes any keys that are not present in values. Use ApplyMap instead if you want to preserve unmanaged keys.

func (*Document) Set

func (d *Document) Set(path string, value any) error

Set sets a value at the given path, creating intermediate sections if necessary. The value can be a string, int, float64, bool, or []interface{}. This method preserves the original style (dotted keys vs sections, quote styles).

Path syntax is the same as Document.Get.

Example
package main

import (
	"fmt"

	"github.com/neongreen/tomlsawyer"
)

func main() {
	doc, _ := tomlsawyer.ParseString(`
[server]
host = "localhost"
port = 8080
`)
	doc.Set("server.port", 9090)
	fmt.Print(doc.String())
}
Output:
[server]
host = "localhost"
port = 9090

func (*Document) String

func (d *Document) String() string

String serializes the document back to TOML format, preserving all comments, formatting, and declaration order.

func (*Document) TopLevelKeys

func (d *Document) TopLevelKeys() []string

TopLevelKeys returns all top-level key names in the document, deduplicated and in document order. The returned strings are raw key names (not paths).

Example
package main

import (
	"fmt"

	"github.com/neongreen/tomlsawyer"
)

func main() {
	doc, _ := tomlsawyer.ParseString(`
name = "myapp"
version = 1

[server]
host = "localhost"

[database]
url = "postgres://localhost"
`)
	keys := doc.TopLevelKeys()
	fmt.Println(keys)
}
Output:
[name version server database]

type KeyValue

type KeyValue struct {
	Key   string
	Value any
}

KeyValue represents an ordered key-value pair for use in inline tables. Pass []KeyValue to Document.Set or FormatValueToString to control the output order of inline table keys. For unordered input, map[string]any is also accepted (keys will be sorted alphabetically).

Directories

Path Synopsis
examples
basic command
config_manager command

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL