README
¶
sdin: an API-first configuration language
sdin is a configuration language designed to have a simple API.
- Find the full description and code examples at https://iio.ie/sdin.
- Go documentation at https://pkg.go.dev/github.com/ypsu/sdin.
- Ask questions and report bugs at https://github.com/ypsu/sdin/discussions.
Documentation
¶
Overview ¶
Package sdin provides the structural, parsing, and formatting functions for sdin objects.
Example ¶
package main
import (
"fmt"
"github.com/ypsu/sdin"
)
const rawListSDIN = `# This is an sdin file that contains only values.
"alice"
"bob"
"charlie"
"dave"
`
const rawStructSDIN = `# This sdin file represents a collection of items.
Items+: {
Name: 'Independence Day'
Kind: DVD
Location: 'living room bookshelf'
}
Items+: {
Name: 'Teddy Bear'
Kind: Toy
Location: 'my bed'
Comment: '''
This is my best friend.
I cannot sleep without it.
'''
}
`
type item struct {
Name string
Kind sdin.Identifier
Location string
Comment string
}
func main() {
var names []string
var err error
// Unmarshaling a list.
names, err = sdin.Unmarshal[[]string](rawListSDIN)
if err != nil {
panic(err)
}
fmt.Printf("Input list has %d names, starting with %q, ending with %q.\n", len(names), names[0], names[len(names)-1])
// Marshaling a list.
fmt.Printf("sdin formatted version of the list:\n%s\n", sdin.Marshal(names))
// Simple AST manipulation: add a new item to the collection.
n, err := sdin.Unmarshal[*sdin.Node](rawStructSDIN)
if err != nil {
panic(err)
}
items := n.Get("Items")
items.Children = append(items.Children, sdin.NewNode("", item{
Name: "Toothbrush",
Kind: "Consumable",
Location: "bathroom",
Comment: "Mine is the blue one.",
}))
fmt.Printf("New, edited sdin:\n%s\n", n.Format())
// Fetching a specific field from an item.
var teddyComment string
teddyComment, err = items.Children[1].Get("Comment").As[string]()
if err != nil {
panic(err)
}
fmt.Printf("Teddy comment:\n%s", teddyComment)
}
Output: Input list has 4 names, starting with "alice", ending with "dave". sdin formatted version of the list: 'alice' 'bob' 'charlie' 'dave' New, edited sdin: # This sdin file represents a collection of items. Items+: { Name: 'Independence Day' Kind: DVD Location: 'living room bookshelf' } Items+: { Name: 'Teddy Bear' Kind: Toy Location: 'my bed' Comment: ''' This is my best friend. I cannot sleep without it. ''' } Items+: { Name: 'Toothbrush' Kind: Consumable Location: 'bathroom' Comment: 'Mine is the blue one.' } Teddy comment: This is my best friend. I cannot sleep without it.
Index ¶
- Variables
- func Marshal(v any) string
- func Tokenizable(v any) bool
- func Unmarshal[T any](s string) (T, error)
- type BareToken
- type Identifier
- type Interface
- type Kind
- func (k Kind) Bare() bool
- func (k Kind) Closing() bool
- func (k Kind) ComplementByte() byte
- func (k Kind) Composite() bool
- func (k Kind) Flat() bool
- func (k Kind) Form() bool
- func (k Kind) FormName() string
- func (k Kind) Identifier() bool
- func (k Kind) IdentifierPath() bool
- func (k Kind) List() bool
- func (k Kind) Name() bool
- func (k Kind) Nested() bool
- func (k Kind) OpenerByte() byte
- func (k Kind) Opening() bool
- func (k Kind) PackageName() string
- func (k Kind) Scalar() bool
- func (k Kind) String() bool
- func (k Kind) Struct() bool
- func (k Kind) Trivia() bool
- type Node
- func (n *Node) As[T any]() (T, error)
- func (n *Node) Directives() []*Node
- func (n *Node) FormFields() *Node
- func (n *Node) FormValues() *Node
- func (n *Node) Format() string
- func (n *Node) Get(path string) *Node
- func (n *Node) Kind() Kind
- func (n *Node) MarshalJSON() ([]byte, error)
- func (n *Node) Replace(fn func(n *Node) (*Node, error)) error
- func (n *Node) String() string
- func (n *Node) UnmarshalJSON(p []byte) error
- type Sanitizer
- type StringToken
- type Token
Examples ¶
Constants ¶
This section is empty.
Variables ¶
var ErrUnsupportedTokenIntoType = errors.New("sdin.UnsupportedTokenIntoType")
ErrUnsupportedTokenIntoType means that the type passed to Token.As doesn't implement the Interface.FromSDINToken function.
Functions ¶
func Marshal ¶
Marshal converts v to a *Node and formats it into a string. Formats into a sdin list if v is a slice. Panics if v contains an unsupported primitive type.
func Tokenizable ¶
Tokenizable returns true for values that MakeToken can convert to a token. For any other value MakeToken will panic.
Types ¶
type BareToken ¶
type BareToken string
BareToken represents any bare token. Bare tokens are all the raw sdin tokens that don't contain the :, [, ], {, }, (, ), #, ', ", space, tab, and newline symbols.
type Identifier ¶
type Identifier string
Identifier represents identifier tokens. Identifiers are bare tokens that start with a letter, and contain only letters, numbers, and underscores. Note that strings are not considered as identifiers.
type Interface ¶
type Interface interface {
FromSDINToken(Token) error
IntoSDINToken() Token
FromSDINNode(*Node) error
IntoSDINNode() *Node
}
Interface contains the list of functions a type can implement to interface with sdin's conversion functions. Note that any subset of these functions can be implemented and sdin will use it if available on the type; they are grouped into a common interface for documentational purposes.
type Kind ¶
type Kind string
Kind interprets a raw sdin token and makes its properties queriable. It's undefined to pass an empty string; the functions might panic.
func (Kind) Closing ¶
Closing returns true iff the token represents the closing token for a list, struct, or a form.
func (Kind) ComplementByte ¶
ComplementByte returns the complement byte of a list, struct, or form. Returns `]})` for `[{(` and vice versa, panics if scalar.
func (Kind) Flat ¶
Flat returns true iff the token represents a flat list, a flat struct, or a flat form.
func (Kind) Form ¶
Form returns true iff the token is in the form of `SomeIdentifier(` or `somepackage.SomeIdentifier(`.
func (Kind) FormName ¶
FormName returns SomeIdentifier or somepackage.SomeIdentifier for tokens like token `SomeIdentifier(` and `somepackage.SomeIdentifier(`. Returns empty string if token is not a form.
func (Kind) Identifier ¶
Identifier returns true iff the token matches the regexp `[A-Za-z_][0-9A-Za-z_]*`.
func (Kind) IdentifierPath ¶
IdentifierPath returns true iff the token is a dot-separated sequence of Identifiers.
func (Kind) Name ¶
Name returns true iff the token is in the form of `SomeIdentifier:` or `SomeIdentifier+:`.
func (Kind) Nested ¶
Nested returns true iff the token represents a nested list, nested struct, or a form.
func (Kind) OpenerByte ¶
OpenerByte returns the opener byte of a list, struct, or form. Returns one of `[{(` for composite nodes.
func (Kind) Opening ¶
Opening returns true iff the token represents the opening token for a list, struct, or a form.
func (Kind) PackageName ¶
PackageName returns the part before the first . for bare and form tokens. E.g. `sdin.KeepSorted(` -> `sdin`.
func (Kind) Scalar ¶
Scalar returns true iff the token represents a string or a bare value like an identifier or number.
type Node ¶
type Node struct {
// List of comments and newline tokens that appear before the value.
Pre []string
// Name, Value, and Post tokens are all defined on a single line.
// In structs, Name is an identifier which matches the regex `[a-zA-Z_][a-zA-Z0-9_]*`.
// In forms, Name can be any bare token.
Name string // the bare token before the colon, or empty if this is an unnamed node (e.g. in lists)
Value Token // the value token; can be `[`, `{`, or `tagname(` for composite nodes (`:[`, `:{`, `:(` for the flat versions)
Post string // the newline or comment, if any, after the value (or opening delimiter in composite nodes)
// The 1-based line on which the Value token was defined.
Line int
// Fields relevant only for struct/list/form nodes follow.
Children []*Node
// List of comments and newline tokens that appear before the matching }]) token for {[( values.
PreClose []string
// For {[( values this token appears after the matching }]) token.
// Can be a comment for a sidecomment, a newline to indicate EOL, or empty to keep the object on a single line.
PostClose string
}
Node represents an sdin value.
func NewFlatNode ¶
NewFlatNode creates an sdin node. Uses Interface.IntoSDINNode or Interface.IntoSDINToken if implemented. Pass "" as name if this is meant to be a list value where the name must be empty. If the result is a struct, a list, or an empty form, then the node is marked as flat. Panics for unsupported primitive types.
func NewNode ¶
NewNode creates an sdin node. Uses Interface.IntoSDINNode or Interface.IntoSDINToken if implemented. Pass "" as name if this is meant to be a list value where the name must be empty. Panics for unsupported primitive types.
Use NewNode(n.Name, n) to make a deep clone of an existing node.
func Parse ¶
Parse parses sdin text into a Node. kind determines whether the input sdin is a struct, a list of nameless values, or form arguments (both named and nameless values). It must be [TFlatForm()], [TFlatList()], or [TFlatStruct()].
func (*Node) As ¶
As converts a node into a T value. In structs, fields are fuzzily matched: case and underscores ignored. By default this function errors out when a struct doesn't have a matching field for a node. Create a dummy `_discardUnknownSDINFields struct{}` marker tag in the struct to discard non-matching fields. Implement Interface.FromSDINNode to override the default merging.
Note that the returned value might retain references to `n` or its children. For example, `As[*sdin.Node]` will return `n` as is.
If n is nil, As() returns an error.
func (*Node) Directives ¶
Directives returns the sdin values from the #% comments. It transforms invalid lines into error() directives with the parse error. Returns nil if no directives were found.
func (*Node) FormFields ¶
FormFields returns a new struct Node whose children are the named values of `n`. Always returns an empty node for list or nil nodes, and an equivalent node for struct nodes. This is primarily intended for forms, which can contain both named and unnamed values. This method never returns nil. The children are copied by reference, not via deep copy, so modifying them modifies `n`'s children too.
func (*Node) FormValues ¶
FormValues returns a new list Node whose children are the unnamed values of `n`. Always returns an empty node for struct or nil nodes, and an equivalent node for list nodes. This is primarily intended for forms, which can contain both named and unnamed values. This method never returns nil. The children are copied by reference, not via deep copy, so modifying them modifies `n`'s children too.
func (*Node) Format ¶
Format formats the given block or list node into well-indented text. Note that this might modify the node tree by adding/removing newlines where appropriate.
func (*Node) Get ¶
Get traverses into a child node specified via `path`. Returns nil if no such child exists or `n` itself is nil. `path` is a dot-separated list of integers and identifiers. For integers, Get traverses to the child with that index. For identifiers each traversal involves a linear scan of all child nodes. Create custom child maps if faster traversal is desired.
If `n` is `{a:{} b:{bb+:"x" bb+:"y"} c:{}}`, then `Get("b.bb.1").Value` will be `"y"`.
Example ¶
package main
import (
"fmt"
"github.com/ypsu/sdin"
)
func main() {
n, err := sdin.Parse(sdin.TFlatStruct(), `
a: {}
b: {bb:[x y]}
c: {}
`)
if err != nil {
fmt.Printf("ParsingForGetFailed: %v", err)
return
}
fmt.Println(n.Get("b.bb.1").Format())
}
Output: y
func (*Node) Kind ¶
Kind returns the Kind object that can be used to query the properties of this node's value.
func (*Node) MarshalJSON ¶
MarshalJSON renders node as a JSON value. This implements json.Marshaler.
func (*Node) Replace ¶
Replace recursively traverses all children, calls the passed function for them, and replaces them in the parent node with the returned node. Return `n, nil` to keep a node intact. If the function returns nil, the node is deleted from the parent's children. Can be used to implement function calls or to simply visit all nodes. The line number and node name are copied from the original child when the replacement node has those fields unset.
Note that the replacer function isn't called for the passed in root node because Replace() doesn't know its parent. If fn() returns a non-nil error, the loop aborts and the tree remains partially processed.
Example ¶
Here's an example where we replace env() forms with values from the environment.
package main
import (
"fmt"
"os"
"github.com/ypsu/sdin"
)
func main() {
n, err := sdin.Parse(sdin.TFlatStruct(), `
command: 'runsql'
arguments: {
query: 'SELECT * FROM mytable'
database: env(DATABASE)
password: env(PASSWORD)
}
`)
if err != nil {
fmt.Printf("ParsingForReplaceFailed: %v", err)
return
}
err = n.Replace(func(n *sdin.Node) (*sdin.Node, error) {
if n.Kind().FormName() != "env" {
return n, nil
}
if len(n.Children) != 1 || n.Children[0].Name != "" || !n.Children[0].Kind().Identifier() {
return nil, fmt.Errorf("InvalidEnvArgs: env takes exactly one identifier arg")
}
value, ok := os.LookupEnv(n.Children[0].Value.Repr())
if !ok {
return nil, fmt.Errorf("EnvvarNotFound var=%s", n.Children[0].Value.Repr())
}
return sdin.NewNode("", value), nil
})
if err != nil {
fmt.Printf("EnvReplaceFailed: %v", err)
return
}
fmt.Print(n.Format())
}
Output: command: 'runsql' arguments: { query: 'SELECT * FROM mytable' database: 'db.example.com' password: 'hunter2' }
func (*Node) String ¶
String returns a compact, one-line string representation without any comments or other fluff.
func (*Node) UnmarshalJSON ¶
UnmarshalJSON parses JSON data into this Node. This implements json.Unmarshaler.
type Sanitizer ¶
type Sanitizer struct {
// The list of allowed packages.
// If nil, no package checks are done unless the root node preamble specifies a list with sdin.FormPackages or sdin.DirectivePackages.
// If both this and the preamble are specified, then the allowed package list is the intersection of the two lists.
DirectivePackages []string
FormPackages []string
}
Sanitizer checks package references and validates whether the tokens are valid. It also reorders sdin.KeepSorted() nodes and reformats sdin tagged strings.
type StringToken ¶
type StringToken struct {
// Opening quote, includes the tag and everything up to the first | or \n for multiquote strings.
// If empty when encoding, a quote will be automatically picked via some heuristics.
// In the multiquote case the number of quotes will be increased as necessary to encode Data correctly.
// In the multiline case this contains the indentation string after the newline.
Quote string
// The string itself with indentation removed for multiline strings.
Data string
}
StringToken represents string tokens losslessly.
func (StringToken) Tag ¶
func (st StringToken) Tag() string
Tag extracts the raw string's tag from its quote. Example:
The tag of '''some tag\n\t\t|...''' is "some tag".
type Token ¶
type Token struct {
// contains filtered or unexported fields
}
Token represents an sdin token.
func MakeToken ¶
MakeToken creates an SDIN bare token to represent v. Uses IntoSDINToken for types implementing that. Integers, floats, times, durations become numeric tokens. Identifier values become identifier tokens. It stringifies the rest and converts them to sdin strings. If the string contains newlines then it becomes a multiline string otherwise a quoted string. It panics if used with the wrong type or invalid value (e.g. empty Identifier).
func MakeTokenFromRepr ¶
MakeTokenFromRepr casts a raw token representation to Token. Only use it if you know what you are doing.
Source Files
¶
Directories
¶
| Path | Synopsis |
|---|---|
|
Package cmd implements the command line tool's internals.
|
Package cmd implements the command line tool's internals. |
|
sdin
command
Command sdin is the command-line interface to sdin related utilities.
|
Command sdin is the command-line interface to sdin related utilities. |
|
The precommit tool runs checks and tests to maintain a high quality codebase.
|
The precommit tool runs checks and tests to maintain a high quality codebase. |
|
Package tokens provides internal helper functions for working with sdin data.
|
Package tokens provides internal helper functions for working with sdin data. |