README
¶
Rudi
Rudi is a Lisp-like, embeddable programming language that focuses on transforming data structures like those available in JSON (numbers, bools, objects, vectors etc.). Rudi programs consist of a series of statements that are evaluated in order:
(if (gt? .replicas 5) (error "too many replicas (%d)" .replicas))
(set! .spec.isAdmin (has-suffix? .spec.Email "@initech.com"))
(map! .spec.usages to-lower)
Rudi is great for making tasks like manipulating data, implementing policies, setting default values or normalizing data configurable.
Contents
Features
- Safe evaluation: Rudi is not Turing-complete and so Rudi programs are always guaranteed to complete in a reasonable time frame. You can add support for functions defined in Rudi code (i.e. at runtime), but this is optional to allow a safe embedded behaviour by default.
- Lightweight: Rudi comes with no runtime dependencies besides the Go stdlib.
- Hackable: Rudi tries to keep the language itself approachable, so that modifications are easier and newcomers have an easier time to get started.
- Variables can be pre-defined or set at runtime.
- JSONPath expressions are first-class citizens and make referring to the current JSON document a breeze.
- Optional Type Safety: Choose between pedantic, strict or humane typing for your programs.
Strict allows nearly no type conversions, humane allows for things like
1(int) turning into"1"(string) when needed. - Flexible: The Rudi CLI interpreter (
rudi) supports reading/writing JSON, JSON5, YAML and TOML.
Installation
Rudi is primarily meant to be embedded into other Go programs, but a standalone CLI application,
rudi, is also available to test your scripts with. rudi can be installed using Git & Go. Rudi
requires Go 1.18 or newer.
git clone https://codeberg.org/xrstf/rudi
cd rudi
make build
This will result in a rudi binary in _build/; to install system-wide, use make install.
Alternatively, you can download the latest release from GitHub.
Documentation
Make yourself familiar with Rudi using the documentation:
- The Language Description describes the Rudi syntax and semantics.
- All built-in functions are described in the standard library.
- Additional functions are available in the extended library.
- Type Handling describes how Rudi handles, converts and compares values.
Usage
Command Line
Rudi comes with a standalone CLI tool called rudi.
Usage of rudi:
-i, --interactive Start an interactive REPL to run expressions.
-s, --script string Load Rudi script from file instead of first argument (only in non-interactive mode).
-l, --library stringArray Load additional Rudi file(s) to be be evaluated before the script (can be given multiple times).
--var stringArray Define additional global variables (can be given multiple times).
-f, --stdin-format string What data format is used for data provided on stdin, one of [raw json json5 yaml yamldocs toml]. (default "yaml")
-o, --output-format string What data format to use for outputting data, one of [raw json yaml yamldocs toml]. (default "json")
--enable-funcs Enable the func! function to allow defining new functions in Rudi code.
-c, --coalesce string Type conversion handling, one of [strict pedantic humane]. (default "strict")
-h, --help Show help and documentation.
-V, --version Show version and exit.
--debug-ast Output syntax tree of the parsed script in non-interactive mode.
rudi can run in one of two modes:
-
Interactive Mode is enabled by passing
--interactive(or-i). This will start a REPL session where Rudi scripts are read from stdin and evaluated against the loaded files. -
Script Mode is used the an Rudi script is passed either as the first argument or read from a file defined by
--script. In this moderudiwill run all statements from the script and print the resulting value, then it exits.Examples:
rudi '.foo' myfile.jsonrudi '(set! .foo "bar") (set! .users 42) .' myfile.jsonrudi --script convert.rudi myfile.json
rudi has extensive help built right into it, try running rudi help to get started.
File Handling
Rudi can load JSON, JSON5, YAML and TOML files and will determine the file format based on the
file extension (.json for JSON, .json5 for JSON5, .yml and .yaml for YAML and .tml /
.toml for TOML). For data provided via stdin, rudi by default assumes YAML (or JSON) encoding.
If you want to use TOML/JSON5 instead, you must use the --stdin-format flag.
The first loaded file is known as the "document". Its content is available via path expressions like
.foo[0]. All loaded files are also available via the $files variable (i.e. . is the same as
$files[0] for reading, but when writing data, there is a difference between both notations; refer
to the docs for set for more information). Additionally the filenames are available in the
$filenames variable.
Additional raw files can be loaded using the --var flag: To load files, the format for this flag
is ENCODING:file:FILENAME, for example --var "myvar=yaml:file:config.kubeconfig". This allows
you to load files regardless of their extension and also allows to load raw files (that will be
kept as strings) using "myvar=raw:file:logo.png". Raw file encoding is not supported for files
given as arguments, those files must have a recognized file extension.
Embedding
Rudi is well suited to be embedded into Go applications. A clean and simple API makes it a breeze:
package main
import (
"fmt"
"log"
"go.xrstf.de/rudi"
"go.xrstf.de/rudi/pkg/coalescing"
)
const script = `(set! .foo 42) (+ $myvar 42 .foo)`
func main() {
// Rudi programs are meant to manipulate a document (path expressions like
// ".foo" resolve within that document). The document can be anything,
// but is most often a JSON object.
documentData := map[string]any{"foo": 9000}
// parse the script (the name is used when generating error strings)
program, err := rudi.Parse("myscript", script)
if err != nil {
log.Fatalf("The script is invalid: %v", err)
}
// evaluate the program;
// this returns an evaluated value, which is the result of the last expression
// that was evaluated, plus the final document state (the updatedData) after
// the script has finished.
updatedData, result, err := program.Run(
context.Background(),
documentData,
// setup the set of variables available by default in the script
rudi.NewVariables().Set("myvar", 42),
// Likewise, setup the functions available (note that this includes
// functions like "if" and "and", so running with an empty function set
// is generally not advisable).
rudi.NewSafeBuiltInFunctions(),
// Decide what kind of type strictness you would like; pedantic, strict
// or humane; choose your own adventure (strict is default if you use nil
// here; humane allows conversions like 1 == "1").
coalescing.NewStrict(),
)
if err != nil {
log.Fatalf("Script failed: %v", err)
}
fmt.Println(result) // => 126
fmt.Println(updatedData) // => {"foo": 42}
}
Alternatives
Rudi doesn't exist in a vacuum; there are many other great embeddable programming/scripting languages out there, allbeit with slightly different ideas and goals than Rudi:
-
Anko – Go-like syntax and allows recursion, making it more dangerous and hard to learn for non-developers than I'd like.
-
ECAL – Is an event-based system using rules which are triggered by events; comes with recursion as well and is therefore out.
-
Expr, GVal, CEL – Great languages for writing a single expression, but not suitable for transforming/mutating data structures.
-
Gentee – Is similar to C/Python and allows recursion, so both to powerful/dangerous and not my preference in terms of syntax.
-
Jsonnet – Probably one of the most obvious alternatives among this list. Jsonnet shines when constructing new elements and complexer configurations out of smaller pieces of information, less so when manipulating objects. Also I personally really am no fan of Jsonnet's syntax, plus: NIH.
-
Starlark – Is the language behind Bazel and actually has an optional nun-Turing-complete mode. However I am really no fan of its syntax and have not investigated it further.
-
Go Templates – I really don't like Go's template syntax for more than simple one-liners. I liked and copied its concept of ranging over things, as templates do not allow unbounded loops (just like Rudi), but apart from being safe to embed, Go templates do not offer enough functionality to modify a data structure. Like Jsonnet, templates shine when creating/outputting entire new documents.
Bonus mention: Mastermind's sprig served as inspiration for quite a few of the functions in Rudi.
Credits
Rudi has been named after my grandfather.
Thanks to @embik and @xmudrii for enduring my constant questions for feedback 😄
Rudi has been made possible by the amazing Pigeon parser generator.
License
MIT
Documentation
¶
Overview ¶
Package rudi contains a convenience API to compile and execute Rudi programs.
Index ¶
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
func NewFunctionBuilder ¶ added in v0.3.0
NewFunctionBuilder is the recommended way to define new Rudi functions. The function builder can take multiple forms (e.g. if you have (foo INT) and (foo STRING)) and will create a function that automatically evaluates and coalesces Rudi expressions and matches them to the given forms. The first matching form is then evaluated.
Types ¶
type Coalescer ¶ added in v0.0.11
type Coalescer = coalescing.Coalescer
Coalescer is responsible for type handling and equality rules. Build your own or use any of the predefined versions:
- coalescing.NewStrict() – mostly strict, but allows nulls to be converted and allows ints to become floats
- coalescing.NewPedantic() – even more strict, allows absolutely no conversions
- coalescing.NewHumane() – gentle type handling that allows lossless conversions like 1 => "1" or allowing (false == nil).
type Context ¶
Context is the evaluation context for a Rudi program, consisting of the global document, variables and functions.
type Document ¶
Document is the global document that is being processed by a Rudi script.
func NewDocument ¶
NewDocument wraps any sort of data as a Rudi document.
type Function ¶
Function is a single Rudi function, available to be used inside a Rudi script.
func NewLowLevelFunction ¶ added in v0.3.0
func NewLowLevelFunction(f types.TupleFunction, description string) Function
NewLowLevelFunction wraps a raw tuple function to be used in Rudi. This is mostly useful for defining really low-level functions and functions with special side effects. Most of the time, you'd want to use NewFunctionBuilder(), which will use reflection to make it much more straight forward to make a Go function available in Rudi.
type Functions ¶
Functions is a map of Rudi functions.
func NewFunctions ¶
func NewFunctions() Functions
NewFunctions returns an empty set of runtime functions.
func NewSafeBuiltInFunctions ¶ added in v0.5.0
func NewSafeBuiltInFunctions() Functions
NewSafeBuiltInFunctions returns a copy of all the safe built-in Rudi functions. These are all the functions that do not break runtime guarantees like programs always terminating in a reasonable time. See also NewUnsafeBuiltInFunctions, which contains functions like func! that allow to define new functions within Rudi code, but could lead to infinite loops or resource exhaustion.
func NewUnsafeBuiltInFunctions ¶ added in v0.5.0
func NewUnsafeBuiltInFunctions() Functions
NewUnsafeBuiltInFunctions returns a copy of all the unsafe built-in Rudi functions. These are functions with extended side effects, please refer to the documentation or code for which functions exactly are considered "unsafe" in Rudi.
type ParseError ¶
type ParseError struct {
// contains filtered or unexported fields
}
ParseErrors can occur while parsing a Rudi program.
func (ParseError) Error ¶
func (p ParseError) Error() string
Error returns the underlying parse error.
func (ParseError) Snippet ¶
func (p ParseError) Snippet() string
Snippet is the line of the program where the error occurred, marked with a caret and the error message in a second line below that.
type Program ¶
type Program interface {
fmt.Stringer
// Run will evaluate the program. The given data value is used as the program's
// document (i.e. available with bare path expressions like `.foo`). Variables
// can be left empty if desired, but funcs must effectively always be set,
// as programs without functions are very limited. Use NewBuiltInFunctions()
// to get the default set of functions in Rudi.
// When no error occurs, Run() returns both the final document value and the
// result of the final expression. Otherwise an error is returned.
Run(ctx context.Context, data any, variables Variables, funcs Functions, coalescer Coalescer) (document any, result any, err error)
// RunContext is like Run(), but uses a pre-setup Context and returns the
// bare final context instead of its document's value. The result is still
// the result of the final expression in the program.
RunContext(ctx Context) (result any, err error)
// DumpSyntaxTree writes the AST to the given writer. Useful for debugging.
// Set indent to false to prevent multiline output from being generated
// according to a simple, conservative linebreak algorithm.
// Note that the output looks like code, but is not executable/parseable. Use
// DumpRudi() if you need to turn a parsed program back into Rudi code.
DumpSyntaxTree(out io.Writer) error
// DumpRudi writes the AST in the form of parseable Rudi code.
DumpRudi(out io.Writer) error
}
Program is a parsed Rudi program, ready to be run (executed). Programs are stateless and can be executed multiple times, even concurrently (as long as a different context is used per goroutine, when using RunContext).
Source Files
Directories