README
¶
JSONTemplate
jsontemplate is a JSON transformation and templating language and library implemented in Go.
In simple terms, it renders arbitrary JSON structures based on a JSON-like template definition, populating it with data from an some other JSON structure.
Feature overview
- Low-clutter template syntax, aimed to look similar to the final output.
- JSONPath expressions to fetch values from the input.
- Array generator expressions with subtemplates, allowing mapping of arrays of objects.
- Ability to call Go functions from within a template.
Getting started
The following is a complete but minimal program that loads a template and transforms an input JSON object using it.
package main
import (
"os"
"strings"
"github.com/Volumental/jsontemplate"
)
const input = `{ "snakeCase": 123 }`
func main() {
template, _ := jsontemplate.ParseString(`{ "CamelCase": $.snakeCase }`, nil)
template.RenderJSON(os.Stdout, strings.NewReader(input))
os.Stdout.Sync()
}
Running the above program will output:
{"CamelCase":123}
Features by example
This example illustrates some of the features in jsontemplate. For further details, please see the library documentation.
Consider the following input JSON structure:
{
"store": {
"book": [
{
"category": "reference",
"author": "Nigel Rees",
"title": "Sayings of the Century",
"price": 8.95
},
{
"category": "fiction",
"author": "Evelyn Waugh",
"title": "Sword of Honour",
"price": 12.99
},
{
"category": "fiction",
"author": "Herman Melville",
"title": "Moby Dick",
"isbn": "0-553-21311-3",
"price": 8.99
},
{
"category": "fiction",
"author": "J. R. R. Tolkien",
"title": "The Lord of the Rings",
"isbn": "0-395-19395-8",
"price": 22.99
}
],
"bicycle": {
"color": "red",
"price": 19.95
}
}
}
When fed through the following template:
{
# Pick an invidual field.
"bicycle_color": $.store.bicycle.color,
"book_info": {
# Slice an array, taking the first three elements.
"top_three": $.store.book[:3],
# Map a list of objects.
"price_list": range $.store.book[*] [
{
"title": $.title,
"price": $.price,
}
],
},
# Calculate the average of all price fields by calling a Go function.
"avg_price": Avg($..price),
}
...the following output is yielded:
{
"avg_price": 14.774000000000001,
"bicycle_color": "red",
"book_info": {
"price_list": [
{
"price": 8.95,
"title": "Sayings of the Century"
},
{
"price": 12.99,
"title": "Sword of Honour"
},
{
"price": 8.99,
"title": "Moby Dick"
},
{
"price": 22.99,
"title": "The Lord of the Rings"
}
],
"top_three": [
{
"author": "Nigel Rees",
"category": "reference",
"price": 8.95,
"title": "Sayings of the Century"
},
{
"author": "Evelyn Waugh",
"category": "fiction",
"price": 12.99,
"title": "Sword of Honour"
},
{
"author": "Herman Melville",
"category": "fiction",
"isbn": "0-553-21311-3",
"price": 8.99,
"title": "Moby Dick"
}
]
}
}
Performance
jsontemplate has first and foremost been designed with correctness and ease of use in mind. As such, optimum performance has not been the primary objective. Nevertheless, you can expect to see in the order of 10 MB/s on a single CPU core, around half of which is JSON parsing/encoding. We expect this to be more than adequate for most production use-cases.
Maturity
jsontemplate is provided as-is, and you should assume it has bugs. That said, at the time of writing, the library is being used for production workloads at Volumental.
Until a 1.0 release is made, incompatible changes may occur, though we will generally strive to maintain full backwards compatibility. Incompatible changes to the template definition format are unlikely to be introduced at this point.
Documentation
¶
Overview ¶
Example ¶
package main
import (
"os"
"strings"
"github.com/Volumental/jsontemplate"
)
// Store example from JSONPath.
const Input = `
{
"store": {
"book": [
{
"category": "reference",
"author": "Nigel Rees",
"title": "Sayings of the Century",
"price": 8.95
},
{
"category": "fiction",
"author": "Evelyn Waugh",
"title": "Sword of Honour",
"price": 12.99
},
{
"category": "fiction",
"author": "Herman Melville",
"title": "Moby Dick",
"isbn": "0-553-21311-3",
"price": 8.99
},
{
"category": "fiction",
"author": "J. R. R. Tolkien",
"title": "The Lord of the Rings",
"isbn": "0-395-19395-8",
"price": 22.99
}
],
"bicycle": {
"color": "red",
"price": 19.95
}
}
}
`
const Template = `
{
# Pick an invidual field.
"bicycle_color": $.store.bicycle.color,
"book_info": {
# Slice an array, taking the first three elements.
"top_three": $.store.book[:3],
# Map a list of objects.
"price_list": range $.store.book[*] [
{
"title": $.title,
"price": $.price,
}
],
},
# Calculate the average of all price fields.
"avg_price": Avg($..price),
}
`
// Helper function we'll use in the template.
func Avg(values []interface{}) float64 {
var sum = 0.0
var cnt = 0
for _, val := range values {
if num, ok := val.(float64); ok {
sum += num
cnt += 1
}
}
return sum / float64(cnt)
}
func main() {
var funcs = jsontemplate.FunctionMap{"Avg": Avg}
var template, _ = jsontemplate.ParseString(Template, funcs)
template.RenderJSON(os.Stdout, strings.NewReader(Input))
os.Stdout.Sync()
}
Output: {"avg_price":14.774000000000001,"bicycle_color":"red","book_info":{"price_list":[{"price":8.95,"title":"Sayings of the Century"},{"price":12.99,"title":"Sword of Honour"},{"price":8.99,"title":"Moby Dick"},{"price":22.99,"title":"The Lord of the Rings"}],"top_three":[{"author":"Nigel Rees","category":"reference","price":8.95,"title":"Sayings of the Century"},{"author":"Evelyn Waugh","category":"fiction","price":12.99,"title":"Sword of Honour"},{"author":"Herman Melville","category":"fiction","isbn":"0-553-21311-3","price":8.99,"title":"Moby Dick"}]}}
Index ¶
Examples ¶
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
This section is empty.
Types ¶
type FunctionMap ¶
type FunctionMap map[string]interface{}
FunctionMap is a map of named functions that may be called from within a template.
type MissingKeyPolicy ¶
type MissingKeyPolicy int
MissingKeyPolicy discates how the rendering should handle references to keys that are missing from the input data.
const ( // NullOnMissing makes query expressions referencing missing values evaluate // to null. NullOnMissing MissingKeyPolicy = iota // ErrorOnMissing causes the renderer to return an error if a query // expression references a missing value. ErrorOnMissing )
type Template ¶
type Template struct {
// MissingKeys defines the policy for how to handle keys referenced in
// queries that are absent in the input data. The default is to substitute
// them with null.
MissingKeys MissingKeyPolicy
// contains filtered or unexported fields
}
Template represents a transformation from one JSON-like structure to another.
func Parse ¶
func Parse(r io.Reader, funcs FunctionMap) (t *Template, err error)
Parse reads a template definition from a textual format.
The template definition format has a grammar similar to a regular JSON value, with some additions that allow interpolation and transformation. These are outlined below. As a special case, a regular JSON file is interpreted as a template taking no inputs.
Queries ¶
A template can pull data from the input data using JSONPath expressions (see https://goessner.net/articles/JsonPath/). Each query expression can evaluate to either a JSON value, which will then be inserted in its place, or a range of values, which will yield an array.
{
"single_x": $.foo.x,
"array_of_all_x_recursively": $..x
}
Generators ¶
Generators is a mechanism that allows repeating a sub-template within an array. It takes the format of the keyword `range` followed an array expression and a sub-template, as such:
range $.some_array_of_xy[*] [
{ "foo": $.x, "bar": $.y }
]
Inside the sub-template, the `$` refers to the root of each element in the input array. Thus, the example above maps the fields `x` and `y` to `foo` and `bar`, respectively, in the objects in the output array.
Functions ¶
Regular Go functions can be exposed to and called from within the template. This allows more complex transformations. For example, by adding a Coalesce function that returns the first non-nil argument, fallback defaults can be introduced as follows:
{ "foo": Coalesce($.some_input, "default value") }
Field annotations ¶
Members in objects can be prefixed with an annotation, starting with an `@` character.
{
@deprecated "field": "value"
}
Annotations are stripped from the final output. However, future versions of this library may allow some control over how annotated fields are rendered. For example, it could be used to elide deprecated fields.
Other differences ¶
To help users clarify and document intentions, the template format allows comments in the template definition. These are preceded by a `#` character. Anything from and including this character to the end of the line will be ignored when parsing the template.
Finally, unlike JSON, the template format tolerates trailing commas after the last element of objects and arrays.
func ParseString ¶
func ParseString(s string, funcs FunctionMap) (*Template, error)
ParseString works like Parse, but takes a string as input rather than a Reader.
func (*Template) Render ¶
Render generates a JSON-like structure based on the template definition, using the passed `data` as source data for query expressions.
func (*Template) RenderJSON ¶
RenderJSON generates JSON output based on the template definition, using JSON input as source data for query expressions.
Note that RenderJSON will only attempt to read a single JSON value from the input stream. If the stream contains multiple white-space delimited JSON values that you wish to transform, RenderJSON can be called repeatedly with the same arguments.
If EOF is encountered on the input stream before the start of a JSON value, RenderJSON will return io.EOF.
Example ¶
const input = `{ "snakeCase": 123 }`
template, _ := ParseString(`{ "CamelCase": $.snakeCase }`, nil)
template.RenderJSON(os.Stdout, strings.NewReader(input))
os.Stdout.Sync()
Output: {"CamelCase":123}
Source Files
Directories