jsontemplate

README

jsontemplate

This library provides a way to template a JSON document using paths extracted from another, typically larger, JSON document. This uses github.com/json-iterator/go to parse the document without marshalling it, only the fields needed are extracted. In addition to extracting fields from the source document, it will also attempt to parse and extract fields from nested JSON document, stored in a string field, if the path requires.

What can it be used for?

I use this library to reduce large JSON documents into smaller JSON documents that contain only the data needed for a specific purpose.

Examples of this are the following:

• S3 event notifications contain a large JSON payload with details of the event, this can be templated into a smaller JSON payload containing only the data needed by a Lambda function.
• API Gateway request payloads received by webhooks often contain large JSON payloads, these can be templated into smaller JSON payloads containing only the data needed by a downstream service.

Example

So we have an event come through, we extract a few fields and insert them into another JSON document.

func ExampleTemplate_ExecuteToString() {

	tpl, _ := jsontemplate.NewTemplate(`{"name": "${msg.name}","age": "${msg.age}","cyclist": "${msg.cyclist}"}`)

	res, _ := tpl.ExecuteToString([]byte(`{"msg":{"name":"markw","age":23,"cyclist":true}}`))
	fmt.Println(res)
	// Output:
	// {"name": "markw","age": 23,"cyclist": true}

}

tags

To customise how the JSON is rendered, you can use tags which are provided after the path and delimited by ;:

• escape, this will escape the JSON output.

func ExampleTemplate_ExecuteToString_encoded() {

	tpl, _ := jsontemplate.NewTemplate(`{"msg": "${msg;escape}"}`)

	res, _ := tpl.ExecuteToString([]byte(`{"msg":{"name":"markw","age":23,"cyclist":true}}`))
	fmt.Println(res)
	// Output:
	// {"msg": "{\"name\":\"markw\",\"age\":23,\"cyclist\":true}"}
}

Performance

Given I expect to run this for large numbers of events I have attempted to keep the code very simple and capitalize on the work done in the underlying libraries.

go test -bench=. -benchmem
goos: darwin
goarch: arm64
pkg: github.com/wolfeidau/jsontemplate
BenchmarkTemplate_ExecuteToString-10    	  676122	      2105 ns/op	    4767 B/op	     134 allocs/op
BenchmarkTemplate_Execute-10            	  559828	      2180 ns/op	    4876 B/op	     136 allocs/op
PASS
ok  	github.com/wolfeidau/jsontemplate	3.001s

License

This project is released under Apache 2.0 license and is copyright Mark Wolfe.

Documentation

Overview

Package jsontemplate provides a library for transforming JSON events using a minimal template.

Index

• Variables
• func JSONEncoder(wr io.Writer, v any) error
• func Valid(data []byte) (bool, error)
• type Document
  • func NewDocument(data []byte) *Document
  • func (d *Document) Read(path string) (result any, err error)
• type Encoder
• type Template
  • func NewTemplate(content string) (*Template, error)
  • func (t *Template) Execute(wr io.Writer, evt []byte) (int64, error)
  • func (t *Template) ExecuteToString(evt []byte) (string, error)

Examples

• Template.ExecuteToString
• Template.ExecuteToString (Encoded)

Variables

var (
	ErrInvalidTemplate = errors.New("invalid JSON format template")
)

Functions

func JSONEncoder

func JSONEncoder(wr io.Writer, v any) error

JSONEncoder encodes the given value v to the given writer wr using the JSON encoding format.

It returns any error encountered during encoding.

func Valid

func Valid(data []byte) (bool, error)

Types

type Document

type Document struct {
	// contains filtered or unexported fields
}

Document holds JSON content.

func NewDocument

func NewDocument(data []byte) *Document

NewDocument returns a new Document.

func (*Document) Read

func (d *Document) Read(path string) (result any, err error)

Read reads a JSON value at a given path using jsoniter.Get. If an error occurs it returns the error it parses the path then iterates over the path otherwise it returns the string value at the path.

type Encoder

type Encoder func(wr io.Writer, v any) error

Encoder encodes the given value v to the given writer wr.

It returns any error encountered during encoding.

type Template

type Template struct {
	// contains filtered or unexported fields
}

NewTemplate parses the given template content into a fasttemplate Template and returns it.

func NewTemplate

func NewTemplate(content string) (*Template, error)

NewTemplate parses the given template content into a fasttemplate Template and returns it.

Parameters:

content: The template content/string to parse.

Returns:

*Template: The parsed Template.
error: Any error encountered parsing the template.

Functionality: This function parses the given template string into a fasttemplate Template which can then be executed against JSON events. It validates the template string is valid JSON before parsing and returns an error if invalid.

ExecuteToString executes the template against the given JSON event and returns the result as a string.

func (*Template) Execute

func (t *Template) Execute(wr io.Writer, evt []byte) (int64, error)

Execute executes the template against the given JSON event and writes the result to the given writer. It returns the number of bytes written and any error.

func (*Template) ExecuteToString

func (t *Template) ExecuteToString(evt []byte) (string, error)

ExecuteToString executes the template against the given JSON event and returns the result as a string.

Example

package main

import (
	"fmt"

	"github.com/wolfeidau/jsontemplate"
)

func main() {

	tpl, _ := jsontemplate.NewTemplate(`{"name": "${msg.name}","age": "${msg.age}","cyclist": "${msg.cyclist}"}`)

	res, _ := tpl.ExecuteToString([]byte(`{"msg":{"name":"markw","age":23,"cyclist":true}}`))
	fmt.Println(res)
}

Output:

{"name": "markw","age": 23,"cyclist": true}

Example (Encoded)

package main

import (
	"fmt"

	"github.com/wolfeidau/jsontemplate"
)

func main() {

	tpl, _ := jsontemplate.NewTemplate(`{"msg": "${msg;escape}"}`)

	res, _ := tpl.ExecuteToString([]byte(`{"msg":{"name":"markw","age":23,"cyclist":true}}`))
	fmt.Println(res)
}

Output:

{"msg": "{\"name\":\"markw\",\"age\":23,\"cyclist\":true}"}