Documentation
¶
Overview ¶
Package mapperkit is a small, fast template engine that maps raw events (TSV fields, JSON, XML, HTTP headers/cookies/query parameters) into JSON documents or plain strings.
Templates are compiled once and executed many times; execution is safe for concurrent use.
Placeholders ¶
{{data[N]}} N-th positional field (Context.RawFields), "" if out of range
{{data.path}} JSON path (gjson syntax, a[0] is accepted as a.0) or, without JSON,
an XML path: "a.b", "item[0].x", "param[key=ID].value", "node.@attr";
namespace prefixes are ignored and the root element may be omitted
{{header["name"]}} Context.Headers
{{cookie["name"]}} Context.Cookies
{{query["name"]}} Context.QueryParams
{{flow_id}} Context.FlowId
{{timestamp}} current Unix time in seconds
{{uuid}} a random UUID v4
Methods ¶
Methods are chained left to right (at most 5 per placeholder):
.format('2006-01-02') parse a date and reformat it with a Go layout
.toUnix() parse a date and return Unix seconds (rendered as a JSON number)
.unixFormat('layout') Unix seconds -> formatted UTC date (default "2006-01-02 15:04:05")
.toInt() integer (floats are truncated, garbage is 0; JSON number)
.hexToInt() hex string (optional 0x) -> decimal (JSON number)
.toBool() "true"/"false"/number >= 1 -> boolean (JSON boolean)
.toLower() .toUpper() .trim()
.replace('old', 'new') replace all occurrences
.substr(from[, len]) rune-based substring; negative from counts from the end
.host() strip scheme, path and port from a URL
Dates are recognized in RFC 3339, "2006-01-02T15:04:05", "2006-01-02 15:04:05", "2006-01-02", "02.01.2006[ 15:04:05]", "01/02/2006[ 15:04:05]", RFC 1123, RFC 822 and as Unix seconds.
Conditionals ¶
{{#if data.status == "ok"}}...{{#else if data.code >= 500}}...{{#else}}...{{/if}}
Operators: ==, !=, >, <, >=, <= (numeric when both sides are numbers, lexicographic otherwise) and in, e.g. data.kind in ["a", "b"].
JSON output ¶
Compile treats the template as text, not as JSON, but understands JSON strings:
- a placeholder inside a JSON string literal is escaped, so values with quotes, backslashes or control characters always produce valid JSON;
- a placeholder that is the whole content of a JSON string and ends with toInt, toUnix, hexToInt or toBool loses its quotes and renders as a bare number or boolean;
- a placeholder outside quotes is inserted as is (e.g. a nested JSON object).
CompileString never escapes: use it for keys and other non-JSON strings.
Validation helpers ¶
Canonicalize turns short-hand references ("4", "[4]", "a.b") into placeholders, and ValidateRef checks a value against a RefMode and a SourceFormat before it is stored.
Index ¶
- Variables
- func Canonicalize(raw string) (canonical string, changed bool)
- func NavigateXMLTree(tree any, path string) string
- func ParseXMLTree(data []byte) (any, error)
- func ResolveXMLPosition(tree any, rawXML []byte, position string) string
- func ValidateRef(mode RefMode, format SourceFormat, s string) error
- func Version() string
- type CompiledString
- type CompiledTemplate
- type Context
- type RefMode
- type SourceFormat
Examples ¶
Constants ¶
This section is empty.
Variables ¶
var ( // ErrCompile wraps every error returned by Compile. ErrCompile = errors.New("mapperkit.compile") // ErrUnclosedPlaceholder means a "{{" has no matching "}}". ErrUnclosedPlaceholder = errors.New("mapperkit.unclosed_placeholder") // ErrInvalidDataIndex means the N in data[N] is not an integer (or is negative in ValidateRef). ErrInvalidDataIndex = errors.New("mapperkit.invalid_data_index") // ErrUnknownPlaceholder means the placeholder base is not one of data[N], data.path, // header["k"], cookie["k"], query["k"], flow_id, timestamp or uuid. ErrUnknownPlaceholder = errors.New("mapperkit.unknown_placeholder") // ErrUnknownMethod means a chained method name is not supported. ErrUnknownMethod = errors.New("mapperkit.unknown_method") // ErrResolve means a placeholder could not be resolved at execution time. ErrResolve = errors.New("mapperkit.resolve") // ErrParseDate means a value could not be parsed as a date. ErrParseDate = errors.New("mapperkit.parse_date") // ErrInvalidIfCondition means a {{#if}} condition has no operator or a malformed operand. ErrInvalidIfCondition = errors.New("mapperkit.invalid_if_condition") // ErrUnclosedIfBlock means a {{#if}} block has no matching {{/if}}. ErrUnclosedIfBlock = errors.New("mapperkit.unclosed_if_block") // ErrIfBlockSyntax means a structural error in an {{#if}} block (stray {{#else}} or {{/if}}, etc.). ErrIfBlockSyntax = errors.New("mapperkit.if_block_syntax") // ErrTooManyMethods means a placeholder chains more methods than allowed. ErrTooManyMethods = errors.New("mapperkit.too_many_methods") // ErrInvalidMethodArg means a method argument is malformed (e.g. a non-integer substr offset). ErrInvalidMethodArg = errors.New("mapperkit.invalid_method_arg") // ErrNotSingleRef is returned by ValidateRef in ModeSingleRef when s is not exactly one placeholder. ErrNotSingleRef = errors.New("mapperkit.not_single_ref") // ErrIfBlockNotAllowed is returned by ValidateRef in ModeStringTemplate when s contains {{#if}}. ErrIfBlockNotAllowed = errors.New("mapperkit.if_block_not_allowed") // ErrIndexBaseOnPathFormat is returned by ValidateRef when data[N] is used for a JSON or XML source. ErrIndexBaseOnPathFormat = errors.New("mapperkit.index_base_on_path_format") // ErrPathBaseOnIndexFormat is returned by ValidateRef when data.path is used for a TSV source. ErrPathBaseOnIndexFormat = errors.New("mapperkit.path_base_on_index_format") // ErrDataBaseOnTextFormat is returned by ValidateRef when any data reference is used for a text source. ErrDataBaseOnTextFormat = errors.New("mapperkit.data_base_on_text_format") )
Sentinel errors. Errors returned by Compile, CompileString, Execute and ValidateRef wrap one of these, so callers can match them with errors.Is.
Functions ¶
func Canonicalize ¶
Canonicalize rewrites a short-hand field reference into its canonical placeholder form without changing its meaning. changed is true if the value was rewritten. Rules, in order:
"" or "none" -> unchanged
contains "{{" -> unchanged (already a template)
"4" -> "{{data[4]}}"
"[4]" -> "{{data[4]}}"
"data[4]" -> "{{data[4]}}"
anything else (path) -> "{{data.<path>}}"
The last rule adds the "data." prefix mechanically, without heuristics: "data.phone" becomes "{{data.data.phone}}" on purpose. Whether the base fits the source is checked by ValidateRef, not by Canonicalize.
Example ¶
package main
import (
"fmt"
"github.com/boostgo/mapperkit"
)
func main() {
for _, raw := range []string{"4", "[4]", "user.name", "{{data[1]}}"} {
c, changed := mapperkit.Canonicalize(raw)
fmt.Println(c, changed)
}
}
Output: {{data[4]}} true {{data[4]}} true {{data.user.name}} true {{data[1]}} false
func NavigateXMLTree ¶
NavigateXMLTree resolves a dot-path against a tree returned by ParseXMLTree. Returns an empty string if the path is not found, tree is nil, or wrong type.
func ParseXMLTree ¶
ParseXMLTree parses XML bytes into an opaque tree handle. Pass the result to NavigateXMLTree to extract field values.
func ResolveXMLPosition ¶
ResolveXMLPosition resolves a position expression against a pre-parsed XML tree. Position may be a raw path ("phone") or a template expression ("{{data.phone}}"). rawXML is only used when position contains template placeholders.
func ValidateRef ¶
func ValidateRef(mode RefMode, format SourceFormat, s string) error
ValidateRef checks that s is allowed in a field with the given mode for a source of the given format. It returns nil for an empty string and for the keyword "none": deciding whether a field must be configured is up to the caller.
Checks run in order and the first problem is returned:
- s compiles (balanced braces, known bases and methods, method chain length, substr args);
- ModeSingleRef: exactly one placeholder and no surrounding text (ErrNotSingleRef);
- ModeStringTemplate: no {{#if}} blocks (ErrIfBlockNotAllowed);
- the placeholder base matches format (ErrIndexBaseOnPathFormat, ErrPathBaseOnIndexFormat, ErrDataBaseOnTextFormat); skipped for FormatUnknown;
- no negative data[N] index (ErrInvalidDataIndex).
Example ¶
package main
import (
"errors"
"fmt"
"github.com/boostgo/mapperkit"
)
func main() {
err := mapperkit.ValidateRef(mapperkit.ModeSingleRef, mapperkit.FormatTSV, "{{data.user}}")
fmt.Println(errors.Is(err, mapperkit.ErrPathBaseOnIndexFormat))
err = mapperkit.ValidateRef(mapperkit.ModeStringTemplate, mapperkit.FormatJSON, "id-{{data.id}}")
fmt.Println(err)
}
Output: true <nil>
Types ¶
type CompiledString ¶
type CompiledString struct {
// contains filtered or unexported fields
}
CompiledString is a compiled single string template (for key templates etc.).
func CompileString ¶
func CompileString(template string) (*CompiledString, error)
CompileString parses a single string template with {{...}} placeholders. Returns nil if template is empty.
Example ¶
package main
import (
"fmt"
"github.com/boostgo/mapperkit"
)
func main() {
key, _ := mapperkit.CompileString(`{{flow_id}}:{{header["X-Tenant"]}}:{{data[0].substr(0, 3)}}`)
s, _ := key.Execute(mapperkit.Context{
FlowId: "orders",
Headers: map[string]string{"X-Tenant": "acme"},
RawFields: []string{"ABCDEF"},
})
fmt.Println(s)
}
Output: orders:acme:ABC
type CompiledTemplate ¶
type CompiledTemplate struct {
// contains filtered or unexported fields
}
CompiledTemplate is a whole output template (usually JSON) parsed as a text template. Placeholders {{...}} are resolved and the result should be valid JSON. This supports unquoted placeholders for int/bool values.
func Compile ¶
func Compile(templateStr string) (*CompiledTemplate, error)
Compile parses an output template as a text template with {{...}} placeholders. The template is NOT parsed as JSON — placeholders can appear unquoted for int/bool values. Returns nil if template is empty (passthrough mode).
Example (Conditional) ¶
package main
import (
"fmt"
"github.com/boostgo/mapperkit"
)
func main() {
tmpl, _ := mapperkit.Compile(
`{"level":{{#if data.code >= 500}}"error"{{#else if data.code >= 400}}"warn"{{#else}}"info"{{/if}}}`)
for _, code := range []string{"503", "404", "200"} {
out, _ := tmpl.Execute(mapperkit.Context{RawJSON: []byte(`{"code":` + code + `}`)})
fmt.Println(string(out))
}
}
Output: {"level":"error"} {"level":"warn"} {"level":"info"}
Example (Escaping) ¶
package main
import (
"fmt"
"github.com/boostgo/mapperkit"
)
func main() {
tmpl, _ := mapperkit.Compile(`{"text":"{{data[0]}}"}`)
out, _ := tmpl.Execute(mapperkit.Context{RawFields: []string{"line 1\nsaid \"hi\""}})
fmt.Println(string(out))
}
Output: {"text":"line 1\nsaid \"hi\""}
Example (Json) ¶
package main
import (
"fmt"
"github.com/boostgo/mapperkit"
)
func main() {
tmpl, err := mapperkit.Compile(`{"user":"{{data.user.name.toUpper()}}","age":"{{data.user.age.toInt()}}"}`)
if err != nil {
panic(err)
}
out, err := tmpl.Execute(mapperkit.Context{
RawJSON: []byte(`{"user":{"name":"alice","age":"42"}}`),
})
if err != nil {
panic(err)
}
fmt.Println(string(out))
}
Output: {"user":"ALICE","age":42}
Example (Tsv) ¶
package main
import (
"fmt"
"github.com/boostgo/mapperkit"
)
func main() {
tmpl, _ := mapperkit.Compile(`{"id":"{{data[0]}}","host":"{{data[2].host()}}","ok":"{{data[1].toBool()}}"}`)
out, _ := tmpl.Execute(mapperkit.Context{
RawFields: []string{"evt-1", "1", "https://example.com:8443/path"},
})
fmt.Println(string(out))
}
Output: {"id":"evt-1","host":"example.com","ok":true}
Example (Xml) ¶
package main
import (
"fmt"
"github.com/boostgo/mapperkit"
)
func main() {
tmpl, _ := mapperkit.Compile(`{"phone":"{{data.phone}}","service":"{{data.params.param[key=SERVICE].value}}"}`)
xml := []byte(`<ns:event xmlns:ns="urn:x">
<ns:phone>15550001234</ns:phone>
<ns:params>
<ns:param><ns:key>SERVICE</ns:key><ns:value>42</ns:value></ns:param>
</ns:params>
</ns:event>`)
out, _ := tmpl.Execute(mapperkit.Context{RawXML: xml})
fmt.Println(string(out))
}
Output: {"phone":"15550001234","service":"42"}
type Context ¶
type Context struct {
// FlowId is the value of the {{flow_id}} placeholder (any caller-defined identifier
// of the pipeline, stream or tenant that produced the event).
FlowId string //nolint:revive // kept for API compatibility
// RawFields are the positional fields of a TSV/CSV-like event, read by {{data[N]}}.
RawFields []string
// RawJSON is a JSON event, read by {{data.path}} (gjson syntax; a[0] is also accepted).
RawJSON []byte
// RawXML is an XML event, read by {{data.path}} when RawJSON is empty.
RawXML []byte
// Headers are read by {{header["name"]}}.
Headers map[string]string
// Cookies are read by {{cookie["name"]}}.
Cookies map[string]string
// QueryParams are read by {{query["name"]}}.
QueryParams map[string]string
// contains filtered or unexported fields
}
Context is the input a template is executed against. Set only the fields your templates use; a placeholder whose source is empty resolves to "".
func (Context) WithParsedXML ¶
WithParsedXML parses RawXML once and returns a Context that can be reused across many Execute calls. Context is passed by value, so without this every Execute parses the XML again; a caller that evaluates several templates against the same event should call WithParsedXML once and pass the result to all of them.
Example ¶
package main
import (
"fmt"
"github.com/boostgo/mapperkit"
)
func main() {
ctx := mapperkit.Context{RawXML: []byte(`<e><a>1</a><b>2</b></e>`)}.WithParsedXML()
a, _ := mapperkit.CompileString("{{data.a}}")
b, _ := mapperkit.CompileString("{{data.b}}")
va, _ := a.Execute(ctx) // the XML is parsed only once
vb, _ := b.Execute(ctx)
fmt.Println(va, vb)
}
Output: 1 2
type RefMode ¶
type RefMode int
RefMode describes how much freedom a configuration field allows in its value.
const ( // ModeSingleRef allows exactly one placeholder and nothing around it, e.g. // "{{data[4]}}" or "{{data.content.phone.trim()}}". A plain literal without "{{" is // also accepted. Typical use: a field reference or the left side of a condition. ModeSingleRef RefMode = iota // ModeStringTemplate allows literals plus any number of placeholders, but no {{#if}} // blocks, e.g. "{{data[5]}}-{{data[6]}}" or "evt:{{flow_id}}:{{data.id}}". Typical use: // keys, deduplication keys, composite identifiers. ModeStringTemplate // ModeFullTemplate allows everything, including {{#if}} blocks. Typical use: a full // JSON output template. ModeFullTemplate )
type SourceFormat ¶
type SourceFormat string
SourceFormat is the message format of a data source; it decides which placeholder base (data[N] or data.path) is valid. FormatUnknown disables the base check (useful when the caller does not know the format yet).
const ( // FormatUnknown disables the source format check in ValidateRef. FormatUnknown SourceFormat = "" // FormatTSV is a tab-separated (positional) source: use {{data[N]}}. FormatTSV SourceFormat = "tsv" // FormatJSON is a JSON source: use {{data.path}}. FormatJSON SourceFormat = "json" // FormatXML is an XML source: use {{data.path}}. FormatXML SourceFormat = "xml" // FormatText is a plain-text source: data references are not supported. FormatText SourceFormat = "text" )
Directories
¶
| Path | Synopsis |
|---|---|
|
examples
|
|
|
json
command
Command json maps an incoming JSON event into a normalized JSON document.
|
Command json maps an incoming JSON event into a normalized JSON document. |
|
tsv
command
Command tsv maps tab-separated log lines into JSON and builds a key for each line.
|
Command tsv maps tab-separated log lines into JSON and builds a key for each line. |
|
xml
command
Command xml maps an XML notification (with namespaces, attributes and repeated elements) into JSON.
|
Command xml maps an XML notification (with namespaces, attributes and repeated elements) into JSON. |