gottp

package module
v0.0.0-...-44883d3 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Feb 14, 2026 License: MIT Imports: 17 Imported by: 0

README

Go TTP - Template Text Parser for Go

A Go implementation of the Template Text Parser (TTP) library, providing semi-structured text parsing using templates with enhanced features.

Features

  • Full TTP Compatibility: Compatible with Python TTP templates and syntax
  • Stateless Compiled Templates: Compile templates once, use many times without state resets ✅
  • Multi-Language Macros: Support for Starlark (default), JavaScript, and Python (optional, requires build tag)
  • Dual API Design: Python-compatible API for easy porting, plus Go-idiomatic API ✅
  • Thread-Safe: Compiled templates are immutable and safe for concurrent use ✅
  • High Performance: Leverages Go's compiled nature for better performance
  • Source Maps: Optional source map generation to track which input lines/characters matched which template patterns (useful for editor visualization and debugging)

Status

Core Features Complete ✅

  • Template parsing (XML-based)
  • Pattern engine (regex generation)
  • Compiled template system
  • Stateless parsing execution
  • Multi-line pattern matching (start, end, line indicators)
  • Template extension ( tag)
  • All match functions (52/52) ✅
  • All group functions (21/21) ✅
  • All input loaders (8/8: text, yaml, json, csv, file, directory, url, database) ✅
  • All output formatters (10/10: raw, json, yaml, csv, table, pprint, tabulate, excel, jinja2, n2g) ✅
  • Macro execution engines (Starlark, JavaScript, Python) ✅
  • Python-compatible API ✅
  • Comprehensive test suite (102 comparison tests passing) ✅
  • Complete documentation (Sphinx + Markdown) ✅

Not Applicable (Go-specific)

  • Multi-processing support (Go uses goroutines - not needed)
  • Lazy loader system (not needed in Go's compiled environment)

Future Enhancements 📋

  • Enhanced error messages with detailed context
  • Additional template validation options

Quick Start

package main

import (
    "fmt"
    "github.com/roc-ops/gottp"
)

func main() {
    template := `
<group name="interfaces">
interface {{ interface }}
 ip address {{ ip }}/{{ mask }}
 description {{ description }}
</group>
`

    data := `
interface Loopback0
 ip address 192.168.0.113/24
 description Router-id-loopback
!
interface Vlan778
 ip address 2002::fd37/124
 description CPE_Acces_Vlan
!
`

    // Compile template once
    compiled, err := gottp.CompileTemplate(template)
    if err != nil {
        panic(err)
    }

    // Use many times with different inputs - no reset needed
    result, err := compiled.Parse(gottp.Inputs{
        "Default_Input": data,
    }, nil, nil)
    if err != nil {
        panic(err)
    }

    fmt.Printf("%+v\n", result)
}

Installation

go get github.com/roc-ops/gottp

Documentation

Online Editor

Try GoTTP in your browser! The editor is live at:

🌐 https://roc-ops.github.io/gottp/index.html

The online editor features:

  • Real-time template processing
  • Monaco Editor with syntax highlighting and auto-completion
  • Multiple output formats (JSON, YAML, Table, CSV)
  • Global variables and lookup tables
  • Source map visualization for debugging
  • Export/import configurations
  • Built-in examples

License

MIT License - see LICENSE file for details.

Documentation

Overview

Package gottp provides a Go implementation of the Template Text Parser (TTP) library.

GoTTP is a semi-structured text parsing library that uses templates to extract structured data from text. It is compatible with Python TTP templates and provides enhanced features including stateless compiled templates and multi-language macro support.

Quick Start 

package main

import (
	"fmt"
	"github.com/roc-ops/gottp"
)

func main() {
	// Compile a template
	template := `
<group name="interfaces">
interface {{ interface }}
 ip address {{ ip }}/{{ mask }}
</group>
`

	compiled, err := gottp.CompileTemplate(template)
	if err != nil {
		panic(err)
	}

	// Parse data (can be called repeatedly without reset)
	data := `
interface Loopback0
 ip address 192.168.0.1/24
!`

	result, err := compiled.Parse(
		gottp.Inputs{"Default_Input": data},
		nil, // vars
		nil, // options
	)
	if err != nil {
		panic(err)
	}

	fmt.Printf("%+v\n", result)

	// With source maps enabled:
	parseResult, err := compiled.ParseWithValidation(
		gottp.Inputs{"Default_Input": data},
		nil, // vars
		&gottp.ParseOptions{EnableSourceMap: true},
	)
	if err != nil {
		panic(err)
	}

	// Access parsed data
	fmt.Printf("Data: %+v\n", parseResult.Data)

	// Access source map to see which input lines matched
	if parseResult.SourceMap != nil {
		inputMap := parseResult.SourceMap.Inputs["Default_Input"]
		for _, line := range inputMap.Lines {
			if line.Matched {
				fmt.Printf("Line %d matched\n", line.LineNumber+1)
			}
		}
	}
}

Key Features

  • Stateless Compiled Templates: Compile once, use many times without state resets
  • Full TTP Compatibility: Works with existing Python TTP templates
  • Multi-Language Macros: Support for Starlark, JavaScript, and Python macros
  • Thread-Safe: Compiled templates are immutable and safe for concurrent use
  • High Performance: Leverages Go's compiled nature for better performance
  • Source Maps: Track which parts of input text matched which template patterns (optional)

Stateless Design

Unlike Python TTP, GoTTP's compiled templates are stateless. The Parse() method takes all necessary parameters and has no internal state, allowing for:

  • Repeated calls without reset
  • Concurrent execution from multiple goroutines
  • Better performance and memory efficiency

Template Syntax

GoTTP uses the same template syntax as Python TTP. Templates are XML-like structures with special tags:

  • <template>: Root template container
  • <group>: Pattern matching group
  • <input>: Input data source configuration
  • <output>: Output formatting configuration
  • <vars>: Template variables
  • <macro>: Macro function definitions
  • <extend>: Template extension

Pattern Matching

Patterns use double curly braces to define variables: 

interface {{ interface }}
 ip address {{ ip }}/{{ mask }}

Variables can have functions applied: 

{{ ip | IP }}
{{ mac | MAC }}
{{ value | upper | split(',') }}

Macros

Macros allow custom processing logic. Supported languages:

  • Starlark (default): Python-like, safe for embedding, no dependencies
  • JavaScript: Using goja runtime, no external dependencies
  • Python: Using CGO (optional, requires -tags python build flag and Python runtime)

Example: 

<macro name="process_data" language="starlark">
def process_data(data):
    return data.upper()
</macro>

Template Extension

Templates can extend other templates using the <extend> tag: 

<extend template="base_template.txt" groups="common,advanced" />

Code Generation

Use the gottp-gen tool to embed templates at compile time: 

//go:generate gottp-gen -template=template.txt -var=MyTemplate

Then run: go generate

Examples

See the examples/ directory for complete working examples.

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

func LoadLookupFromCSV 

func LoadLookupFromCSV(name string, data []byte, keyColumn string) (map[string]map[string]interface{}, error)

LoadLookupFromCSV parses CSV data into a named lookup table suitable for ParseOptions.Lookups. The first row is treated as headers. The keyColumn parameter specifies which column to use as the lookup key. If keyColumn is empty, the first column is used. Each row becomes an entry in the lookup table keyed by the keyColumn value, with all columns (including the key column) as fields.

All values are strings since CSV does not carry type information.

Returns a single-entry map (name -> table) that can be directly assigned or merged into ParseOptions.Lookups.

func LoadLookupFromJSON 

func LoadLookupFromJSON(name string, data []byte) (map[string]map[string]interface{}, error)

LoadLookupFromJSON parses JSON data into a named lookup table suitable for ParseOptions.Lookups. The JSON should be an object where keys map to objects (map of maps). Example JSON: {"65100": {"as_name": "Subs", "prefix_num": "734"}}

Returns a single-entry map (name -> table) that can be directly assigned or merged into ParseOptions.Lookups.

func LoadLookupFromYAML 

func LoadLookupFromYAML(name string, data []byte) (map[string]map[string]interface{}, error)

LoadLookupFromYAML parses YAML data into a named lookup table suitable for ParseOptions.Lookups. Example YAML: 

"65100":
  as_name: Subs
  prefix_num: "734"

Returns a single-entry map (name -> table) that can be directly assigned or merged into ParseOptions.Lookups.

func LoadLookupsFromJSON 

func LoadLookupsFromJSON(data []byte) (map[string]map[string]interface{}, error)

LoadLookupsFromJSON parses JSON data containing multiple named lookup tables. The JSON should be a nested object: {"table_name": {"key": {"field": "value"}}}. This is useful when a single JSON file contains all lookup tables.

Returns a map that can be directly assigned to ParseOptions.Lookups.

func MergeLookups 

func MergeLookups(lookups ...map[string]map[string]interface{}) map[string]map[string]interface{}

MergeLookups merges multiple lookup maps into a single map suitable for ParseOptions.Lookups. Later entries override earlier entries with the same name.

func SaveCompiledTemplate 

func SaveCompiledTemplate(ct *CompiledTemplate, format string) ([]byte, error)

SaveCompiledTemplate saves a compiled template to bytes in the specified format Supported formats: "gob", "json", "yaml"

func SaveCompiledTemplateToBytes 

func SaveCompiledTemplateToBytes(ct *CompiledTemplate, format SerializationFormat) ([]byte, error)

SaveCompiledTemplateToBytes saves a compiled template to bytes

func ValidateTemplate 

func ValidateTemplate(templateStr string) error

ValidateTemplate validates a template string before compilation. Returns an error if the template has validation issues.

Types

type CompileOptions 

type CompileOptions struct {
	// Functions provides custom functions baked into the compiled template,
	// organized by scope. These functions are available for all Parse() calls
	// and can be overridden per-parse via ParseOptions.Functions.
	Functions *Functions
}

CompileOptions contains options for template compilation. Functions registered here are baked into the compiled template and available for all subsequent Parse() calls without needing to pass them in ParseOptions.

Precedence: built-in functions < CompileOptions functions < ParseOptions functions

type CompiledTemplate 

type CompiledTemplate struct {
	// contains filtered or unexported fields
}

CompiledTemplate is a stateless, immutable compiled template.

Once compiled, a template can be used to parse multiple inputs without any state resets. The Parse() method is safe for concurrent use from multiple goroutines, though each goroutine should create its own Runtime instance (which happens automatically in Parse()).

Compiled templates can be serialized and saved for later use, or embedded in Go code using the gottp-gen code generation tool.

func CompileTemplate 

func CompileTemplate(templateText string) (*CompiledTemplate, error)

CompileTemplate compiles a TTP template text into a CompiledTemplate.

The compiled template is immutable and stateless, allowing it to be used repeatedly without reset. It can be safely shared across goroutines.

Example: 

template := `<group name="test">{{ value }}</group>`
compiled, err := gottp.CompileTemplate(template)
if err != nil {
	// handle error
}

func CompileTemplateWithOptions 

func CompileTemplateWithOptions(templateText string, options *CompileOptions) (*CompiledTemplate, error)

CompileTemplateWithOptions compiles a TTP template text with compile-time options.

Functions provided in CompileOptions are baked into the compiled template and available for all subsequent Parse() calls. They can be overridden per-parse via ParseOptions.Functions.

Precedence for function resolution:

  • built-in functions (lowest)
  • CompileOptions.Functions (middle)
  • ParseOptions.Functions (highest, per-parse)

Example: 

compiled, err := gottp.CompileTemplateWithOptions(template, &gottp.CompileOptions{
	Functions: &gottp.Functions{
		Match: map[string]gottp.MatchFunc{
			"custom_transform": myTransformFunc,
		},
	},
})

func LoadCompiledTemplate 

func LoadCompiledTemplate(data []byte, format string) (*CompiledTemplate, error)

LoadCompiledTemplate loads a compiled template from bytes in the specified format

func LoadCompiledTemplateFromBytes 

func LoadCompiledTemplateFromBytes(data []byte, format SerializationFormat) (*CompiledTemplate, error)

LoadCompiledTemplateFromBytes loads a compiled template from bytes

func (*CompiledTemplate) GetWarnings 

func (ct *CompiledTemplate) GetWarnings() []string

GetWarnings returns compilation warnings (non-fatal issues like Python-specific syntax in Starlark macros)

func (*CompiledTemplate) NewRuntime deprecated 

func (ct *CompiledTemplate) NewRuntime() *Runtime

NewRuntime creates a reusable Runtime instance for a compiled template. This allows you to register Go macros and reuse the runtime for multiple parses.

Deprecated: For registering custom macros, use CompileTemplateWithOptions with CompileOptions.Functions.Macro instead, which provides a cleaner API and proper precedence handling. The RegisterGoMacro API continues to work for backward compatibility.

func (*CompiledTemplate) Parse 

func (ct *CompiledTemplate) Parse(inputs Inputs, vars Vars, options *ParseOptions) (interface{}, error)

Parse executes the compiled template with given inputs and variables.

This method is stateless and can be called repeatedly without reset. It does not modify the CompiledTemplate instance, making it safe for concurrent use. However, for best performance with concurrent parsing, create a new CompiledTemplate instance for each goroutine or use the parallel parsing utilities.

Parameters:

  • inputs: Map of input names to input data strings
  • vars: Map of variable names to variable values (can be nil)
  • options: Parse options (can be nil for defaults)

Returns:

  • Parsed results as interface{} (typically map[string]interface{} or []interface{})
  • error if parsing fails

Example: 

result, err := compiled.Parse(
	gottp.Inputs{"Default_Input": "interface Loopback0\n ip address 1.1.1.1/24"},
	gottp.Vars{"site": "datacenter1"},
	nil,
)

func (*CompiledTemplate) ParseWithValidation 

func (ct *CompiledTemplate) ParseWithValidation(inputs Inputs, vars Vars, options *ParseOptions) (*ParseResult, error)

ParseWithValidation parses and returns both data and validation results

type Functions 

type Functions struct {
	Match  map[string]MatchFunc
	Group  map[string]GroupFunc
	Output map[string]OutputFunc
	Input  map[string]InputFunc
	Macro  map[string]MacroFunc
}

Functions contains custom functions organized by scope.

type GroupFunc 

type GroupFunc func(data map[string]interface{}, args []string, kwargs map[string]interface{}) (map[string]interface{}, bool, error)

GroupFunc is the signature for custom group functions. Matches internal/functions/group.Function.

type InputFunc 

type InputFunc func(data string, args []string, kwargs map[string]interface{}) (string, bool, error)

InputFunc is the signature for custom input functions. Matches internal/functions/input.Function.

type InputSourceMap 

type InputSourceMap struct {
	Lines []*LineMapping // one entry per input line
}

InputSourceMap contains source mapping for a single input

type Inputs 

type Inputs map[string]string

Inputs represents input data for parsing

type LineMapping 

type LineMapping struct {
	LineNumber int             // 0-indexed line number
	Matched    bool            // whether this line matched
	Matches    []*MatchMapping // matches on this line
}

LineMapping represents the mapping for a single line of input

type MacroFunc 

type MacroFunc func(data map[string]interface{}, args []string, kwargs map[string]interface{}) (map[string]interface{}, bool, error)

MacroFunc is the signature for custom Go macro functions. Matches internal/macro.GoMacroFunc.

type MacroRegistry 

type MacroRegistry struct {
	// contains filtered or unexported fields
}

MacroRegistry provides access to register native Go macros

func (*MacroRegistry) RegisterGoMacro deprecated 

func (mr *MacroRegistry) RegisterGoMacro(name string, fn func(data map[string]interface{}, args []string, kwargs map[string]interface{}) (map[string]interface{}, bool, error))

RegisterGoMacro registers a native Go macro function. This allows high-performance macro execution without conversion overhead. The function signature matches group functions: 

func(data map[string]interface{}, args []string, kwargs map[string]interface{}) (map[string]interface{}, bool, error)

Deprecated: Use CompileTemplateWithOptions with CompileOptions.Functions.Macro instead, which provides a cleaner API and proper precedence handling. RegisterGoMacro continues to work for backward compatibility.

type MatchFunc 

type MatchFunc func(value interface{}, args []string, kwargs map[string]interface{}) (interface{}, error)

MatchFunc is the signature for custom match functions. Matches internal/functions/match.Function.

type MatchMapping 

type MatchMapping struct {
	StartCol     int                  // start column (0-indexed)
	EndCol       int                  // end column (exclusive)
	GroupName    string               // group name that matched
	PatternIndex int                  // pattern index within group
	Variables    map[string]*VarRange // variable name -> character range
	ResultPath   string               // path in result structure (e.g., "interfaces[0]")
}

MatchMapping represents a single match on a line

type OutputFunc 

type OutputFunc func(data interface{}, args []string, kwargs map[string]interface{}) (interface{}, error)

OutputFunc is the signature for custom output functions. Matches internal/functions/output.Function.

type ParseOptions 

type ParseOptions struct {
	YANGModules     *YANGModules // YANG modules for validation
	EnableSourceMap bool         // Enable source map collection (zero overhead when false)

	// Lookups provides runtime lookup tables merged with compiled lookups at parse time.
	// Runtime lookups override compiled lookups with the same name.
	Lookups map[string]map[string]interface{}

	// Vars provides runtime variables merged with compiled template vars and Parse() vars.
	// Precedence: compiled vars < Parse() vars < ParseOptions.Vars.
	Vars map[string]interface{}

	// Functions provides custom functions injected at parse time, organized by scope.
	// Custom functions override built-in functions with the same name.
	Functions *Functions
}

ParseOptions represents options for parsing

type ParseResult 

type ParseResult struct {
	Data              interface{}                       // Parsed data
	ValidationResults map[string]*yang.ValidationResult // YANG validation results by group name
	SourceMap         *SourceMap                        // Source map tracking input positions to results (optional)
}

ParseResult contains the parsed results and validation results

type PythonParser 

type PythonParser struct {
	// contains filtered or unexported fields
}

PythonParser provides Python-compatible API

func NewParser 

func NewParser() *PythonParser

NewParser creates a new Python-compatible parser (stateful API)

func (*PythonParser) AddInput 

func (p *PythonParser) AddInput(data, inputName string)

AddInput adds input data

func (*PythonParser) AddTemplate 

func (p *PythonParser) AddTemplate(template string) error

AddTemplate adds a template

func (*PythonParser) AddVars 

func (p *PythonParser) AddVars(vars Vars)

AddVars adds variables

func (*PythonParser) ClearInput 

func (p *PythonParser) ClearInput()

ClearInput clears inputs

func (*PythonParser) ClearResult 

func (p *PythonParser) ClearResult()

ClearResult clears results

func (*PythonParser) Parse 

func (p *PythonParser) Parse() error

Parse parses all inputs

func (*PythonParser) Result 

func (p *PythonParser) Result() []interface{}

Result returns results

type Runtime 

type Runtime struct {
	// contains filtered or unexported fields
}

Runtime provides access to the underlying runtime for advanced operations such as registering native Go macros before parsing

func (*Runtime) GetMacroRegistry 

func (r *Runtime) GetMacroRegistry() *MacroRegistry

GetMacroRegistry returns the macro registry for registering Go macros This allows you to register native Go macros for high-performance execution

func (*Runtime) Parse 

func (r *Runtime) Parse(inputs Inputs, vars Vars, options *ParseOptions) (interface{}, error)

Parse executes the runtime with given inputs and variables

func (*Runtime) ParseWithValidation 

func (r *Runtime) ParseWithValidation(inputs Inputs, vars Vars, options *ParseOptions) (*ParseResult, error)

ParseWithValidation parses and returns both data and validation results

type SerializationFormat 

type SerializationFormat = compiler.SerializationFormat

SerializationFormat represents the format for serialization 

const (
	FormatGob  SerializationFormat = compiler.FormatGob
	FormatJSON SerializationFormat = compiler.FormatJSON
	FormatYAML SerializationFormat = compiler.FormatYAML
)

type SourceMap 

type SourceMap struct {
	Inputs map[string]*InputSourceMap // input name -> source map
}

SourceMap tracks which parts of input text matched which template patterns

type VarRange 

type VarRange struct {
	StartCol int // start column (0-indexed)
	EndCol   int // end column (exclusive)
}

VarRange represents the character range for a variable within a match

type Vars 

type Vars map[string]interface{}

Vars represents template variables

type YANGModules 

type YANGModules struct {
	Modules map[string]string // name -> content
	Files   []string          // file paths
	URLs    []string          // URLs to fetch
}

YANGModules represents YANG modules for validation

Source Files

View all Source files

Directories

Path Synopsis
api
python
examples
basic command
codegen command
macro command
python-api command
serialize command
internal
compiled
compiler
errors
formatters
functions/group
functions/input
functions/match
functions/output
input
lookup
macro
parser
pattern
returners
validator
variable
yang

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.