hcllight

README

HCL Light

A lightweight, position-agnostic HCL (HashiCorp Configuration Language) parser and manipulator for Go, built on top of hclsyntax.

Table of Contents

• Overview
• Features
• Installation
• Quick Start
• Usage
  • Parsing HCL
  • Marshaling to HCL
  • Evaluating Expressions
  • Custom Functions
• Package Structure
• API Reference
• Examples
• Performance Benchmarks
• License

Overview

hcllight removes position and location tags from the official HCL AST parsing package, enabling flexible programmatic manipulation of HCL documents. This makes it ideal for:

• Dynamic configuration generation - Build HCL configurations programmatically
• Configuration transformation - Parse, modify, and regenerate HCL files
• Template processing - Evaluate HCL expressions with custom functions and variables
• Configuration analysis - Traverse and inspect HCL document structures

Unlike the standard HCL parser which maintains source positions for error reporting, hcllight focuses on the logical structure, making it easier to create, modify, and regenerate HCL programmatically.

Note: This package is designed for parsing and manipulating HCL strings programmatically. It is not a primary marshaler/unmarshaler between HCL strings and Go structs. For marshaling and unmarshaling between HCL and Go structs, use github.com/genelet/horizon.

The main functions you'll use to parse and manipulate HCL strings are:

• ParseBody - Parse HCL into a Body structure
• Body.Evaluate - Evaluate expressions within the Body
• Body.MarshalHCL - Marshal Body structure to HCL string
• Body.UnmarshalHCL - Unmarshal HCL string into Body structure

Features

• Position-independent AST - Manipulate HCL structures without worrying about source locations
• Dynamic manipulation - Add, modify, or remove attributes, blocks, and expressions after parsing
• Bidirectional conversion - Parse HCL to AST and marshal AST back to HCL
• Expression evaluation - Evaluate HCL expressions with or without custom variables and functions
• Custom function support - Register custom functions for expression evaluation
• Protobuf integration - AST structures are defined using Protocol Buffers for serialization
• Full HCL feature support - Handle all HCL constructs including blocks, attributes, expressions, and for-loops

Installation

go get github.com/genelet/hcllight

Requirements:

• Go 1.23.0 or later

Quick Start

package main

import (
    "fmt"
    "github.com/genelet/hcllight/light"
)

func main() {
    // Parse HCL data
    hclData := []byte(`
        name = "example"
        version = 1

        config {
            enabled = true
        }
    `)

    body, err := light.Parse(hclData)
    if err != nil {
        panic(err)
    }

    // Marshal back to HCL
    output, err := body.MarshalHCL()
    if err != nil {
        panic(err)
    }

    fmt.Println(string(output))
}

Usage

Parsing HCL

Parse HCL data into a Body structure:

import "github.com/genelet/hcllight/light"

hclData := []byte(`
    environment = "production"
    max_retries = 3
`)

body, err := light.Parse(hclData)
if err != nil {
    // Handle error
}

You can also unmarshal directly into a Body:

body := &light.Body{}
err := body.UnmarshalHCL(hclData)

Marshaling to HCL

Convert a Body structure back to HCL format:

hclOutput, err := body.MarshalHCL()
if err != nil {
    // Handle error
}
fmt.Println(string(hclOutput))

Evaluating Expressions

Evaluate HCL expressions with variables and functions:

import (
    "github.com/zclconf/go-cty/cty"
    "github.com/zclconf/go-cty/cty/function"
)

// Define evaluation context
context := map[string]any{
    "variables": map[string]cty.Value{
        "env": cty.StringVal("production"),
    },
    "functions": map[string]function.Function{
        // Custom functions here
    },
}

// Evaluate expressions
evaluated, err := body.Evaluate(context)
if err != nil {
    // Handle error
}

fmt.Println(string(evaluated))

Custom Functions

Register custom functions for use in HCL expressions:

import (
    "github.com/zclconf/go-cty/cty"
    "github.com/zclconf/go-cty/cty/function"
    "math/rand"
)

randomFunc := function.New(&function.Spec{
    Params: []function.Parameter{
        {Type: cty.Number},
    },
    Type: func(args []cty.Value) (cty.Type, error) {
        return cty.String, nil
    },
    Impl: func(args []cty.Value, retType cty.Type) (cty.Value, error) {
        n, _ := args[0].AsBigFloat().Int64()
        letterRunes := []rune("abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ")
        b := make([]rune, n)
        for i := range b {
            b[i] = letterRunes[rand.Intn(len(letterRunes))]
        }
        return cty.StringVal(string(b)), nil
    },
})

context := map[string]any{
    "functions": map[string]function.Function{
        "random": randomFunc,
    },
}

Package Structure

The project is organized into the following packages:

light/

The main package providing HCL parsing, marshaling, and evaluation functionality.

Key types:

• Body - Represents an HCL document or block body
• Block - Represents an HCL block
• Attribute - Represents an HCL attribute
• Expression - Represents HCL expressions

Key functions:

• Parse([]byte) (*Body, error) - Parse HCL data into a Body
• Body.MarshalHCL() ([]byte, error) - Marshal Body to HCL format
• Body.Evaluate(context) ([]byte, error) - Evaluate expressions in the Body

internal/ast/

Internal package for AST conversion between hclsyntax and hcllight representations.

internal/

Protocol Buffer definitions for OpenAPI v3 structures.

API Reference

For detailed API documentation, see:

Main packages:

• github.com/genelet/hcllight/light - Core HCL parsing and manipulation
• github.com/genelet/hcllight/internal/ast - AST conversion utilities

Examples

Complete Example with Evaluation

This example demonstrates parsing HCL, marshaling it back, and evaluating expressions:

package main

import (
    "fmt"
    "math/rand"
    "os"

    "github.com/genelet/hcllight/light"
    "github.com/zclconf/go-cty/cty"
    "github.com/zclconf/go-cty/cty/function"
)

func main() {
    // Sample HCL content
    hclContent := `
TEST_FOLDER = "__test__"
EXECUTION_ID = random(6)
version = 2
say = {
    for k, v in {hello: "world"}: k => v if k == "hello"
}

job check "this is a temporal job" {
    python "run.py" {}
}

job e2e "running integration tests" {
    python "app-e2e.py" {
        root_dir = var.TEST_FOLDER
        python_version = version + 6
    }

    slack {
        channel  = "slack-my-channel"
        message = "Job execution ${EXECUTION_ID} completed successfully"
    }
}
`

    // Parse HCL
    body, err := light.Parse([]byte(hclContent))
    if err != nil {
        panic(err)
    }

    // Marshal back to HCL (without evaluation)
    hcl, err := body.MarshalHCL()
    if err != nil {
        panic(err)
    }
    fmt.Println("HCL Output:")
    fmt.Println(string(hcl))

    // Evaluate expressions
    context := getEvaluationContext()
    evaluated, err := body.Evaluate(context)
    if err != nil {
        panic(err)
    }
    fmt.Println("\nEvaluated Output:")
    fmt.Println(string(evaluated))
}

func getEvaluationContext() map[string]any {
    return map[string]any{
        "functions": map[string]function.Function{
            "random": function.New(&function.Spec{
                Params: []function.Parameter{
                    {Type: cty.Number},
                },
                Type: func(args []cty.Value) (cty.Type, error) {
                    return cty.String, nil
                },
                Impl: func(args []cty.Value, retType cty.Type) (cty.Value, error) {
                    letterRunes := []rune("abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ")
                    n, _ := args[0].AsBigFloat().Int64()
                    b := make([]rune, n)
                    for i := range b {
                        b[i] = letterRunes[rand.Intn(len(letterRunes))]
                    }
                    return cty.StringVal(string(b)), nil
                },
            }),
        },
    }
}

Output:

HCL Output:
TEST_FOLDER = "__test__"
EXECUTION_ID = random(6)
version = 2
say = {
  for k, v in {hello:"world"}: k => v if k == "hello"
}
job "check" "this is a temporal job" {
  python "run.py" {}
}
job "e2e" "running integration tests" {
  python "app-e2e.py" {
    root_dir = var.TEST_FOLDER
    python_version = version + 6
  }
  slack {
    message = "Job execution ${EXECUTION_ID} completed successfully"
    channel = "slack-my-channel"
  }
}

Evaluated Output:
say = {
  hello = "world"
}
TEST_FOLDER = "__test__"
EXECUTION_ID = "nOVqNf"
version = 2
job "check" "this is a temporal job" {
  python "run.py" {}
}
job "e2e" "running integration tests" {
  python "app-e2e.py" {
    root_dir = "__test__"
    python_version = 8
  }
  slack {
    message = "Job execution nOVqNf completed successfully"
    channel = "slack-my-channel"
  }
}

Performance Benchmarks

hcllight includes comprehensive benchmarks to measure and track performance. See BENCHMARKS.md for detailed analysis and results.

Quick Benchmark Overview

Fast Operations (< 1 μs):

• Primitive conversions (int, float, bool): 5 ns/op, 0 allocations ✨
• Boolean expressions: 187 ns/op
• String expressions: 320 ns/op

Medium Operations (1-10 μs):

• Small body marshal: 3.1 μs/op
• List expressions: 2.1 μs/op
• Map expressions: 3.5 μs/op

Heavy Operations (> 10 μs):

• Medium body marshal: 15.5 μs/op
• HCL parsing: 45-195 μs/op
• Expression evaluation: 100-150 μs/op

Running Benchmarks

Run all benchmarks:

go test -bench=. -benchmem ./light/ -run=^$

Run specific benchmarks:

# Marshal benchmarks
go test -bench=BenchmarkBodyMarshalHCL -benchmem ./light/ -run=^$

# Parse benchmarks
go test -bench=BenchmarkParseBody -benchmem ./light/ -run=^$

# Expression benchmarks
go test -bench=BenchmarkExpressionHCL -benchmem ./light/ -run=^$

Compare before/after changes:

# Save baseline
go test -bench=. -benchmem ./light/ -run=^$ > old.txt

# Make changes, then compare
go test -bench=. -benchmem ./light/ -run=^$ > new.txt
benchstat old.txt new.txt  # requires: go install golang.org/x/perf/cmd/benchstat@latest

CPU Profiling

Profile CPU usage to find bottlenecks:

go test -bench=. -cpuprofile=cpu.prof ./light/ -run=^$
go tool pprof -http=:8080 cpu.prof

Memory Profiling

Profile memory allocations:

go test -bench=. -memprofile=mem.prof ./light/ -run=^$
go tool pprof -http=:8080 mem.prof

Benchmark Coverage

The benchmark suite includes:

• Body operations - Marshaling, parsing, evaluation (4-6 scenarios each)
• Expression operations - All expression types (strings, numbers, lists, maps, etc.)
• Type conversions - Go types to HCL expressions
• Deterministic iteration - Sorted map overhead analysis (5-100 attributes)
• Round-trip performance - Full parse → marshal cycle

For detailed results, analysis, and optimization history, see BENCHMARKS.md.

License

This project is licensed under the MIT License - see the LICENSE file for details.

Contributing

Contributions are welcome! Please feel free to submit issues or pull requests.

Related Projects

• hashicorp/hcl - The official HCL parser
• zclconf/go-cty - Type system for HCL expressions
• google/gnostic - Tools for working with OpenAPI specifications