vclparser

README

VCL Parser for Go

A VCL (Varnish Configuration Language) parser implemented in Go that parses VCL files into Abstract Syntax Trees (AST).

Features

• Complete lexical analysis of VCL syntax
• Recursive descent parser with error recovery
• Type-safe AST representation
• VCL renderer to convert AST back to source code
• Integrated include resolution with parser options
• Functional options pattern for flexible configuration
• Symbol table and semantic analysis
• Visitor pattern for AST traversal
• VMOD and variable semantics loaded from varnishd build

Semantics

Semantics like what variables are available in a given context are defined in the metadata package, in a JSON file that is generated by the generate.py script inside varnishd. This file is embedded into the library at compile time.

VMOD semantics are loaded from a collection of VCC files in vcclib. These are embedded into the library at compile time.

VMOD Registry Initialization

The VMOD registry supports multiple initialization modes, depending on how much control you want:

import "github.com/perbu/vclparser/pkg/vmod"

1. Default registry (recommended for most users)

registry := vmod.NewRegistry()

NewRegistry() does two best-effort steps:

• Loads embedded VMOD interfaces from VCC files (from vcclib)
• Tries to load libvmod_*.so from local vmods directories if present

1. Empty registry (full manual control)

registry := vmod.NewEmptyRegistry()

NewEmptyRegistry() starts with no modules loaded.

1. Explicit loading into a registry

Load a specific VCC file:

registry := vmod.NewEmptyRegistry()
if err := registry.LoadVCCFile("path/to/vmod_std.vcc"); err != nil {
    // handle error
}

Load a specific shared object:

registry := vmod.NewEmptyRegistry()
if err := registry.LoadSOFile("path/to/libvmod_std.so"); err != nil {
    // handle error
}

Load all VMOD shared objects in a directory:

registry := vmod.NewEmptyRegistry()
if err := registry.LoadSODirectory("path/to/vmods"); err != nil {
    // handle error
}

Load embedded VCC definitions explicitly (when using an empty registry):

registry := vmod.NewEmptyRegistry()
if err := registry.LoadEmbeddedVCCs(); err != nil {
    // handle error
}

Usage

Basic Parsing

package main

import (
	"fmt"
	"log"

	"github.com/perbu/vclparser/pkg/parser"
)

func main() {
	vclCode := `
    vcl 4.0;

    backend default {
        .host = "127.0.0.1";
        .port = "8080";
    }

    sub vcl_recv {
        if (req.method == "GET") {
            return (hash);
        }
    }
    `

	program, err := parser.Parse(vclCode, "example.vcl")
	if err != nil {
		log.Fatal(err)
	}

	fmt.Printf("Parsed VCL with %d declarations\n", len(program.Declarations))
}

Parsing with Options

The parser uses a functional options pattern for configuration:

// Parse with custom error limit
program, err := parser.Parse(vclCode, "example.vcl",
    parser.WithMaxErrors(10),
)

// Parse included files (allow missing version declaration)
program, err := parser.Parse(includedFileContent, "backend.vcl",
    parser.WithAllowMissingVersion(true),
)

// Multiple options
program, err := parser.Parse(vclCode, "example.vcl",
    parser.WithMaxErrors(5),
    parser.WithDisableInlineC(true),
)

Parsing with Include Resolution

The parser can automatically resolve and merge include statements:

package main

import (
	"fmt"
	"log"
	"os"

	"github.com/perbu/vclparser/pkg/parser"
)

func main() {
	content, _ := os.ReadFile("main.vcl")

	// Parse and automatically resolve all includes
	program, err := parser.Parse(string(content), "main.vcl",
		parser.WithResolveIncludes("/etc/varnish"),
		parser.WithIncludeMaxDepth(10),
	)
	if err != nil {
		log.Fatal(err)
	}

	fmt.Printf("Parsed VCL with %d declarations (includes merged)\n",
		len(program.Declarations))
}

Rendering VCL

package main

import (
	"fmt"
	"log"

	"github.com/perbu/vclparser/pkg/parser"
	"github.com/perbu/vclparser/pkg/renderer"
)

func main() {
	// Parse VCL
	program, err := parser.Parse(vclCode, "example.vcl")
	if err != nil {
		log.Fatal(err)
	}

	// Render back to VCL source code
	rendered := renderer.Render(program)
	fmt.Println(rendered)
}

Command-Line Tools

# Format a VCL file
go run ./examples/render/main.go -file input.vcl

# Merge includes into a single file
go run ./examples/render/main.go -file main.vcl -resolve-includes -output merged.vcl

# Parse and analyze VCL
go run ./examples/parse/main.go file.vcl

Architecture

• pkg/lexer/ - Lexical analysis and tokenization
• pkg/ast/ - AST node definitions and visitor pattern
• pkg/parser/ - Recursive descent parser implementation
• pkg/renderer/ - AST to VCL source code renderer
• pkg/types/ - Type system and symbol table
• pkg/include/ - Include statement resolution
• examples/ - Usage examples (parse, render, includes)
• tests/testdata/ - Test VCL files

VCL Language Support

This parser supports the full VCL language including:

• Version declarations (vcl 4.0;)
• Backend definitions with properties
• Access Control Lists (ACLs)
• Probe definitions
• Subroutine definitions
• All VCL statements (if/else, set, unset, call, return, etc.)
• Expression parsing with proper operator precedence
• Built-in variables and functions
• C-code blocks (C{ }C)

Testing

go test ./...

Documentation

Index

• func GetEmbeddedVCCContent(filename string) ([]byte, error)
• func GetEmbeddedVCCFiles() embed.FS
• func ListEmbeddedVCCFiles() ([]string, error)
• func OpenEmbeddedVCCFile(filename string) (io.ReadCloser, error)

Functions

func GetEmbeddedVCCContent

func GetEmbeddedVCCContent(filename string) ([]byte, error)

GetEmbeddedVCCContent reads the entire content of an embedded VCC file

func GetEmbeddedVCCFiles

func GetEmbeddedVCCFiles() embed.FS

GetEmbeddedVCCFiles returns the embedded filesystem containing all VCC files

func ListEmbeddedVCCFiles

func ListEmbeddedVCCFiles() ([]string, error)

ListEmbeddedVCCFiles returns a list of all embedded VCC file paths

func OpenEmbeddedVCCFile

func OpenEmbeddedVCCFile(filename string) (io.ReadCloser, error)

OpenEmbeddedVCCFile opens a specific embedded VCC file for reading