abnf

README

Forked from https://github.com/funwithbots/go-abnf

Augmented BNF for Syntax Specifications: ABNF

Internet technical specifications often need to define a formal syntax and are free to employ whatever notation their authors deem useful. Over the years, a modified version of Backus-Naur Form (BNF), called Augmented BNF (ABNF), has been popular among many Internet specifications. It balances compactness and simplicity with reasonable representational power.

RFC 5234

Contents

! []byte(...) should be UTF-8 encoded!

Function Generator

A way to generate the operators in memory.

g := ParserGenerator{
	RawABNF: rawABNF,
}
functions := g.GenerateABNFAsOperators()
// e.g. functions["ALPHA"]([]byte("a"))

Code Generator

Both the Core ABNF and the ABNF Definition contained within this package where created by the generator.

corePkg := externalABNF{
	operator:    true,
	packageName: "github.com/selesy/go-abnf/core",
}
g := Generator{
	PackageName:  "definition",
	RawABNF:      rawABNF,
	ExternalABNF: map[string]ExternalABNF{
		"ALPHA":  corePkg,
		"BIT":    corePkg,
		// etc.
	},
}
f := g.GenerateABNFAsAlternatives()
// e.g. ioutil.WriteFile("./definition/abnf_definition.go", []byte(fmt.Sprintf("%#v", f)), 0644)

(Currently) Not Supported

• free-form prose
• incremental alternatives

Core ABNF

"Core" rules that are used variously among higher-level rules. The "core" rules might be formed into a lexical analyzer or simply be part of the main ruleset.

Operators

Elements form a sequence of one or more rule names and/or value definitions, combined according to the various operators defined in this package, such as alternative and repetition.

HEXDIG

In the spec HEXDIG is case insensitive.
i.e. 0x6e != 0x6E

HEXDIG = DIGIT / "A" / "B" / "C" / "D" / "E" / "F"

In this implementation it is so that 0x6e == 0x6E.

HEXDIG = DIGIT / "A" / "B" / "C" / "D" / "E" / "F"
               / "a" / "b" / "c" / "d" / "e" / "f"

EOL

Text files created on DOS/Windows machines have different line endings than files created on Unix/Linux. DOS uses carriage return and line feed (\r\n) as a line ending, which Unix uses just line feed (\n).

This is why this package also allows LF which is NOT compliant with the specification.

CRLF = CR LF / LF

Operator Precedence

RFC 5234 3.10

highest

1. Rule name, prose-val, Terminal value
2. Comment
3. Value range
4. Repetition
5. Grouping, Optional
6. Concatenation
7. Alternative

lowest

Automation

It is possible to specify a configuration using a YAML file to automate the code generation. To install the command:

go install github.com/selesy/go-abnf/cmd/abnf@latest
abnf generate

The generate command expects to find an abnf.yml file in its working directory. This YAML file defines and controls the code generation. To specify a different YAML file:

 abnf -f ./path/to/foo.yml generate

Creation of an empty YAML file is done by invoking the init function.

Generating an empty YAML file

abnf init

The abnf.yml file is written out in the current working directory. To specify a different location to write out the file, use the same -f flag:

abnf -f ./path/to/foo.yml init

Code Generation Configuration Properties

• version: must be defined as "1"
• spec: is the path to the ABNF specification file, e.g. ./testdata/core.abnf
• gofile: the name of the Go file to generated, e.g. core_abnf.go
• package: the name of the package for the Go file generated, e.g. core
• output: the output path where the package (folder) and Go file are written, e.g. .
• generate: the type of generation, {operators, alternatives}
• verbose: displays additional debugging information

The generate option either invokes GenerateABNFAsOperators or GenerateABNFAsAlternatives

Documentation

Index

• type AlternationOperator
  • func (alt AlternationOperator) Key() string
• type CharacterValueOperator
  • func (value CharacterValueOperator) Key() string
• type CodeGenerator
  • func (g *CodeGenerator) GenerateABNFAsAlternatives(w io.Writer)
  • func (g *CodeGenerator) GenerateABNFAsOperators(w io.Writer)
• type ConcatenationOperator
  • func (concat ConcatenationOperator) Key() string
• type ExternalABNF
• type NumericValueOperator
  • func (value NumericValueOperator) Key() string
• type Operator
• type OptionOperator
  • func (opt OptionOperator) Key() string
• type ParserGenerator
  • func (g *ParserGenerator) GenerateABNFAsOperators() map[string]operators.Operator
• type RepetitionOperator
  • func (rep RepetitionOperator) Key() string
• type Rule
  • func (r Rule) Equals(other Rule) error
  • func (r Rule) Name() string
• type RuleList
  • func (list RuleList) RuleSet() RuleSet
• type RuleNameOperator
  • func (name RuleNameOperator) Key() string
• type RuleSet
  • func NewRuleSet(rawABNF []byte) RuleSet
  • func (set RuleSet) RuleList() RuleList

Types

type AlternationOperator

type AlternationOperator struct {
	// contains filtered or unexported fields
}

AlternationOperator represents an alternation node of a rule.

func (AlternationOperator) Key

func (alt AlternationOperator) Key() string

type CharacterValueOperator

type CharacterValueOperator struct {
	// contains filtered or unexported fields
}

CharacterValueOperator represents a character value node of a rule.

func (CharacterValueOperator) Key

func (value CharacterValueOperator) Key() string

type CodeGenerator

type CodeGenerator struct {

	// PackageName of the generated file
	PackageName string
	// RawABNF syntax to parse
	RawABNF []byte
	// ExternalABNF reference to abnf syntax
	// e.g. ALPHA from github.com/selesy/go-abnf/core
	ExternalABNF map[string]ExternalABNF
	// contains filtered or unexported fields
}

func (*CodeGenerator) GenerateABNFAsAlternatives

func (g *CodeGenerator) GenerateABNFAsAlternatives(w io.Writer)

GenerateABNFAsAlternatives returns a *jen.File containing the given ABNF syntax as Go functions that return Alternatives.

func (*CodeGenerator) GenerateABNFAsOperators

func (g *CodeGenerator) GenerateABNFAsOperators(w io.Writer)

GenerateABNFAsOperators returns a *jen.File containing the given ABNF syntax as Go Operator functions.

type ConcatenationOperator

type ConcatenationOperator struct {
	// contains filtered or unexported fields
}

ConcatenationOperator represents a concatenation node of a rule.

func (ConcatenationOperator) Key

func (concat ConcatenationOperator) Key() string

type ExternalABNF

type ExternalABNF struct {
	// IsOperator: operator / alternatives
	IsOperator bool
	// PackagePath: e.g. github.com/selesy/go-abnf/core
	PackagePath string
	// PackageName: e.g. core
	PackageName string
}

type NumericValueOperator

type NumericValueOperator struct {
	// contains filtered or unexported fields
}

NumericValueOperator represents a numeric value node of a rule.

func (NumericValueOperator) Key

func (value NumericValueOperator) Key() string

type Operator

type Operator interface {
	// Key returns that key (name) of the operator.s
	Key() string
	// contains filtered or unexported methods
}

Operator represents a node of a rule.

type OptionOperator

type OptionOperator struct {
	// contains filtered or unexported fields
}

OptionOperator represents an option node of a rule.

func (OptionOperator) Key

func (opt OptionOperator) Key() string

type ParserGenerator

type ParserGenerator struct {
	// RawABNF syntax to parse
	RawABNF []byte
	// ExternalABNF reference to abnf syntax
	// e.g. ALPHA from github.com/selesy/go-abnf/core
	ExternalABNF map[string]operators.Operator

	sync.WaitGroup
	// contains filtered or unexported fields
}

func (*ParserGenerator) GenerateABNFAsOperators

func (g *ParserGenerator) GenerateABNFAsOperators() map[string]operators.Operator

type RepetitionOperator

type RepetitionOperator struct {
	// contains filtered or unexported fields
}

RepetitionOperator represents a repetition node of a rule.

func (RepetitionOperator) Key

func (rep RepetitionOperator) Key() string

type Rule

type Rule struct {
	// contains filtered or unexported fields
}

Rule represents an ABNF rule.

func (Rule) Equals

func (r Rule) Equals(other Rule) error

Equals checks whether both rule trees are equal to each other.

func (Rule) Name

func (r Rule) Name() string

Name returns the name of the rule.

type RuleList

type RuleList []Rule

RuleList is a list of ABNF rules.

func (RuleList) RuleSet

func (list RuleList) RuleSet() RuleSet

RuleSet converts a rule list to a set.

type RuleNameOperator

type RuleNameOperator struct {
	// contains filtered or unexported fields
}

RuleNameOperator represents a rule name node of a rule.

func (RuleNameOperator) Key

func (name RuleNameOperator) Key() string

type RuleSet

type RuleSet map[string]Rule

RuleSet is a set of ABNF rules.

func NewRuleSet

func NewRuleSet(rawABNF []byte) RuleSet

NewRuleSet converts given raw data to a set of ABNF rules.

func (RuleSet) RuleList

func (set RuleSet) RuleList() RuleList

RuleList converts a rule set to a list.