lexer

README

All code in this package was pulled from caddy 's caddyfile package.

The commit the code was at is caddy@3903642

There has been some modifications made to this code in order to make it not require caddy as a dependency. Specifically replacer.go and replacer_test.go was pulled from Caddy's main package.

Documentation

Overview

Package lexer ...

Index

• func Format(input []byte) []byte
• type Block
  • func Parse(filename string, input []byte) ([]Block, error)
  • func (sb Block) DispenseDirective(dir string) *Dispenser
• type CtxKey
• type Dispenser
  • func NewDispenser(tokens []Token) *Dispenser
  • func NewTestDispenser(input string) *Dispenser
  • func (d *Dispenser) AllArgs(targets ...*string) bool
  • func (d *Dispenser) ArgErr() error
  • func (d *Dispenser) Args(targets ...*string) bool
  • func (d *Dispenser) Delete() []Token
  • func (d *Dispenser) EOFErr() error
  • func (d *Dispenser) Err(msg string) error
  • func (d *Dispenser) Errf(format string, args ...interface{}) error
  • func (d *Dispenser) File() string
  • func (d *Dispenser) Line() int
  • func (d *Dispenser) Nesting() int
  • func (d *Dispenser) NewFromNextSegment() *Dispenser
  • func (d *Dispenser) Next() bool
  • func (d *Dispenser) NextArg() bool
  • func (d *Dispenser) NextBlock(initialNestingLevel int) bool
  • func (d *Dispenser) NextLine() bool
  • func (d *Dispenser) NextSegment() Segment
  • func (d *Dispenser) Prev() bool
  • func (d *Dispenser) RemainingArgs() []string
  • func (d *Dispenser) Reset()
  • func (d *Dispenser) SyntaxErr(expected string) error
  • func (d *Dispenser) Token() Token
  • func (d *Dispenser) Val() string
• type ReplacementFunc
• type Replacer
  • func NewReplacer() *Replacer
  • func (r *Replacer) Delete(variable string)
  • func (r *Replacer) Get(variable string) (interface{}, bool)
  • func (r *Replacer) GetString(variable string) (string, bool)
  • func (r *Replacer) Map(mapFunc ReplacerFunc)
  • func (r *Replacer) ReplaceAll(input, empty string) string
  • func (r *Replacer) ReplaceFunc(input string, f ReplacementFunc) (string, error)
  • func (r *Replacer) ReplaceKnown(input, empty string) string
  • func (r *Replacer) ReplaceOrErr(input string, errOnEmpty, errOnUnknown bool) (string, error)
  • func (r *Replacer) Set(variable string, value interface{})
• type ReplacerFunc
• type Segment
  • func (s Segment) Directive() string
• type Token
  • func Tokenize(filename string, v []byte) ([]Token, error)

Functions

func Format

func Format(input []byte) []byte

Format formats the input Caddyfile to a standard, nice-looking appearance. It works by reading each rune of the input and taking control over all the bracing and whitespace that is written; otherwise, words, comments, placeholders, and escaped characters are all treated literally and written as they appear in the input.

Types

type Block

type Block struct {
	Keys     []string
	Segments []Segment
}

Block associates any number of keys from the head of the server block with tokens, which are grouped by segments.

func Parse

func Parse(filename string, input []byte) ([]Block, error)

Parse parses the input just enough to group tokens, in order, by server block. No further parsing is performed. Server blocks are returned in the order in which they appear. Directives that do not appear in validDirectives will cause an error. If you do not want to check for valid directives, pass in nil instead.

Environment variables in {$ENVIRONMENT_VARIABLE} notation will be replaced before parsing begins.

func (Block) DispenseDirective

func (sb Block) DispenseDirective(dir string) *Dispenser

DispenseDirective returns a dispenser that contains all the tokens in the server block.

type CtxKey

type CtxKey string

const ReplacerCtxKey CtxKey = "replacer"

ReplacerCtxKey is the context key for a replacer.

type Dispenser

type Dispenser struct {
	// contains filtered or unexported fields
}

Dispenser is a type that dispenses tokens, similarly to a lexer, except that it can do so with some notion of structure. An empty Dispenser is invalid; call NewDispenser to make a proper instance.

func NewDispenser

func NewDispenser(tokens []Token) *Dispenser

NewDispenser returns a Dispenser filled with the given tokens.

func NewTestDispenser

func NewTestDispenser(input string) *Dispenser

NewTestDispenser parses input into tokens and creates a new Dispenser for test purposes only; any errors are fatal.

func (*Dispenser) AllArgs

func (d *Dispenser) AllArgs(targets ...*string) bool

AllArgs is like Args, but if there are more argument tokens available than there are targets, false is returned. The number of available argument tokens must match the number of targets exactly to return true.

func (*Dispenser) ArgErr

func (d *Dispenser) ArgErr() error

ArgErr returns an argument error, meaning that another argument was expected but not found. In other words, a line break or open curly brace was encountered instead of an argument.

func (*Dispenser) Args

func (d *Dispenser) Args(targets ...*string) bool

Args is a convenience function that loads the next arguments (tokens on the same line) into an arbitrary number of strings pointed to in targets. If there are not enough argument tokens available to fill targets, false is returned and the remaining targets are left unchanged. If all the targets are filled, then true is returned.

func (*Dispenser) Delete

func (d *Dispenser) Delete() []Token

Delete deletes the current token and returns the updated slice of tokens. The cursor is not advanced to the next token. Because deletion modifies the underlying slice, this method should only be called if you have access to the original slice of tokens and/or are using the slice of tokens outside this Dispenser instance. If you do not re-assign the slice with the return value of this method, inconsistencies in the token array will become apparent (or worse, hide from you like they did me for 3 and a half freaking hours late one night).

func (*Dispenser) EOFErr

func (d *Dispenser) EOFErr() error

EOFErr returns an error indicating that the dispenser reached the end of the input when searching for the next token.

func (*Dispenser) Err

func (d *Dispenser) Err(msg string) error

Err generates a custom parse-time error with a message of msg.

func (*Dispenser) Errf

func (d *Dispenser) Errf(format string, args ...interface{}) error

Errf is like Err, but for formatted error messages

func (*Dispenser) File

func (d *Dispenser) File() string

File gets the filename where the current token originated.

func (*Dispenser) Line

func (d *Dispenser) Line() int

Line gets the line number of the current token. If there is no token loaded, it returns 0.

func (*Dispenser) Nesting

func (d *Dispenser) Nesting() int

Nesting returns the current nesting level. Necessary if using NextBlock()

func (*Dispenser) NewFromNextSegment

func (d *Dispenser) NewFromNextSegment() *Dispenser

NewFromNextSegment returns a new dispenser with a copy of the tokens from the current token until the end of the "directive" whether that be to the end of the line or the end of a block that starts at the end of the line; in other words, until the end of the segment.

func (*Dispenser) Next

func (d *Dispenser) Next() bool

Next loads the next token. Returns true if a token was loaded; false otherwise. If false, all tokens have been consumed.

func (*Dispenser) NextArg

func (d *Dispenser) NextArg() bool

NextArg loads the next token if it is on the same line and if it is not a block opening (open curly brace). Returns true if an argument token was loaded; false otherwise. If false, all tokens on the line have been consumed except for potentially a block opening. It handles imported tokens correctly.

func (*Dispenser) NextBlock

func (d *Dispenser) NextBlock(initialNestingLevel int) bool

NextBlock can be used as the condition of a for loop to load the next token as long as it opens a block or is already in a block nested more than initialNestingLevel. In other words, a loop over NextBlock() will iterate all tokens in the block assuming the next token is an open curly brace, until the matching closing brace. The open and closing brace tokens for the outer-most block will be consumed internally and omitted from the iteration.

Proper use of this method looks like this:

for nesting := d.Nesting(); d.NextBlock(nesting); {
}

However, in simple cases where it is known that the Dispenser is new and has not already traversed state by a loop over NextBlock(), this will do:

for d.NextBlock(0) {
}

As with other token parsing logic, a loop over NextBlock() should be contained within a loop over Next(), as it is usually prudent to skip the initial token.

func (*Dispenser) NextLine

func (d *Dispenser) NextLine() bool

NextLine loads the next token only if it is not on the same line as the current token, and returns true if a token was loaded; false otherwise. If false, there is not another token or it is on the same line. It handles imported tokens correctly.

func (*Dispenser) NextSegment

func (d *Dispenser) NextSegment() Segment

NextSegment returns a copy of the tokens from the current token until the end of the line or block that starts at the end of the line.

func (*Dispenser) Prev

func (d *Dispenser) Prev() bool

Prev moves to the previous token. It does the inverse of Next(), except this function may decrement the cursor to -1 so that the next call to Next() points to the first token; this allows dispensing to "start over". This method returns true if the cursor ends up pointing to a valid token.

func (*Dispenser) RemainingArgs

func (d *Dispenser) RemainingArgs() []string

RemainingArgs loads any more arguments (tokens on the same line) into a slice and returns them. Open curly brace tokens also indicate the end of arguments, and the curly brace is not included in the return value nor is it loaded.

func (*Dispenser) Reset

func (d *Dispenser) Reset()

Reset sets d's cursor to the beginning, as if this was a new and unused dispenser.

func (*Dispenser) SyntaxErr

func (d *Dispenser) SyntaxErr(expected string) error

SyntaxErr creates a generic syntax error which explains what was found and what was expected.

func (*Dispenser) Token

func (d *Dispenser) Token() Token

Token returns the current token.

func (*Dispenser) Val

func (d *Dispenser) Val() string

Val gets the text of the current token. If there is no token loaded, it returns empty string.

type ReplacementFunc

type ReplacementFunc func(variable string, val interface{}) (interface{}, error)

ReplacementFunc is a function that is called when a replacement is being performed. It receives the variable (i.e. placeholder name) and the value that will be the replacement, and returns the value that will actually be the replacement, or an error. Note that errors are sometimes ignored by replacers.

type Replacer

type Replacer struct {
	// contains filtered or unexported fields
}

Replacer can replace values in strings. A default/empty Replacer is not valid; use NewReplacer to make one.

func NewReplacer

func NewReplacer() *Replacer

NewReplacer returns a new Replacer.

func (*Replacer) Delete

func (r *Replacer) Delete(variable string)

Delete removes a variable with a static value that was created using Set.

func (*Replacer) Get

func (r *Replacer) Get(variable string) (interface{}, bool)

Get gets a value from the replacer. It returns the value and whether the variable was known.

func (*Replacer) GetString

func (r *Replacer) GetString(variable string) (string, bool)

GetString is the same as Get, but coerces the value to a string representation.

func (*Replacer) Map

func (r *Replacer) Map(mapFunc ReplacerFunc)

Map adds mapFunc to the list of value providers. mapFunc will be executed only at replace-time.

func (*Replacer) ReplaceAll

func (r *Replacer) ReplaceAll(input, empty string) string

ReplaceAll efficiently replaces placeholders in input with their values. All placeholders are replaced in the output whether they are recognized or not. Values that are empty string will be substituted with empty.

func (*Replacer) ReplaceFunc

func (r *Replacer) ReplaceFunc(input string, f ReplacementFunc) (string, error)

ReplaceFunc is the same as ReplaceAll, but calls f for every replacement to be made, in case f wants to change or inspect the replacement.

func (*Replacer) ReplaceKnown

func (r *Replacer) ReplaceKnown(input, empty string) string

ReplaceKnown is like ReplaceAll but only replaces placeholders that are known (recognized). Unrecognized placeholders will remain in the output.

func (*Replacer) ReplaceOrErr

func (r *Replacer) ReplaceOrErr(input string, errOnEmpty, errOnUnknown bool) (string, error)

ReplaceOrErr is like ReplaceAll, but any placeholders that are empty or not recognized will cause an error to be returned.

func (*Replacer) Set

func (r *Replacer) Set(variable string, value interface{})

Set sets a custom variable to a static value.

type ReplacerFunc

type ReplacerFunc func(key string) (interface{}, bool)

ReplacerFunc is a function that returns a replacement for the given key along with true if the function is able to service that key (even if the value is blank). If the function does not recognize the key, false should be returned.

type Segment

type Segment []Token

Segment is a list of tokens which begins with a directive and ends at the end of the directive (either at the end of the line, or at the end of a block it opens).

func (Segment) Directive

func (s Segment) Directive() string

Directive returns the directive name for the segment. The directive name is the text of the first token.

type Token

type Token struct {
	// File .
	File string
	// Line .
	Line int
	// Text .
	Text string
	// contains filtered or unexported fields
}

Token .

func Tokenize

func Tokenize(filename string, v []byte) ([]Token, error)

Tokenize .