Documentation
¶
Overview ¶
Package ast provides AST nodes and ancillary structures and algorithms.
Package ast provides AST nodes and ancillary structures and algorithms.
Index ¶
- Variables
- func FodderAppend(a *Fodder, elem FodderElement)
- func FodderCountNewlines(fodder Fodder) int
- func FodderElementCountNewlines(elem FodderElement) int
- func FodderEnsureCleanNewline(fodder *Fodder)
- func FodderHasCleanEndline(fodder Fodder) bool
- func FodderMoveFront(a *Fodder, b *Fodder)
- func LocationBefore(a Location, b Location) bool
- type Apply
- type ApplyBrace
- type Arguments
- type Array
- type ArrayComp
- type Assert
- type Binary
- type BinaryOp
- type CommaSeparatedExpr
- type CommaSeparatedID
- type Conditional
- type Context
- type DesugaredObject
- type DesugaredObjectField
- type DesugaredObjectFields
- type DiagnosticFileName
- type Dollar
- type Error
- type Fodder
- type FodderElement
- type FodderKind
- type ForSpec
- type Function
- type Identifier
- type IdentifierSet
- func (set IdentifierSet) Add(ident Identifier) bool
- func (set IdentifierSet) AddIdentifiers(idents Identifiers)
- func (set IdentifierSet) Clone() IdentifierSet
- func (set IdentifierSet) Contains(ident Identifier) bool
- func (set IdentifierSet) Remove(ident Identifier)
- func (set IdentifierSet) ToOrderedSlice() []Identifier
- func (set IdentifierSet) ToSlice() Identifiers
- type Identifiers
- type IfSpec
- type Import
- type ImportBin
- type ImportStr
- type InSuper
- type Index
- type LiteralBoolean
- type LiteralNull
- type LiteralNumber
- type LiteralString
- type LiteralStringKind
- type Local
- type LocalBind
- type LocalBinds
- type Location
- type LocationRange
- func LineBeginning(loc *LocationRange) LocationRange
- func LineEnding(loc *LocationRange) LocationRange
- func LocationRangeBetween(a, b *LocationRange) LocationRange
- func MakeLocationRange(fn string, fc *Source, begin Location, end Location) LocationRange
- func MakeLocationRangeMessage(msg string) LocationRange
- type NamedArgument
- type Node
- type NodeBase
- type Nodes
- type Object
- type ObjectComp
- type ObjectField
- type ObjectFieldHide
- type ObjectFieldKind
- type ObjectFields
- type Parameter
- type Parens
- type Self
- type Slice
- type Source
- type SourceProvider
- type SuperIndex
- type Unary
- type UnaryOp
- type Var
Constants ¶
This section is empty.
Variables ¶
var BopMap = map[string]BinaryOp{ "*": BopMult, "/": BopDiv, "%": BopPercent, "+": BopPlus, "-": BopMinus, "<<": BopShiftL, ">>": BopShiftR, ">": BopGreater, ">=": BopGreaterEq, "<": BopLess, "<=": BopLessEq, "in": BopIn, "==": BopManifestEqual, "!=": BopManifestUnequal, "&": BopBitwiseAnd, "^": BopBitwiseXor, "|": BopBitwiseOr, "&&": BopAnd, "||": BopOr, }
BopMap is a map from binary operator token strings to BinaryOp values.
var UopMap = map[string]UnaryOp{ "!": UopNot, "~": UopBitwiseNot, "+": UopPlus, "-": UopMinus, }
UopMap is a map from unary operator token strings to UnaryOp values.
Functions ¶
func FodderAppend ¶ added in v0.11.2
func FodderAppend(a *Fodder, elem FodderElement)
FodderAppend appends to the fodder but preserves constraints.
See FodderConcat below.
func FodderCountNewlines ¶ added in v0.11.2
FodderCountNewlines returns the number of new line chars represented by the fodder.
func FodderElementCountNewlines ¶ added in v0.11.2
func FodderElementCountNewlines(elem FodderElement) int
FodderElementCountNewlines returns the number of new line chars represented by the fodder element
func FodderEnsureCleanNewline ¶ added in v0.11.2
func FodderEnsureCleanNewline(fodder *Fodder)
FodderEnsureCleanNewline adds a LineEnd to the fodder if necessary.
func FodderHasCleanEndline ¶ added in v0.11.2
FodderHasCleanEndline is true if the fodder doesn't end with an interstitial.
func FodderMoveFront ¶ added in v0.11.2
FodderMoveFront moves b to the front of a.
func LocationBefore ¶ added in v0.16.0
LocationBefore returns whether one code location refers to the location closer to the beginning of the file than the other one.
Types ¶
type Apply ¶
type Apply struct { Target Node FodderLeft Fodder Arguments Arguments FodderRight Fodder TailStrictFodder Fodder NodeBase // Always false if there were no arguments. TrailingComma bool TailStrict bool }
Apply represents a function call
type ApplyBrace ¶
ApplyBrace represents e { }. Desugared to e + { }.
type Arguments ¶
type Arguments struct { Positional []CommaSeparatedExpr Named []NamedArgument }
Arguments represents positional and named arguments to a function call f(x, y, z=1).
type Array ¶
type Array struct { Elements []CommaSeparatedExpr CloseFodder Fodder NodeBase // Always false if there were no elements. TrailingComma bool }
Array represents array constructors [1, 2, 3].
type ArrayComp ¶
type ArrayComp struct { Body Node TrailingCommaFodder Fodder Spec ForSpec CloseFodder Fodder NodeBase TrailingComma bool }
ArrayComp represents array comprehensions (which are like Python list comprehensions)
type Assert ¶
type Assert struct { Cond Node Message Node Rest Node ColonFodder Fodder SemicolonFodder Fodder NodeBase }
Assert represents an assert expression (not an object-level assert).
After parsing, message can be nil indicating that no message was specified. This AST is elimiated by desugaring.
type BinaryOp ¶
type BinaryOp int
BinaryOp represents a binary operator.
type CommaSeparatedExpr ¶ added in v0.15.0
CommaSeparatedExpr represents an expression that is an element of a comma-separated list of expressions (e.g. in an array or the arguments of a call)
type CommaSeparatedID ¶ added in v0.15.0
type CommaSeparatedID struct { NameFodder Fodder Name Identifier CommaFodder Fodder }
CommaSeparatedID represents an expression that is an element of a comma-separated list of identifiers (e.g. an array of parameters)
type Conditional ¶
type Conditional struct { Cond Node BranchTrue Node BranchFalse Node ThenFodder Fodder ElseFodder Fodder NodeBase }
Conditional represents if/then/else.
After parsing, branchFalse can be nil indicating that no else branch was specified. The desugarer fills this in with a LiteralNull
type Context ¶
type Context *string
Context represents the surrounding context of a node (e.g. a function it's in)
type DesugaredObject ¶
type DesugaredObject struct { Asserts Nodes Fields DesugaredObjectFields Locals LocalBinds NodeBase }
DesugaredObject represents object constructors { f: e ... } after desugaring.
The assertions either return true or raise an error.
type DesugaredObjectField ¶
type DesugaredObjectField struct { Name Node Body Node LocRange LocationRange Hide ObjectFieldHide PlusSuper bool }
DesugaredObjectField represents a desugared object field.
type DesugaredObjectFields ¶
type DesugaredObjectFields []DesugaredObjectField
DesugaredObjectFields represents a DesugaredObjectField slice.
type DiagnosticFileName ¶ added in v0.17.0
type DiagnosticFileName string
DiagnosticFileName is a file name used for diagnostics. It might be a dummy value, such as <std> or <extvar:something>. It should never be passed to an importer.
type Fodder ¶ added in v0.11.2
type Fodder []FodderElement
Fodder is stuff that is usually thrown away by lexers/preprocessors but is kept so that the source can be round tripped with near full fidelity.
func FodderConcat ¶ added in v0.11.2
FodderConcat concats the two fodders but also preserves constraints.
Namely, a FodderLineEnd is not allowed to follow a FodderParagraph or a FodderLineEnd.
type FodderElement ¶ added in v0.11.2
type FodderElement struct { Comment []string Kind FodderKind Blanks int Indent int }
FodderElement is a single piece of fodder.
func MakeFodderElement ¶ added in v0.11.2
func MakeFodderElement(kind FodderKind, blanks int, indent int, comment []string) FodderElement
MakeFodderElement is a helper function that checks some preconditions.
type FodderKind ¶ added in v0.11.2
type FodderKind int
FodderKind is an enum.
const ( // FodderLineEnd represents a line ending. // // It indicates that the next token, paragraph, or interstitial // should be on a new line. // // A single comment string is allowed, which flows before the new line. // // The LineEnd fodder specifies the indentation level and vertical spacing // before whatever comes next. FodderLineEnd FodderKind = iota // FodderInterstitial represents a comment in middle of a line. // // They must be /* C-style */ comments. // // If it follows a token (i.e., it is the first fodder element) then it // appears after the token on the same line. If it follows another // interstitial, it will also flow after it on the same line. If it follows // a new line or a paragraph, it is the first thing on the following line, // after the blank lines and indentation specified by the previous fodder. // // There is exactly one comment string. FodderInterstitial // FodderParagraph represents a comment consisting of at least one line. // // // and # style comments have exactly one line. C-style comments can have // more than one line. // // All lines of the comment are indented according to the indentation level // of the previous new line / paragraph fodder. // // The Paragraph fodder specifies the indentation level and vertical spacing // before whatever comes next. FodderParagraph )
type ForSpec ¶
type ForSpec struct { ForFodder Fodder VarFodder Fodder Conditions []IfSpec Outer *ForSpec Expr Node VarName Identifier InFodder Fodder }
ForSpec represents a for-specification in a comprehension. Example: expr for x in arr1 for y in arr2 for z in arr3 The order is the same as in python, i.e. the leftmost is the outermost.
Our internal representation reflects how they are semantically nested: ForSpec(z, outer=ForSpec(y, outer=ForSpec(x, outer=nil))) Any ifspecs are attached to the relevant ForSpec.
Ifs are attached to the one on the left, for example: expr for x in arr1 for y in arr2 if x % 2 == 0 for z in arr3 The if is attached to the y forspec.
It desugares to:
flatMap(\x -> flatMap(\y -> flatMap(\z -> [expr], arr3) arr2) arr3)
type Function ¶
type Function struct { ParenLeftFodder Fodder ParenRightFodder Fodder Body Node Parameters []Parameter NodeBase // Always false if there were no parameters. TrailingComma bool }
Function represents a function definition
type IdentifierSet ¶
type IdentifierSet map[Identifier]struct{}
IdentifierSet represents an Identifier set.
func NewIdentifierSet ¶
func NewIdentifierSet(idents ...Identifier) IdentifierSet
NewIdentifierSet creates a new IdentifierSet.
func (IdentifierSet) Add ¶
func (set IdentifierSet) Add(ident Identifier) bool
Add adds an Identifier to the set.
func (IdentifierSet) AddIdentifiers ¶
func (set IdentifierSet) AddIdentifiers(idents Identifiers)
AddIdentifiers adds a slice of identifiers to the set.
func (IdentifierSet) Clone ¶
func (set IdentifierSet) Clone() IdentifierSet
Clone returns a clone of the set.
func (IdentifierSet) Contains ¶
func (set IdentifierSet) Contains(ident Identifier) bool
Contains returns true if an Identifier is in the set.
func (IdentifierSet) Remove ¶
func (set IdentifierSet) Remove(ident Identifier)
Remove removes an Identifier from the set.
func (IdentifierSet) ToOrderedSlice ¶
func (set IdentifierSet) ToOrderedSlice() []Identifier
ToOrderedSlice returns the elements of the current set as an ordered slice.
func (IdentifierSet) ToSlice ¶
func (set IdentifierSet) ToSlice() Identifiers
ToSlice returns an Identifiers slice from the set.
type ImportBin ¶ added in v0.19.0
type ImportBin struct { File *LiteralString NodeBase }
ImportBin represents importbin "file".
type ImportStr ¶
type ImportStr struct { File *LiteralString NodeBase }
ImportStr represents importstr "file".
type Index ¶
type Index struct { Target Node Index Node // When Index is being used, this is the fodder before the ']'. // When Id is being used, this is the fodder before the id. RightBracketFodder Fodder // When Index is being used, this is the fodder before the '['. // When Id is being used, this is the fodder before the '.'. LeftBracketFodder Fodder //nolint: golint,stylecheck // keeping Id instead of ID for now to avoid breaking 3rd parties Id *Identifier NodeBase }
Index represents both e[e] and the syntax sugar e.f.
One of index and id will be nil before desugaring. After desugaring id will be nil.
type LiteralBoolean ¶
LiteralBoolean represents true and false
type LiteralNumber ¶
LiteralNumber represents a JSON number
type LiteralString ¶
type LiteralString struct { Value string BlockIndent string BlockTermIndent string NodeBase Kind LiteralStringKind }
LiteralString represents a JSON string
type LiteralStringKind ¶
type LiteralStringKind int
LiteralStringKind represents the kind of a literal string.
const ( StringSingle LiteralStringKind = iota StringDouble StringBlock VerbatimStringDouble VerbatimStringSingle )
Literal string kinds
func (LiteralStringKind) FullyEscaped ¶
func (k LiteralStringKind) FullyEscaped() bool
FullyEscaped returns true iff the literal string kind may contain escape sequences that require unescaping.
type Local ¶
type Local struct { Binds LocalBinds Body Node NodeBase }
Local represents local x = e; e. After desugaring, functionSugar is false.
type LocalBind ¶
type LocalBind struct { VarFodder Fodder // If Fun is set then its body == Body. Body Node EqFodder Fodder Variable Identifier // The fodder before the closing ',' or ';' (whichever it is) CloseFodder Fodder // There is no base fodder in Fun because there was no `function` keyword. Fun *Function LocRange LocationRange }
LocalBind is a helper struct for astLocal
type Location ¶
type Location struct { Line int // Column is a byte offset from the beginning of the line Column int }
Location represents a single location in an (unspecified) file.
type LocationRange ¶
type LocationRange struct { File *Source // FileName should be the imported path or "" for snippets etc. FileName string Begin Location End Location // TODO(sbarzowski) inclusive? exclusive? a gap? }
LocationRange represents a range of a source file.
func LineBeginning ¶
func LineBeginning(loc *LocationRange) LocationRange
LineBeginning returns the part of a line directly before LocationRange for example:
local x = foo() ^^^^^ <- LocationRange loc then local x = foo() ^^^^^^^^^^ <- lineBeginning(loc)
func LineEnding ¶
func LineEnding(loc *LocationRange) LocationRange
LineEnding returns the part of a line directly after LocationRange for example:
local x = foo() + test ^^^^^ <- LocationRange loc then local x = foo() + test ^^^^^^^ <- lineEnding(loc)
func LocationRangeBetween ¶
func LocationRangeBetween(a, b *LocationRange) LocationRange
LocationRangeBetween returns a LocationRange containing both a and b.
func MakeLocationRange ¶
func MakeLocationRange(fn string, fc *Source, begin Location, end Location) LocationRange
MakeLocationRange creates a LocationRange.
func MakeLocationRangeMessage ¶
func MakeLocationRangeMessage(msg string) LocationRange
MakeLocationRangeMessage creates a pseudo-LocationRange with a message but no location information. This is useful for special locations, e.g. manifestation entry point.
func (*LocationRange) IsSet ¶
func (lr *LocationRange) IsSet() bool
IsSet returns if this LocationRange has been set.
func (*LocationRange) String ¶
func (lr *LocationRange) String() string
func (*LocationRange) WithCode ¶
func (lr *LocationRange) WithCode() bool
WithCode returns true iff the LocationRange is linked to code. TODO: This is identical to lr.IsSet(). Is it required at all?
type NamedArgument ¶
type NamedArgument struct { NameFodder Fodder Name Identifier EqFodder Fodder Arg Node CommaFodder Fodder }
NamedArgument represents a named argument to function call x=1.
type Node ¶
type Node interface { Context() Context Loc() *LocationRange FreeVariables() Identifiers SetFreeVariables(Identifiers) SetContext(Context) // OpenFodder returns the fodder before the first token of an AST node. // Since every AST node has opening fodder, it is defined here. // If the AST node is left recursive (e.g. BinaryOp) then it is ambiguous // where the fodder should be stored. This is resolved by storing it as // far inside the tree as possible. OpenFodder returns a pointer to allow // the caller to modify the fodder. OpenFodder() *Fodder }
Node represents a node in the AST.
type NodeBase ¶
type NodeBase struct { // This is the fodder that precedes the first token of the node. // If the node is left-recursive, i.e. the first token is actually // a token of a sub-expression, then Fodder is nil. Fodder Fodder Ctx Context FreeVars Identifiers LocRange LocationRange }
NodeBase holds fields common to all node types.
func NewNodeBase ¶
func NewNodeBase(loc LocationRange, fodder Fodder, freeVariables Identifiers) NodeBase
NewNodeBase creates a new NodeBase from initial LocationRange and Identifiers.
func NewNodeBaseLoc ¶
func NewNodeBaseLoc(loc LocationRange, fodder Fodder) NodeBase
NewNodeBaseLoc creates a new NodeBase from an initial LocationRange.
func (*NodeBase) FreeVariables ¶
func (n *NodeBase) FreeVariables() Identifiers
FreeVariables returns a NodeBase's freeVariables.
func (*NodeBase) OpenFodder ¶ added in v0.15.0
OpenFodder returns a NodeBase's opening fodder.
func (*NodeBase) SetContext ¶
SetContext sets a NodeBase's context.
func (*NodeBase) SetFreeVariables ¶
func (n *NodeBase) SetFreeVariables(idents Identifiers)
SetFreeVariables sets a NodeBase's freeVariables.
type Object ¶
type Object struct { Fields ObjectFields CloseFodder Fodder NodeBase TrailingComma bool }
Object represents object constructors { f: e ... }.
The trailing comma is only allowed if len(fields) > 0. Converted to DesugaredObject during desugaring.
type ObjectComp ¶
type ObjectComp struct { Fields ObjectFields TrailingCommaFodder Fodder CloseFodder Fodder Spec ForSpec NodeBase TrailingComma bool }
ObjectComp represents object comprehension
{ [e]: e for x in e for.. if... }.
type ObjectField ¶
type ObjectField struct { // f(x, y, z): ... (ignore if kind == astObjectAssert) // If Method is set then Expr2 == Method.Body. // There is no base fodder in Method because there was no `function` // keyword. Method *Function //nolint: golint,stylecheck // keeping Id instead of ID for now to avoid breaking 3rd parties Id *Identifier Fodder2 Fodder Fodder1 Fodder OpFodder Fodder CommaFodder Fodder Expr1 Node // Not in scope of the object Expr2, Expr3 Node // In scope of the object (can see self). LocRange LocationRange Kind ObjectFieldKind Hide ObjectFieldHide // (ignore if kind != astObjectFieldID/Expr/Str) SuperSugar bool // +: (ignore if kind != astObjectFieldID/Expr/Str) }
ObjectField represents a field of an object or object comprehension. TODO(sbarzowski) consider having separate types for various kinds
func ObjectFieldLocalNoMethod ¶
func ObjectFieldLocalNoMethod(id *Identifier, body Node, loc LocationRange) ObjectField
ObjectFieldLocalNoMethod creates a non-method local object field.
type ObjectFieldHide ¶
type ObjectFieldHide int
ObjectFieldHide represents the visibility of an object field.
const ( ObjectFieldHidden ObjectFieldHide = iota // f:: e ObjectFieldInherit // f: e ObjectFieldVisible // f::: e )
Object field visibilities
type ObjectFieldKind ¶
type ObjectFieldKind int
ObjectFieldKind represents the kind of an object field.
const ( // <f1> 'assert' <expr2> '[' <opF> ':' <expr3> ']' <commaF> // where expr3 can be nil ObjectAssert ObjectFieldKind = iota // <f1> <id> <colon> <expr2> <commaF> ObjectFieldID // <f1> '[' <expr1> <f2> ']' <colon> <expr2> <commaF> ObjectFieldExpr // <expr1> <colon> <expr2> <commaF> ObjectFieldStr // <f1> 'local' <f2> <id> '=' <expr2> <commaF> ObjectLocal )
Kinds of object fields
type Parameter ¶ added in v0.16.0
type Parameter struct { NameFodder Fodder Name Identifier CommaFodder Fodder EqFodder Fodder DefaultArg Node LocRange LocationRange }
Parameter represents a parameter of function. If DefaultArg is set, it's an optional named parameter. Otherwise, it's a positional parameter and EqFodder is not used.
type Slice ¶
type Slice struct { Target Node LeftBracketFodder Fodder // Each of these can be nil BeginIndex Node EndColonFodder Fodder EndIndex Node StepColonFodder Fodder Step Node RightBracketFodder Fodder NodeBase }
Slice represents an array slice a[begin:end:step].
type Source ¶
type Source struct { // DiagnosticFileName is the imported path or a special string // for indicating stdin, extvars and other non-imported sources. DiagnosticFileName DiagnosticFileName Lines []string }
Source represents a source file.
func BuildSource ¶
func BuildSource(dFilename DiagnosticFileName, s string) *Source
BuildSource transforms a source file string into a Source struct. TODO: This seems like a job for strings.Split() with a final \n touch-up.
type SourceProvider ¶
type SourceProvider struct { }
SourceProvider represents a source provider. TODO: Need an explanation of why this exists.
func (*SourceProvider) GetSnippet ¶
func (sp *SourceProvider) GetSnippet(loc LocationRange) string
GetSnippet returns a code snippet corresponding to loc.
type SuperIndex ¶
type SuperIndex struct { // If super.f, the fodder before the 'f' // If super[e], the fodder before the ']'. IDFodder Fodder Index Node // If super.f, the fodder before the '.' // If super[e], the fodder before the '['. DotFodder Fodder //nolint: golint,stylecheck // keeping Id instead of ID for now to avoid breaking 3rd parties Id *Identifier NodeBase }
SuperIndex represents the super[e] and super.f constructs.
Either index or identifier will be set before desugaring. After desugaring, id will be nil.
type Var ¶
type Var struct { //nolint: golint,stylecheck // keeping Id instead of ID for now to avoid breaking 3rd parties Id Identifier NodeBase }
Var represents variables.