Documentation
¶
Overview ¶
Package ast holds the Python AST data model. Mirrors the asdl-driven node tree from cpython/Parser/Python.asdl plus the helpers in cpython/Python/asdl.c and cpython/Include/internal/pycore_asdl.h.
The asdl source-of-truth is `cpython/Parser/Python.asdl`; the generated node structs live in nodes_gen.go (produced by tools/asdl_go from that .asdl file). This file ports the runtime helpers.
Index ¶
- Constants
- Variables
- func CopyLocation(newNode, oldNode any) any
- func Dump(node any) string
- func Equal(a, b any, compareAttributes bool) bool
- func FixMissingLocations(node any) any
- func GetDocstring(node any, clean bool) (string, bool)
- func GetSourceSegment(source string, node any, padded bool) (string, bool)
- func IncrementLineno(node any, n int) any
- func IsDocString(s Stmt) bool
- func IterChildNodes(node any) []any
- func LiteralEval(node any) (any, error)
- func Unparse(e Expr) (string, error)
- func Validate(mod Mod) error
- func Walk(node any) []any
- type Alias
- type AnnAssign
- type Arg
- type Arguments
- type Assert
- type Assign
- type AsyncFor
- type AsyncFunctionDef
- type AsyncWith
- type Attribute
- type AugAssign
- type Await
- type BinOp
- type BoolOp
- type Boolop
- type Break
- type Call
- type ClassDef
- type Cmpop
- type Compare
- type Comprehension
- type Constant
- type Continue
- type Delete
- type Dict
- type DictComp
- type EllipsisType
- type ExceptHandler
- type Excepthandler
- type Expr
- type ExprContext
- type ExprStmt
- type Expression
- type Field
- type For
- type FormattedValue
- type FrozenSet
- type FunctionDef
- type FunctionType
- type GeneratorExp
- type Global
- type If
- type IfExp
- type Import
- type ImportFrom
- type Interactive
- type Interpolation
- type JoinedStr
- type Keyword
- type Lambda
- type List
- type ListComp
- type Match
- type MatchAs
- type MatchCase
- type MatchClass
- type MatchMapping
- type MatchOr
- type MatchSequence
- type MatchSingleton
- type MatchStar
- type MatchValue
- type Mod
- type Module
- type Name
- type NamedExpr
- type NodeTransformer
- type NodeVisitor
- type Nonlocal
- type Operator
- type ParamSpec
- type Pass
- type Pattern
- type Pos
- type PreprocessOptions
- type Raise
- type Return
- type Seq
- type Set
- type SetComp
- type Slice
- type Starred
- type Stmt
- type Subscript
- type TemplateStr
- type Try
- type TryStar
- type Tuple
- type TypeAlias
- type TypeIgnore
- type TypeIgnoreNode
- type TypeParam
- type TypeVar
- type TypeVarTuple
- type UnaryOp
- type Unaryop
- type VisitFunc
- type Warning
- type While
- type With
- type Withitem
- type Yield
- type YieldFrom
Constants ¶
const CoFutureAnnotations uint32 = 0x1000000
CoFutureAnnotations is the CO_FUTURE_ANNOTATIONS bit (PEP 563). Mirrors the value used by future.Annotations so the preprocess pass can be told to skip annotation expressions without taking a future-package dependency (which would import this package transitively).
CPython: Include/cpython/code.h CO_FUTURE_ANNOTATIONS
Variables ¶
var Ellipsis = EllipsisType{}
Ellipsis is the canonical sentinel for `...` as a Constant value.
var NoPos = Pos{-1, -1, -1, -1}
NoPos is the sentinel CPython spells (_Py_SourceLocation){-1,-1,-1,-1}.
Functions ¶
func CopyLocation ¶ added in v0.10.2
CopyLocation ports cpython/Lib/ast.py:193 copy_location. Copies the source-position quadruple from oldNode to newNode and returns newNode. If either node lacks a Pos field, returns newNode unchanged.
func Dump ¶ added in v0.10.2
Dump returns a textual representation of node matching CPython's ast.dump(node) with the documented defaults.
func Equal ¶ added in v0.10.2
Equal ports cpython/Lib/ast.py:400 compare. Returns true iff a and b are the same AST type and every field is recursively equal. When compareAttributes is true, the source-position quadruple (lineno, col_offset, end_lineno, end_col_offset) must also match.
CPython names this function `ast.compare`; gopy uses `Equal` because `Compare` is taken by the asdl Compare expr-node (`a < b`).
func FixMissingLocations ¶ added in v0.10.2
FixMissingLocations ports cpython/Lib/ast.py:210 fix_missing_locations. Walks the tree and fills any zero-valued position field on a node from the nearest ancestor that has a non-zero value. Returns node.
func GetDocstring ¶ added in v0.10.2
GetDocstring ports cpython/Lib/ast.py:294 get_docstring. Returns the docstring of node (Module/FunctionDef/AsyncFunctionDef/ClassDef) and ok=true; ok=false signals "no docstring" (empty body, first stmt is not a string-literal Expr, or node is not a docstring-bearing type). When clean is true, the result is post-processed by cleandoc to strip uniform leading whitespace.
func GetSourceSegment ¶ added in v0.10.2
GetSourceSegment ports cpython/Lib/ast.py:349 get_source_segment. Returns the source code segment that produced node, or ok=false if any required position attribute (lineno, end_lineno, col_offset, end_col_offset) is missing. With padded=true, the first line of a multi-line statement is padded with spaces to its original column.
func IncrementLineno ¶ added in v0.10.2
IncrementLineno ports cpython/Lib/ast.py:245 increment_lineno. Adds n to lineno and end_lineno on every node in the subtree, leaving columns alone. Returns node.
func IsDocString ¶
IsDocString reports whether stmt is the bare-string-literal form recognized by _PyAST_GetDocString. Mirrors the C predicate: an ExprStmt whose value is a string-typed Constant.
CPython: Python/ast.c _PyAST_GetDocString
func IterChildNodes ¶ added in v0.10.2
IterChildNodes ports cpython/Lib/ast.py:280 iter_child_nodes. Yields every direct child of node that is itself an AST node, plus every item of any list-valued field that is an AST node.
func LiteralEval ¶ added in v0.10.2
LiteralEval ports literal_eval. Strings are first parsed; passing a *Module or *Expression unwraps the body before evaluating.
func Unparse ¶
Unparse renders an expression back to source text. Mirrors the limited unparser in cpython/Python/ast_unparse.c which CPython uses to stringize PEP 563/649 annotations during compilation.
CPython: Python/ast_unparse.c expr_as_unicode
func Validate ¶
Validate is the gopy port of _PyAST_Validate. It walks mod and returns nil if the tree is well-formed, or an error matching CPython's ValueError/TypeError text.
This is the foundation pass: covers Module/Interactive/Expression/ FunctionType plus the node kinds emitted by future and other early pipeline stages (ImportFrom, ExprStmt, Constant). The full validator grows as the asdl-generated nodes land in nodes_gen.go.
CPython: Python/ast.c:L1047 _PyAST_Validate
Types ¶
type AnnAssign ¶
AnnAssign is the asdl `AnnAssign` node.
type Arguments ¶
type Arguments struct {
Posonlyargs Seq[*Arg]
Args Seq[*Arg]
Vararg *Arg
Kwonlyargs Seq[*Arg]
KwDefaults Seq[Expr]
Kwarg *Arg
Defaults Seq[Expr]
}
Arguments is the asdl `arguments` node.
type AsyncFor ¶
type AsyncFor struct {
Target Expr
Iter Expr
Body Seq[Stmt]
Orelse Seq[Stmt]
TypeComment *string
Pos Pos
}
AsyncFor is the asdl `AsyncFor` node.
type AsyncFunctionDef ¶
type AsyncFunctionDef struct {
Name string
Args *Arguments
Body Seq[Stmt]
DecoratorList Seq[Expr]
Returns Expr
TypeComment *string
TypeParams Seq[TypeParam]
Pos Pos
}
AsyncFunctionDef is the asdl `AsyncFunctionDef` node.
func (*AsyncFunctionDef) Position ¶
func (n *AsyncFunctionDef) Position() Pos
Position returns the source location.
func (*AsyncFunctionDef) SetPos ¶ added in v0.12.3
func (n *AsyncFunctionDef) SetPos(p Pos)
SetPos installs the source location.
type AsyncWith ¶
AsyncWith is the asdl `AsyncWith` node.
type Attribute ¶
type Attribute struct {
Value Expr
Attr string
Ctx ExprContext
Pos Pos
}
Attribute is the asdl `Attribute` node.
type AugAssign ¶
AugAssign is the asdl `AugAssign` node.
type ClassDef ¶
type ClassDef struct {
Name string
Bases Seq[Expr]
Keywords Seq[*Keyword]
Body Seq[Stmt]
DecoratorList Seq[Expr]
TypeParams Seq[TypeParam]
Pos Pos
}
ClassDef is the asdl `ClassDef` node.
type Comprehension ¶
Comprehension is the asdl `comprehension` node.
type DictComp ¶
type DictComp struct {
Key Expr
Value Expr
Generators Seq[*Comprehension]
Pos Pos
}
DictComp is the asdl `DictComp` node.
type EllipsisType ¶
type EllipsisType struct{}
EllipsisType is the singleton type used to spell Python's `...` as a Constant value. CPython uses the dedicated Py_Ellipsis singleton.
type ExceptHandler ¶
ExceptHandler is the asdl `ExceptHandler` node.
func (*ExceptHandler) Position ¶
func (n *ExceptHandler) Position() Pos
Position returns the source location.
func (*ExceptHandler) SetPos ¶ added in v0.12.3
func (n *ExceptHandler) SetPos(p Pos)
SetPos installs the source location.
type Excepthandler ¶
type Excepthandler interface {
Position() Pos
SetPos(Pos)
// contains filtered or unexported methods
}
Excepthandler is the asdl `excepthandler` sum.
type ExprContext ¶
type ExprContext int
ExprContext is the asdl `expr_context` enum.
const ( Load ExprContext = iota + 1 Store Del )
type Field ¶ added in v0.10.2
Field is a (name, value) pair as yielded by IterFields. The name matches the asdl identifier for the field (e.g. "context_expr", "type_comment"), not the Go struct field name.
func IterFields ¶ added in v0.10.2
IterFields ports cpython/Lib/ast.py:268 iter_fields. Yields one entry per declared field of node in source order, skipping the internal `Pos` source-position field (which CPython tracks under _attributes, not _fields).
type For ¶
type For struct {
Target Expr
Iter Expr
Body Seq[Stmt]
Orelse Seq[Stmt]
TypeComment *string
Pos Pos
}
For is the asdl `For` node.
type FormattedValue ¶
FormattedValue is the asdl `FormattedValue` node.
func (*FormattedValue) Position ¶
func (n *FormattedValue) Position() Pos
Position returns the source location.
func (*FormattedValue) SetPos ¶ added in v0.12.3
func (n *FormattedValue) SetPos(p Pos)
SetPos installs the source location.
type FrozenSet ¶
type FrozenSet []any
FrozenSet wraps a slice of constant items for a frozenset literal. CPython tracks frozenset as a real type; v0.5 only needs a marker so Validate can recurse.
type FunctionDef ¶
type FunctionDef struct {
Name string
Args *Arguments
Body Seq[Stmt]
DecoratorList Seq[Expr]
Returns Expr
TypeComment *string
TypeParams Seq[TypeParam]
Pos Pos
}
FunctionDef is the asdl `FunctionDef` node.
func (*FunctionDef) Position ¶
func (n *FunctionDef) Position() Pos
Position returns the source location.
func (*FunctionDef) SetPos ¶ added in v0.12.3
func (n *FunctionDef) SetPos(p Pos)
SetPos installs the source location.
type FunctionType ¶
FunctionType is the asdl `FunctionType` node.
type GeneratorExp ¶
type GeneratorExp struct {
Elt Expr
Generators Seq[*Comprehension]
Pos Pos
}
GeneratorExp is the asdl `GeneratorExp` node.
func (*GeneratorExp) Position ¶
func (n *GeneratorExp) Position() Pos
Position returns the source location.
func (*GeneratorExp) SetPos ¶ added in v0.12.3
func (n *GeneratorExp) SetPos(p Pos)
SetPos installs the source location.
type ImportFrom ¶
ImportFrom is the asdl `ImportFrom` node.
func (*ImportFrom) Position ¶
func (n *ImportFrom) Position() Pos
Position returns the source location.
func (*ImportFrom) SetPos ¶ added in v0.12.3
func (n *ImportFrom) SetPos(p Pos)
SetPos installs the source location.
type Interactive ¶
Interactive is the asdl `Interactive` node.
type Interpolation ¶
Interpolation is the asdl `Interpolation` node.
func (*Interpolation) Position ¶
func (n *Interpolation) Position() Pos
Position returns the source location.
func (*Interpolation) SetPos ¶ added in v0.12.3
func (n *Interpolation) SetPos(p Pos)
SetPos installs the source location.
type JoinedStr ¶
JoinedStr is the asdl `JoinedStr` node.
type List ¶
type List struct {
Elts Seq[Expr]
Ctx ExprContext
Pos Pos
}
List is the asdl `List` node.
type ListComp ¶
type ListComp struct {
Elt Expr
Generators Seq[*Comprehension]
Pos Pos
}
ListComp is the asdl `ListComp` node.
type MatchClass ¶
type MatchClass struct {
Cls Expr
Patterns Seq[Pattern]
KwdAttrs Seq[string]
KwdPatterns Seq[Pattern]
Pos Pos
}
MatchClass is the asdl `MatchClass` node.
func (*MatchClass) Position ¶
func (n *MatchClass) Position() Pos
Position returns the source location.
func (*MatchClass) SetPos ¶ added in v0.12.3
func (n *MatchClass) SetPos(p Pos)
SetPos installs the source location.
type MatchMapping ¶
MatchMapping is the asdl `MatchMapping` node.
func (*MatchMapping) Position ¶
func (n *MatchMapping) Position() Pos
Position returns the source location.
func (*MatchMapping) SetPos ¶ added in v0.12.3
func (n *MatchMapping) SetPos(p Pos)
SetPos installs the source location.
type MatchSequence ¶
MatchSequence is the asdl `MatchSequence` node.
func (*MatchSequence) Position ¶
func (n *MatchSequence) Position() Pos
Position returns the source location.
func (*MatchSequence) SetPos ¶ added in v0.12.3
func (n *MatchSequence) SetPos(p Pos)
SetPos installs the source location.
type MatchSingleton ¶
MatchSingleton is the asdl `MatchSingleton` node.
func (*MatchSingleton) Position ¶
func (n *MatchSingleton) Position() Pos
Position returns the source location.
func (*MatchSingleton) SetPos ¶ added in v0.12.3
func (n *MatchSingleton) SetPos(p Pos)
SetPos installs the source location.
type MatchStar ¶
MatchStar is the asdl `MatchStar` node.
type MatchValue ¶
MatchValue is the asdl `MatchValue` node.
func (*MatchValue) Position ¶
func (n *MatchValue) Position() Pos
Position returns the source location.
func (*MatchValue) SetPos ¶ added in v0.12.3
func (n *MatchValue) SetPos(p Pos)
SetPos installs the source location.
type Mod ¶
type Mod interface {
// contains filtered or unexported methods
}
Mod is the asdl `mod` sum.
type Module ¶
type Module struct {
Body Seq[Stmt]
TypeIgnores Seq[TypeIgnore]
}
Module is the asdl `Module` node.
type NamedExpr ¶
NamedExpr is the asdl `NamedExpr` node.
type NodeTransformer ¶ added in v0.10.2
NodeTransformer is the gopy port of cpython/Lib/ast.py:519 NodeTransformer. Handlers may return a replacement node, nil to delete a list element / clear an optional field, or the original node to leave it in place.
func (*NodeTransformer) GenericVisit ¶ added in v0.10.2
func (t *NodeTransformer) GenericVisit(node any) any
GenericVisit walks the fields of node and rewrites child entries according to handler return values. Mirrors generic_visit() at cpython/Lib/ast.py:555.
func (*NodeTransformer) Visit ¶ added in v0.10.2
func (t *NodeTransformer) Visit(node any) any
Visit dispatches like NodeVisitor.Visit but uses the transformer's GenericVisit (which mutates child slices and fields in place).
type NodeVisitor ¶ added in v0.10.2
NodeVisitor is the gopy port of cpython/Lib/ast.py:482 NodeVisitor. Walks the tree calling Handlers[ClassName](node) for each node, or GenericVisit if no handler is registered.
func (*NodeVisitor) GenericVisit ¶ added in v0.10.2
func (v *NodeVisitor) GenericVisit(node any) any
GenericVisit calls Visit on every direct child of node and returns nil. Mirrors generic_visit() at cpython/Lib/ast.py:508.
func (*NodeVisitor) Visit ¶ added in v0.10.2
func (v *NodeVisitor) Visit(node any) any
Visit dispatches node to the registered handler for its class name, or to GenericVisit if no handler is registered. Mirrors visit() at cpython/Lib/ast.py:502.
type ParamSpec ¶
ParamSpec is the asdl `ParamSpec` node.
type Pos ¶
Pos is the source position quadruple. CPython uses _Py_SourceLocation; lineno=-1 marks "no position".
CPython: Include/internal/pycore_location.h _Py_SourceLocation
type PreprocessOptions ¶
type PreprocessOptions struct {
Filename string
OptimizeLevel int // -1 default, 0 = no, 1 = -O, 2 = -OO.
FFFeatures uint32 // future feature bits (e.g. CO_FUTURE_ANNOTATIONS).
EnableWarnings bool // false under -W ignore.
SyntaxCheckOnly bool // true suppresses mutating folds.
}
PreprocessOptions controls which preprocess passes run. Mirrors the _PyASTPreprocessState flag bag.
CPython: Python/ast_preprocess.c _PyASTPreprocessState
type Seq ¶
type Seq[T any] []T
Seq is the asdl_seq* equivalent. CPython stores sequence length and elements inline in a heap struct allocated from the per-compile arena; in Go a typed slice is the same shape with an O(1) length field (cap/len) and arena-friendly bulk allocation through the underlying slice header.
CPython: Include/internal/pycore_asdl.h:L30 asdl_seq
func NewSeq ¶
NewSeq mirrors _Py_asdl_generic_seq_new (and the typed variants _Py_asdl_identifier_seq_new, _Py_asdl_int_seq_new). A zero-length sequence is allocated with an empty slice, not nil, so that Len() is 0 but the sequence itself is non-nil. CPython does the same: _Py_asdl_*_seq_new(0, arena) returns a valid asdl_seq* with size=0.
CPython: Python/asdl.c:L4 GENERATE_ASDL_SEQ_CONSTRUCTOR
func (Seq[T]) Get ¶
Get mirrors asdl_seq_GET. The bounds check matches CPython's debug-build check; release CPython elides it but a panic on Go slice access is the equivalent guarantee.
CPython: Include/internal/pycore_asdl.h:L82 asdl_seq_GET
type SetComp ¶
type SetComp struct {
Elt Expr
Generators Seq[*Comprehension]
Pos Pos
}
SetComp is the asdl `SetComp` node.
type Starred ¶
type Starred struct {
Value Expr
Ctx ExprContext
Pos Pos
}
Starred is the asdl `Starred` node.
type Subscript ¶
type Subscript struct {
Value Expr
Slice Expr
Ctx ExprContext
Pos Pos
}
Subscript is the asdl `Subscript` node.
type TemplateStr ¶
TemplateStr is the asdl `TemplateStr` node.
func (*TemplateStr) Position ¶
func (n *TemplateStr) Position() Pos
Position returns the source location.
func (*TemplateStr) SetPos ¶ added in v0.12.3
func (n *TemplateStr) SetPos(p Pos)
SetPos installs the source location.
type Try ¶
type Try struct {
Body Seq[Stmt]
Handlers Seq[Excepthandler]
Orelse Seq[Stmt]
Finalbody Seq[Stmt]
Pos Pos
}
Try is the asdl `Try` node.
type TryStar ¶
type TryStar struct {
Body Seq[Stmt]
Handlers Seq[Excepthandler]
Orelse Seq[Stmt]
Finalbody Seq[Stmt]
Pos Pos
}
TryStar is the asdl `TryStar` node.
type Tuple ¶
type Tuple struct {
Elts Seq[Expr]
Ctx ExprContext
Pos Pos
}
Tuple is the asdl `Tuple` node.
type TypeAlias ¶
TypeAlias is the asdl `TypeAlias` node.
type TypeIgnore ¶
type TypeIgnore interface {
// contains filtered or unexported methods
}
TypeIgnore is the asdl `type_ignore` sum.
type TypeIgnoreNode ¶
TypeIgnoreNode is the asdl `TypeIgnoreNode` node.
type TypeVarTuple ¶
TypeVarTuple is the asdl `TypeVarTuple` node.
func (*TypeVarTuple) Position ¶
func (n *TypeVarTuple) Position() Pos
Position returns the source location.
func (*TypeVarTuple) SetPos ¶ added in v0.12.3
func (n *TypeVarTuple) SetPos(p Pos)
SetPos installs the source location.
type VisitFunc ¶ added in v0.10.2
VisitFunc is one node-class handler registered with a NodeVisitor. Returning a non-nil node from a NodeTransformer handler replaces the node in the parent; returning nil deletes it.
type Warning ¶
Warning is a non-fatal preprocess diagnostic. PEP 765 emits one of these for each `return`/`break`/`continue` that escapes a `finally` block. CPython spells them via _PyErr_EmitSyntaxWarning.
CPython: Python/ast_preprocess.c control_flow_in_finally_warning
func Preprocess ¶
func Preprocess(mod Mod, opt PreprocessOptions) []Warning
Preprocess walks mod, emits PEP 765 control-flow-in-finally warnings, and applies the small set of constant folds CPython 3.14 keeps in the AST layer (printf-format folding, `__debug__` substitution, MatchValue numeric folding, docstring removal at -OO).
CPython: Python/ast_preprocess.c _PyAST_Preprocess