Documentation
¶
Overview ¶
Package symtable ports CPython's Python/symtable.c into Go.
Two passes drive the package: a block-visit pass over the AST that records raw def/use facts on per-scope Entry objects, and an analysis pass that resolves each name to one of Local / GlobalExplicit / GlobalImplicit / Free / Cell.
CPython: Python/symtable.c, Include/internal/pycore_symtable.h
Index ¶
Constants ¶
const DefBound = DefLocal | DefParam | DefImport
DefBound is the CPython mask for "name is bound in this block".
CPython: Include/internal/pycore_symtable.h:L170
const ScopeOffset = 12
ScopeOffset is the bit position of the resolved Scope inside SymbolFlags. CPython packs the analyzer output into bits [12,16).
CPython: Include/internal/pycore_symtable.h:L176
Variables ¶
This section is empty.
Functions ¶
func Mangle ¶
Mangle implements PEP 8 private name mangling: identifiers starting with two underscores (and not ending with two underscores) inside a class body get rewritten to "_<ClassName><name>". The class name's leading underscores are stripped before splicing.
`private` is the name of the lexically enclosing class, or "" when no class is in scope. With `private` empty, or when the rules below disqualify mangling, the original name is returned unchanged.
Specific rules ported from _Py_Mangle:
- the identifier must start with "__"
- identifiers ending with "__" are left alone (e.g. "__init__")
- identifiers containing "." are left alone (the qualifier in a dotted import is not a Python name)
- if `private` is all underscores ("_", "__", ...), no mangling
CPython: Python/symtable.c:L3207 _Py_Mangle
func MaybeMangle ¶
MaybeMangle is the variant the visit pass calls. Most blocks mangle every name (`Mangle`); a class scope's enclosed type-parameters block carries a MangledNames allow-list and only mangles the names in that set. Mirrors the PEP 695 rule that lets a type parameter with a private name be referenced as the unmangled identifier outside the class body.
CPython: Python/symtable.c:L3187 _Py_MaybeMangle
Types ¶
type Block ¶
type Block int
Block names the kind of namespace an Entry represents. The numeric order tracks pycore_symtable.h _Py_block_ty so that ports of error messages and dispatch tables can mirror the C source line by line.
CPython: Include/internal/pycore_symtable.h:L13 _Py_block_ty
const ( // FunctionBlock is a def or lambda body. FunctionBlock Block = iota // ClassBlock is a class body. ClassBlock // ModuleBlock is the top-level module body. ModuleBlock // AnnotationBlock is the implicit scope around an annotation // expression for PEP 649 deferred evaluation. Inert under PEP 563. AnnotationBlock // TypeAliasBlock is the implicit scope around `type X = ...`. TypeAliasBlock // TypeParametersBlock is the implicit scope around `[T, U]` on // a generic def or class. TypeParametersBlock // TypeVariableBlock is the implicit scope around the bound, // constraint, or default of a single TypeVar / ParamSpec / // TypeVarTuple. TypeVariableBlock )
type ComprehensionType ¶
type ComprehensionType int
ComprehensionType marks which comprehension flavor (if any) a FunctionBlock encodes.
CPython: Include/internal/pycore_symtable.h:L38 _Py_comprehension_ty
const ( // NoComprehension is the default for non-comprehension blocks. NoComprehension ComprehensionType = iota // ListComprehension marks a `[expr for ...]` body block. ListComprehension // DictComprehension marks a `{k: v for ...}` body block. DictComprehension // SetComprehension marks a `{expr for ...}` body block. SetComprehension // GeneratorExpression marks a `(expr for ...)` body block. GeneratorExpression )
type Directive ¶
Directive records the position of a `global` / `nonlocal` statement for later "x is used prior to global declaration" error reporting.
CPython: Python/symtable.c:L1791 Py_BuildValue("(Niiii)", ...)
type Entry ¶
type Entry struct {
// ID is the AST node pointer used as the dict key in
// st_blocks. Go uses the AST node value directly; this field
// preserves the ordering for diagnostics.
ID int
// Type is the kind of namespace this entry represents.
Type Block
// Name is the block name. Module is "top"; functions and
// classes use the def / class statement name; comprehensions
// use "<listcomp>" / "<setcomp>" / "<dictcomp>" / "<genexpr>";
// lambdas use "<lambda>".
Name string
// Symbols maps each name appearing in the block to its packed
// SymbolFlags. The visit pass writes only DEF_* / USE bits;
// the analyze pass ORs in the resolved Scope at bit 12.
Symbols map[string]SymbolFlags
// Varnames records function parameter names in declaration
// order. Used by the codegen to lay out fastlocals.
Varnames []string
// Children is the ordered list of nested blocks discovered by
// the visit pass. The analyze pass drops inlined comprehension
// children and splices their grandchildren into our list.
Children []*Entry
// Directives stores (name, location) tuples for `global` and
// `nonlocal` statements; consulted to point error messages at
// the right line.
Directives []Directive
// MangledNames is the set of identifiers (currently only
// PEP 695 type parameter names inside a class scope) that
// receive private name mangling on lookup.
MangledNames map[string]struct{}
// ScopeInfo is set during visits inside type-variable blocks;
// reproduces ste_scope_info ("a TypeVar default", etc).
ScopeInfo string
// Nested is true when this block is lexically inside a
// FunctionBlock-like ancestor (function / annotation / type
// alias / type params / type variable). Used to decide whether
// names may resolve to Free.
Nested bool
// Generator marks a function block that contains a `yield` or
// `yield from`.
Generator bool
// Coroutine marks an async def, or a module under
// PyCF_ALLOW_TOP_LEVEL_AWAIT, that contains an `await`.
Coroutine bool
// AnnotationsUsed is true if any annotation appears in this
// block. Consulted by the compiler to emit __annotations__
// scaffolding at class scope.
AnnotationsUsed bool
// Comprehension tags this block as the body of a list, set,
// dict, or generator comprehension. NoComprehension otherwise.
Comprehension ComprehensionType
// Varargs is true for functions declared with `*args`.
Varargs bool
// Varkeywords is true for functions declared with `**kwargs`.
Varkeywords bool
// ReturnsValue is true when `return expr` (with a value)
// appears in this function block.
ReturnsValue bool
// NeedsClassClosure is set on class scopes that observe a
// `__class__` reference from a method.
NeedsClassClosure bool
// NeedsClassDict is set on class scopes that observe a
// `__classdict__` reference from a method body.
NeedsClassDict bool
// CompInlined marks a comprehension entry whose body has been
// inlined into the parent scope by the analyze pass.
CompInlined bool
// CompIterTarget is set transiently while visiting a
// comprehension target so addDef can stamp DefCompIter.
CompIterTarget bool
// CanSeeClassScope is set on annotation, type alias, and type
// params blocks immediately enclosed by a class so name
// resolution can mediate through that class.
CanSeeClassScope bool
// HasDocstring is true for module / function / class blocks
// whose body opens with a string literal.
HasDocstring bool
// Method is true for FunctionBlocks defined directly inside a
// ClassBlock. Drives the codegen choice of LOAD_METHOD.
Method bool
// HasConditionalAnnotations is set on blocks where a
// conditionally-executed annotation appeared (PEP 649).
HasConditionalAnnotations bool
// InConditionalBlock is a transient flag set by the visit pass
// while inside a conditional annotation context.
InConditionalBlock bool
// InUnevaluatedAnnotation is set transiently while visiting an
// annotation that will not be evaluated under PEP 563.
InUnevaluatedAnnotation bool
// CompIterExpr counts how many comprehension iterable
// expressions enclose the current visit point. CPython tracks
// this so it can reject walrus inside a comprehension's
// outermost iterable.
CompIterExpr int
// AnnotationBlock is the lazy AnnotationBlock attached to a
// function or class for PEP 649 deferred annotation
// evaluation. Nil when the block carries no annotations.
AnnotationBlock *Entry
// Loc is the source location of the def / class / comprehension
// that opened this block.
Loc ast.Pos
}
Entry is the per-block symbol table state. It mirrors PySTEntryObject field-for-field so that algorithms ported from symtable.c read line-for-line. The visit pass populates Symbols / Varnames / Children plus the boolean attribute flags; the analyze pass rewrites Symbols to embed each name's resolved Scope.
CPython: Include/internal/pycore_symtable.h:L88 PySTEntryObject
func (*Entry) GetScope ¶
GetScope returns the resolved Scope for name, or zero if the analyze pass has not stamped one (e.g. unknown name).
CPython: Python/symtable.c:L555 _PyST_GetScope
func (*Entry) GetSymbol ¶
func (e *Entry) GetSymbol(name string) SymbolFlags
GetSymbol returns the packed SymbolFlags for name, or zero if name is not recorded in this block.
CPython: Python/symtable.c:L535 _PyST_GetSymbol
func (*Entry) IsFunctionLike ¶
IsFunctionLike reports whether the block participates in name binding the way a function does (its locals can be cell variables).
CPython: Python/symtable.c:L566 _PyST_IsFunctionLike
type Scope ¶
type Scope int
Scope is the resolved disposition of a name inside a block. Values match the CPython numeric constants so that bit-packed symbol flags stay byte-equal across a port boundary.
CPython: Include/internal/pycore_symtable.h:L180
const ( // Local is a name bound in this block by assignment / parameter / // import. Local Scope = 1 // GlobalExplicit is a name declared by a `global` statement. GlobalExplicit Scope = 2 // GlobalImplicit is a name treated as global because no binding // covers it. GlobalImplicit Scope = 3 // Free is a name bound in some enclosing function block. Free Scope = 4 // Cell is a local that is captured by an enclosed block as a Free // variable. Cell Scope = 5 )
type SymbolFlags ¶
type SymbolFlags uint32
SymbolFlags packs CPython's per-name DEF_* / USE bits in the low 12 bits and the resolved Scope in bits [12,16). The numeric layout is fixed by Include/internal/pycore_symtable.h: bit-for-bit compatibility lets a future bytecode marshaller round-trip flags against a CPython produced .pyc.
CPython: Include/internal/pycore_symtable.h:L156
const ( // DefGlobal records a `global` statement covering this name. DefGlobal SymbolFlags = 1 // DefLocal records an assignment to this name in the block. DefLocal SymbolFlags = 2 // DefParam records this name as a function parameter. DefParam SymbolFlags = 4 // DefNonlocal records a `nonlocal` statement covering this name. DefNonlocal SymbolFlags = 8 // Use records that this name is read in the block. Use SymbolFlags = 0x10 // DefFreeClass marks a free variable that aliases a class-scope // name; consulted to emit DEREF / CLASS_DEREF correctly. DefFreeClass SymbolFlags = 0x40 // DefImport marks a binding produced by `import`. DefImport SymbolFlags = 0x80 // DefAnnot marks that the name carries an annotation in this // block. Only used to flag class-level annotations into the // __annotations__ dict. DefAnnot SymbolFlags = 0x100 // DefCompIter marks a comprehension iteration variable; used to // enforce the walrus-no-rebind rule. DefCompIter SymbolFlags = 0x200 // DefTypeParam marks a PEP 695 type parameter. DefTypeParam SymbolFlags = 0x400 // DefCompCell marks a cell variable in an inlined comprehension. DefCompCell SymbolFlags = 0x800 )
DEF_* and USE bits.
CPython: Include/internal/pycore_symtable.h:L158
const ScopeMask SymbolFlags = DefGlobal | DefLocal | DefParam | DefNonlocal
ScopeMask carves the four-bit slot reserved for Scope.
CPython: Include/internal/pycore_symtable.h:L177
func (SymbolFlags) Defs ¶
func (f SymbolFlags) Defs() SymbolFlags
Defs returns just the DEF_* / USE bits, stripping the packed Scope.
CPython: Include/internal/pycore_symtable.h DEF_BOUND mask
func (SymbolFlags) Scope ¶
func (f SymbolFlags) Scope() Scope
Scope returns the resolved Scope packed into f, or zero if the analyze pass has not yet stamped a value.
CPython: Include/internal/pycore_symtable.h SCOPE_MASK extract
type SyntaxError ¶
SyntaxError is the build-time error raised by the symtable visit and analyze passes. The Msg shape mirrors CPython's SyntaxError strings line-for-line so existing test suites and IDE diagnostics match. The future package follows the same pattern.
CPython: Python/symtable.c uses PyErr_Format/PyErr_SetString with PyExc_SyntaxError plus PyErr_RangedSyntaxLocationObject.
func (*SyntaxError) Error ¶
func (e *SyntaxError) Error() string
Error implements the error interface.
CPython: Python/symtable.c PyErr_Format(PyExc_SyntaxError, ...) message body
type Table ¶
type Table struct {
// Filename is the source file name, used to address syntax errors.
Filename string
// Top is the module-level Entry. It is also reachable via
// Blocks[<the Module node>].
Top *Entry
// Blocks maps each AST node that opens a block to the Entry
// produced for it. CPython keys this dict by the C void*
// address of the AST node; we key by the Go AST node pointer
// (held inside an interface), which is just as unique within
// a single Build call.
Blocks map[any]*Entry
// Future captures the module's __future__ feature flags. The
// build pass consults Future.Bits & Annotations to skip
// AnnotationBlock evaluation under PEP 563.
Future *future.Features
}
Table holds the per-module symbol table built by Build. It mirrors `struct symtable` in CPython: a top-level module Entry plus a flat map of every block's Entry keyed by the AST node that introduced it.
CPython: Include/internal/pycore_symtable.h:L71 struct symtable
Source Files