codeast

Documentation

Overview

Package codeast defines internal AST parsing abstractions shared by code-aware readers.

Index

• Constants
• Variables
• func IsExamplePath(filePath string, basePath string) bool
• func ParseConcurrency(opts []ParseOption) int
• func ParseIncludeFiles(opts []ParseOption) []string
• func RegisterDirectoryParser(fileType string, parser DirectoryParser)
• type DirectoryParser
  • func GetDirectoryParser(fileType string) (DirectoryParser, bool)
• type DocumentPayload
  • func NodeToDocumentPayload(node *Node, opts NodeDocumentPayloadOptions) *DocumentPayload
  • func NodesToDocumentPayloads(result *Result, opts NodeDocumentPayloadOptions) []*DocumentPayload
• type Edge
• type EntityType
• type FileInfo
• type Language
• type Node
• type NodeDocumentPayloadOptions
• type ParseOption
  • func WithParseConcurrency(n int) ParseOption
  • func WithParseIncludeFiles(files []string) ParseOption
• type RelationType
• type Result
• type Scope

Constants

const (
	FileTypeGo     = "go"
	FileTypeProto  = "proto"
	FileTypePython = "python"
)

FileType constants for directory parser registration. These mirror source.FileReaderType values and live here to avoid import cycles between codeast and source packages.

const (
	MetadataKeyCodeChunkIndex string = "code_chunk_index"
	MetadataKeyReceiverType   string = "receiver_type"
	MetadataKeyScope          string = "scope"
	MetadataKeyLanguage       string = "language"
)

MetadataKeyCodeChunkIndex stores the code chunk index in node metadata. MetadataKeyReceiverType stores the receiver type for method nodes. MetadataKeyScope stores the retrieval scope label. MetadataKeyLanguage stores the language label.

const TrpcAstMetaPrefix string = "trpc_ast_"

TrpcAstMetaPrefix is the prefix for all metadata keys written by AST readers.

Variables

var ErrParserUnavailable = errors.New("codeast: directory parser runtime unavailable")

ErrParserUnavailable indicates a directory parser is registered but its runtime dependency is missing or incompatible (e.g. the external interpreter it shells out to is not installed or too old). Callers that aggregate multiple languages may treat it as "skip this language" rather than failing the whole operation.

Functions

func IsExamplePath

func IsExamplePath(filePath string, basePath string) bool

IsExamplePath checks if a file path is under an example directory within a repository.

func ParseConcurrency

func ParseConcurrency(opts []ParseOption) int

ParseConcurrency resolves the concurrency value from the given options.

func ParseIncludeFiles

func ParseIncludeFiles(opts []ParseOption) []string

ParseIncludeFiles resolves the include-file list from the given options.

func RegisterDirectoryParser

func RegisterDirectoryParser(fileType string, parser DirectoryParser)

RegisterDirectoryParser registers a directory parser for the given file type. Last registration wins.

Types

type DirectoryParser

type DirectoryParser interface {
	ParseDirectory(dirPath string, opts ...ParseOption) (*Result, error)
}

DirectoryParser parses code under a directory into a code AST result.

func GetDirectoryParser

func GetDirectoryParser(fileType string) (DirectoryParser, bool)

GetDirectoryParser returns the registered directory parser for the given file type.

type DocumentPayload

type DocumentPayload struct {
	Name          string
	Content       string
	Metadata      map[string]any
	EmbeddingText string
}

DocumentPayload is a document-ready payload derived from AST parsing.

func NodeToDocumentPayload

func NodeToDocumentPayload(node *Node, opts NodeDocumentPayloadOptions) *DocumentPayload

NodeToDocumentPayload converts a single AST node into a document payload.

func NodesToDocumentPayloads

func NodesToDocumentPayloads(result *Result, opts NodeDocumentPayloadOptions) []*DocumentPayload

NodesToDocumentPayloads converts a parse result into document payloads.

type Edge

type Edge struct {
	FromID   string         `json:"from_id"`
	ToID     string         `json:"to_id"`
	Type     RelationType   `json:"type"`
	Metadata map[string]any `json:"metadata"`
}

Edge represents a directed relationship between two nodes.

type EntityType

type EntityType string

EntityType defines the type of code entity.

const (
	// EntityFunction represents a standalone function declaration.
	EntityFunction EntityType = "Function"
	// EntityMethod represents a method bound to a receiver type.
	EntityMethod EntityType = "Method"
	// EntityStruct represents a struct type declaration.
	EntityStruct EntityType = "Struct"
	// EntityInterface represents an interface type declaration.
	EntityInterface EntityType = "Interface"
	// EntityVariable represents a variable or constant declaration.
	EntityVariable EntityType = "Variable"
	// EntityAlias represents a type alias declaration.
	EntityAlias EntityType = "Alias"
	// EntityPackage represents a package-level grouping node.
	EntityPackage EntityType = "Package"
	// EntityClass represents a class declaration in class-based languages.
	EntityClass EntityType = "Class"
	// EntityModule represents a module declaration, such as in Python.
	EntityModule EntityType = "Module"
	// EntityNamespace represents a namespace declaration, such as in C++.
	EntityNamespace EntityType = "Namespace"
	// EntityTemplate represents a template declaration in C++.
	EntityTemplate EntityType = "Template"
	// EntityEnum represents an enum declaration.
	EntityEnum EntityType = "Enum"
	// EntityService represents a proto service declaration.
	EntityService EntityType = "Service"
	// EntityRPC represents a proto RPC declaration.
	EntityRPC EntityType = "RPC"
	// EntityMessage represents a proto message declaration.
	EntityMessage EntityType = "Message"
	// EntityDocument represents a non-code document node.
	EntityDocument EntityType = "Document"
)

type FileInfo

type FileInfo struct {
	Name     string
	Language Language
	Package  string
	Imports  []string
	Metadata map[string]any
}

FileInfo contains file-level parse metadata produced alongside AST results.

type Language

type Language string

Language defines the programming language.

const (
	// LanguageGo identifies Go source code.
	LanguageGo Language = "go"
	// LanguageCpp identifies C++ source code.
	LanguageCpp Language = "cpp"
	// LanguagePython identifies Python source code.
	LanguagePython Language = "python"
	// LanguageProto identifies Protocol Buffers source code.
	LanguageProto Language = "proto"
	// LanguageJavascript identifies JavaScript source code.
	LanguageJavascript Language = "javascript"
)

type Node

type Node struct {
	ID       string     `json:"id"`
	Type     EntityType `json:"type"`
	Name     string     `json:"name"`
	FullName string     `json:"full_name"`

	Scope    Scope    `json:"scope"`
	Language Language `json:"language"`

	Signature string `json:"signature"`
	Comment   string `json:"comment"`
	Code      string `json:"code"`

	FilePath   string `json:"file_path"`
	LineStart  int    `json:"line_start"`
	LineEnd    int    `json:"line_end"`
	ChunkIndex int    `json:"chunk_index"`

	RepoURL  string `json:"repo_url"`
	RepoName string `json:"repo_name"`
	Branch   string `json:"branch"`

	Package         string   `json:"package"`
	Namespace       string   `json:"namespace,omitempty"`
	UsingNamespaces []string `json:"using_namespaces,omitempty"`
	Imports         []string `json:"imports,omitempty"`

	Metadata map[string]any `json:"metadata"`

	Embedding     []float64 `json:"embedding,omitempty"`
	SparseIndices []int32   `json:"sparse_indices,omitempty"`
	SparseValues  []float32 `json:"sparse_values,omitempty"`
}

Node represents a code entity in the graph.

type NodeDocumentPayloadOptions

type NodeDocumentPayloadOptions struct {
	BaseMetadata       map[string]any
	ScopeBasePath      string
	FileInfo           *FileInfo
	FormatType         func(EntityType) string
	BuildEmbeddingText func(*Node) string
}

NodeDocumentPayloadOptions configures how AST nodes are mapped to document payloads.

type ParseOption

type ParseOption func(*parseOptions)

ParseOption configures a ParseDirectory call.

func WithParseConcurrency

func WithParseConcurrency(n int) ParseOption

WithParseConcurrency sets the parser concurrency. Zero or negative values mean use the parser's default.

func WithParseIncludeFiles

func WithParseIncludeFiles(files []string) ParseOption

WithParseIncludeFiles limits directory parsing to the given absolute or directory-relative files when the parser supports scoped loading.

type RelationType

type RelationType string

RelationType defines the relationship between entities.

const (
	// RelationCalls marks a call-site dependency between code entities.
	RelationCalls RelationType = "CALLS"
	// RelationMethod links a receiver type to one of its methods.
	RelationMethod RelationType = "METHOD"
	// RelationField links a composite type to one of its fields.
	RelationField RelationType = "FIELD"
	// RelationImplements marks that a type implements an interface.
	RelationImplements RelationType = "IMPLEMENTS"
	// RelationParam links a callable entity to one of its parameters.
	RelationParam RelationType = "PARAM"
	// RelationReturns links a callable entity to one of its return values.
	RelationReturns RelationType = "RETURNS"
	// RelationAliasOf links a type alias to its target type.
	RelationAliasOf RelationType = "ALIAS_OF"
	// RelationTyped links a declaration to its referenced type.
	RelationTyped RelationType = "TYPE"
	// RelationImports marks an import dependency between files or modules.
	RelationImports RelationType = "IMPORTS"
	// RelationInherits marks an inheritance relationship.
	RelationInherits RelationType = "INHERITS"
	// RelationContains marks a namespace-style containment relationship.
	RelationContains RelationType = "CONTAINS"
)

type Result

type Result struct {
	File  *FileInfo
	Nodes []*Node
	Edges []*Edge
}

Result contains the full parse result for a source file or code unit.

type Scope

type Scope string

Scope defines the search scope category.

const (
	// ScopeCode marks code entities intended for code-aware retrieval.
	ScopeCode Scope = "code"
	// ScopeExample marks example or tutorial-style content.
	ScopeExample Scope = "example"
)