lexical

Documentation

Overview

Package lexical extracts doc comments and bounded source snippets from already-discovered files, populating CodeNode properties used by the lexical intelligence layer.

Index

• Constants
• func Extract(lines []string, language string, lineStart int) string
• func InferLanguage(filePath string) string
• type CodeSnippet
• type Enricher
  • func NewEnricher() *Enricher
  • func (e *Enricher) Enrich(nodes []*model.CodeNode, root string)
• type FullTextStore
• type QueryService
  • func NewQueryService(store FullTextStore, snippets *SnippetStore, root string) *QueryService
  • func (q *QueryService) FindByConfigKey(query string, limit int) []Result
  • func (q *QueryService) FindByDocComment(query string, limit int) []Result
  • func (q *QueryService) FindByIdentifier(name string, limit int) []Result
• type Result
• type SnippetStore
  • func NewSnippetStore() *SnippetStore
  • func (s *SnippetStore) Extract(node *model.CodeNode, root string) (CodeSnippet, bool)
  • func (s *SnippetStore) ExtractWithContext(node *model.CodeNode, root string, ctx int) (CodeSnippet, bool)

Constants

const (
	KeyLexComment    = "lex_comment"
	KeyLexConfigKeys = "lex_config_keys"
)

Property keys stamped onto CodeNode.Properties; persisted to the graph as prop_lex_* via the prop_* round-trip convention and indexed by the lexical_index full-text index.

const (
	MaxSnippetLines     = 50
	DefaultContextLines = 3
)

Snippet sizing — matches SnippetStore.java.

Functions

func Extract

func Extract(lines []string, language string, lineStart int) string

Extract returns the doc comment for the symbol declared at lineStart (1-based) in the given source lines. The language string selects extraction style: "python" -> triple-quoted docstring, "go"/"rust" -> contiguous //, anything else -> block /** ... */ immediately preceding the declaration.

Mirrors src/main/java/.../intelligence/lexical/DocCommentExtractor.java.

func InferLanguage

func InferLanguage(filePath string) string

InferLanguage maps a file extension to a language identifier. Mirrors SnippetStore.inferLanguage on the Java side; returns "unknown" for unknown or missing extensions.

Types

type CodeSnippet

type CodeSnippet struct {
	Source    string
	FilePath  string
	LineStart int
	LineEnd   int
	Language  string
}

CodeSnippet is a bounded source extract for evidence packs / lexical results. Mirrors the Java CodeSnippet record (Provenance is intentionally omitted on the Go side until the intelligence/provenance port lands).

type Enricher

type Enricher struct{}

Enricher populates lexical metadata on CodeNodes prior to graph bulk-load. Mirrors LexicalEnricher.java.

func NewEnricher

func NewEnricher() *Enricher

NewEnricher returns a stateless lexical enricher.

func (*Enricher) Enrich

func (e *Enricher) Enrich(nodes []*model.CodeNode, root string)

Enrich populates `lex_comment` on doc-comment candidate nodes and `lex_config_keys` on config nodes. Source files are grouped by filePath so each file is read at most once across the input slice. File-group iteration order is sorted for deterministic output.

type FullTextStore

type FullTextStore interface {
	SearchByLabel(query string, limit int) ([]*model.CodeNode, error)
	SearchLexical(query string, limit int) ([]*model.CodeNode, error)
}

FullTextStore is the small surface QueryService needs from the graph package. *graph.Store satisfies this interface once its SearchByLabel / SearchLexical helpers land (plan Task 7). Defining it here keeps this package compilable independently and lets tests stand up a fake without CGO/Kuzu.

type QueryService

type QueryService struct {
	// contains filtered or unexported fields
}

QueryService bridges the lexical layer to the FTS-backed search helpers. Mirrors LexicalQueryService.java.

func NewQueryService

func NewQueryService(store FullTextStore, snippets *SnippetStore, root string) *QueryService

NewQueryService constructs a QueryService bound to a fulltext-capable store. The snippets store and root path may be nil/empty when snippet attachment is not needed (e.g. unit tests).

func (*QueryService) FindByConfigKey

func (q *QueryService) FindByConfigKey(query string, limit int) []Result

FindByConfigKey returns config-typed nodes whose lex_config_keys match the query. The same lexical index is queried as FindByDocComment, then the result set is filtered to config kinds.

func (*QueryService) FindByDocComment

func (q *QueryService) FindByDocComment(query string, limit int) []Result

FindByDocComment returns nodes whose lex_comment matches the query. When the QueryService was constructed with a non-nil SnippetStore and a non-empty root, a bounded source snippet is attached to each result.

func (*QueryService) FindByIdentifier

func (q *QueryService) FindByIdentifier(name string, limit int) []Result

FindByIdentifier returns nodes matching the query against the label / fqn fulltext index. The Source attribution is "identifier".

type Result

type Result struct {
	Node    *model.CodeNode
	Score   float32
	Snippet *CodeSnippet
	Source  string // "identifier" | "lex_comment" | "lex_config_keys"
}

Result is a single lexical search hit with source attribution. Score is reserved for downstream integration with the underlying FTS index (Kuzu QUERY_FTS_INDEX returns a score column); the bridging logic does not populate it yet.

type SnippetStore

type SnippetStore struct{}

SnippetStore is a stateless extractor. Mirrors SnippetStore.java; held as a type so the same shape can be DI'd into LexicalQueryService.

func NewSnippetStore

func NewSnippetStore() *SnippetStore

NewSnippetStore returns a stateless snippet extractor.

func (*SnippetStore) Extract

func (s *SnippetStore) Extract(node *model.CodeNode, root string) (CodeSnippet, bool)

Extract returns a snippet centred on the node's line range with the default context (±DefaultContextLines lines). Returns ok=false when the node has no location, the file is missing, or the resolved path escapes root.

func (*SnippetStore) ExtractWithContext

func (s *SnippetStore) ExtractWithContext(node *model.CodeNode, root string, ctx int) (CodeSnippet, bool)

ExtractWithContext is Extract with a caller-supplied context-line count. When the symbol's natural span (with context) exceeds MaxSnippetLines, the window is recentred on the midpoint of the symbol range and clamped.