normalize

Documentation

Overview

Package normalize converts grammar-specific trees into shared feature bags.

Index

• Constants
• type Profile
  • func Build(ctx context.Context, root *tree_sitter.Node, source []byte, ...) (Profile, error)
  • func BuildCollection(ctx context.Context, roots []*tree_sitter.Node, source []byte, ...) (Profile, error)
  • func BuildSequence(ctx context.Context, roots []*tree_sitter.Node, source []byte, ...) (Profile, error)

Constants

const Version = 14

Version identifies the normalization contract used to build feature bags. Bump it whenever the selected comparison-unit contract, feature vocabulary, weights, canonical mappings, or semantic-hint list changes.

Types

type Profile

type Profile struct {
	Features       model.FeatureBag
	TokenCount     int
	LiteralDigests []string
}

Profile is a normalized, language-neutral view of one syntax fragment. It is not the stable content identity exposed in reports.

func Build

func Build(
	ctx context.Context,
	root *tree_sitter.Node,
	source []byte,
	isBoundary func(string) bool,
	excludeNestedBoundaries bool,
) (Profile, error)

Build creates a fingerprint and excludes nested function bodies. Nested functions are analyzed as independent fragments by the parser.

func BuildCollection

func BuildCollection(
	ctx context.Context,
	roots []*tree_sitter.Node,
	source []byte,
	isBoundary func(string) bool,
	excludeNestedBoundaries bool,
	canonicalRoot string,
) (Profile, error)

BuildCollection merges ordered syntax roots beneath one synthetic canonical node. It is used only when an opt-in comparison unit has no single Tree-sitter node spanning exactly the selected roots.

func BuildSequence

func BuildSequence(
	ctx context.Context,
	roots []*tree_sitter.Node,
	source []byte,
	isBoundary func(string) bool,
	excludeNestedBoundaries bool,
) (Profile, error)

BuildSequence creates one bounded comparison profile from an ordered list of statement roots. The synthetic block feature distinguishes these opt-in partial-function units from independently extracted functions.