index

Documentation

Overview

Package index implements the reindex flow: it lists the target tree (HEAD by default) and compares each file's git blob sha against the blob shas already recorded in the store, then indexes/updates/deletes only the files that differ. There is no watcher; reindex runs to completion and exits.

Index

• Constants
• func Contextualize(comment, headingContext, text string) string
• func Reconstruct(path paths.GitRootRelativePath, content []byte, spans []byteSpan) ([]reconstructedChunk, error)
• func Search(o *Options, query string, topK int) ([]store.SearchResult, error)
• func SyncCache(o *Options) error
• type CostEstimate
  • func Estimate(o *Options, full bool) (CostEstimate, error)
• type HealthIssue
• type HealthReport
  • func Healthcheck(o *Options) (HealthReport, error)
• type Ignore
  • func NewIgnore(patterns []string) *Ignore
  • func (i *Ignore) Match(rel paths.GitRootRelativePath) bool
• type InputOffset
• type Options
  • func (o *Options) ValidateChunking() error
• type State
  • func Reindex(o *Options) (State, error)

Constants

const (
	ChunkingAuto   = "auto"
	ChunkingManual = "manual"
)

Chunking modes. ChunkingAuto sends each whole file to the model's contextual auto-chunk endpoint and lets the model pick the chunks; ChunkingManual chunks locally (tree-sitter/cAST for code, markdown headers for text) and embeds each chunk with the model's plain embedding call, so a non-contextual provider is usable.

Functions

func Contextualize

func Contextualize(comment, headingContext, text string) string

Contextualize builds the text actually embedded for a chunk: the raw chunk text prefixed by its heading breadcrumb. When the file has a known line-comment prefix the metadata is rendered as zero-indented comments in the file's own language, which keeps the embedded text in distribution for code embedding models; the chunk text itself is left untouched so its original indentation is preserved. Files without a comment prefix (markdown/plaintext) fall back to <context>...</context> blocks. Empty components are omitted.

func Reconstruct

func Reconstruct(path paths.GitRootRelativePath, content []byte, spans []byteSpan) ([]reconstructedChunk, error)

Reconstruct rehydrates chunk rows from byte-offset spans against the exact blob content they were generated from. It slices each span's text, derives its breadcrumb from the file's structure (tree-sitter symbol path for code, markdown headers for text), computes line/col positions from the offsets, and builds the contextualized text via Contextualize. It is the single load path shared by the cache-sync reconstruction so code and text are rehydrated uniformly. A span out of range for the blob is a hard failure: a broken offset must surface immediately rather than degrade silently.

func Search

func Search(o *Options, query string, topK int) ([]store.SearchResult, error)

Search embeds the query with every active model, queries each model's vec table, and merges results by descending score (truncated to topK). It first syncs the derived cache from the mirror tree so results reflect the committed index even when the cache is stale or absent.

func SyncCache

func SyncCache(o *Options) error

SyncCache reconciles the derived SQLite cache with the mirror tree so read commands observe up-to-date results. It ensures each active model's vec table exists, then upserts artifacts whose fingerprint changed and evicts those no longer in the tree. It is safe to call on a cold cache (a full build) and on a warm one (an incremental diff); a missing/stale cache only affects latency, never results, because the mirror tree is the source of truth.

Types

type CostEstimate

type CostEstimate struct {
	Files       int
	Chunks      int
	EmbedTokens int
	Dollars     float64
}

CostEstimate is a read-only projection of a reindex run's cost: how many files/chunks need fresh work and the token/dollar totals that work is expected to incur. It is the public form of the internal costEstimate.

func Estimate

func Estimate(o *Options, full bool) (CostEstimate, error)

Estimate projects the dollar cost of a reindex without performing any paid embedding/inference work or mutating index data. When full is true it projects a from-scratch reindex of the entire repository, ignoring per-chunk reuse so every chunk is charged; otherwise it projects the next incremental reindex against HEAD, mirroring Reindex's skip and reuse decisions exactly.

type HealthIssue

type HealthIssue struct {
	Path string
	Msg  string
}

HealthIssue is a single discrepancy found by Healthcheck. Path is the repo-relative file the issue concerns, or empty for index-wide issues.

type HealthReport

type HealthReport struct {
	HeadCommit    string
	StateCommit   string
	StateMissing  bool
	ExpectedFiles int
	IndexedFiles  int
	IndexedChunks int
	Issues        []HealthIssue
}

HealthReport is the result of Healthcheck: the commits compared, the headline counts, and every discrepancy found between the git tree, the SQLite index, and the state marker. It is healthy exactly when Issues is empty.

func Healthcheck

func Healthcheck(o *Options) (HealthReport, error)

Healthcheck verifies the SQLite index and the state marker against the git tree at HEAD without mutating anything. It checks that every expected file (an indexable, non-ignored path in the tree) is indexed with the matching blob sha, that no stale files linger in the index, and that the state marker's file/chunk counts agree with the index. Blob-sha coverage -- not the recorded commit -- is the source of correctness, so a mismatched marker commit (e.g. after an amend, rebase, or a staged/pre-commit index) is not flagged.

type Ignore

type Ignore struct {
	// contains filtered or unexported fields
}

Ignore matches paths against the configured exclude globs using simple segment/prefix matching (full gitignore semantics are out of scope for v1).

func NewIgnore

func NewIgnore(patterns []string) *Ignore

NewIgnore builds an Ignore from the configured exclude patterns. Blank lines and leading/trailing whitespace or trailing slashes are normalized away.

func (*Ignore) Match

func (i *Ignore) Match(rel paths.GitRootRelativePath) bool

Match reports whether relPath is excluded by any pattern.

type InputOffset

type InputOffset int

InputOffset is a byte offset into a window's transformed input string (the synthetic context prefix followed by the verbatim body). It is meaningful only within a single window and is never persisted. It is a defined type so the compiler refuses to mix it with a raw-blob mirror.RawOffset without an explicit conversion; the only sanctioned bridge between the two systems is autoChunkWindow.toRaw.

type Options

type Options struct {
	Repo  *git.Repo
	Store *store.Store
	// Model embeds all files (code and text).
	Model  embed.EmbeddingModel
	Ignore *Ignore
	// Chunking selects the chunking mode, ChunkingAuto or ChunkingManual. Empty
	// means ChunkingAuto.
	Chunking string
	// ExtOverrides forces a file extension to a file type ("code"/"text").
	ExtOverrides map[string]string
	// MaxReindexCost caps the projected dollar cost of a single run (a
	// per-run cap, not a cumulative spend limit). Reindex estimates the cost
	// before any paid work and aborts when it exceeds MaxReindexCost. A
	// non-positive value disables the gate.
	MaxReindexCost float64
	// Staged, when true, indexes the staging area (git write-tree) rather than
	// HEAD, so a pre-commit hook can index not-yet-committed content. The staged
	// blob shas equal the blob shas the commit will contain, so a subsequent
	// post-commit reindex is a no-op.
	Staged bool
}

Options configures a reindex run.

func (*Options) ValidateChunking

func (o *Options) ValidateChunking() error

ValidateChunking checks that the configured chunking mode is known and that the model supports it. It is called before any indexing work so an unsupported combination (auto chunking on a non-contextual model) fails at startup rather than mid-reindex.

type State

type State struct {
	Commit     string `toml:"commit"`
	FileCount  int    `toml:"fileCount"`
	ChunkCount int    `toml:"chunkCount"`
}

State is the persisted marker recording how far indexing has progressed.

func Reindex

func Reindex(o *Options) (State, error)

Reindex brings the index in sync with HEAD and, only on success, advances the marker.