index

Documentation

Overview

Package index is the rebuildable cache.

Nothing here is authoritative. Delete .brain/index.db, run `brain index`, and you are back to identical state derived entirely from the markdown.

Index

• type Hit
• type Index
  • func Open(vaultDir string) (*Index, error)
  • func (ix *Index) Ask(p *provider.Provider, embedModel, chatModel, question string, k, budget int) (string, []Hit, error)
  • func (ix *Index) Close() error
  • func (ix *Index) EdgeCount() (int, error)
  • func (ix *Index) EmbedPending(p *provider.Provider, model string, batch int) (int, error)
  • func (ix *Index) Expand(hits []Hit, minConf float64, limit int) ([]Hit, error)
  • func (ix *Index) HitBySlug(slug string) (Hit, bool)
  • func (ix *Index) HybridSearch(p *provider.Provider, model, query string, k int) ([]Hit, error)
  • func (ix *Index) LexicalSearch(query string, k int) ([]Hit, error)
  • func (ix *Index) NoteCount() (int, error)
  • func (ix *Index) Search(p *provider.Provider, model, query string, k int) ([]Hit, error)
  • func (ix *Index) Sync() (SyncReport, error)
  • func (ix *Index) SyncMemories(p *provider.Provider, embedModel string) (int, error)
  • func (ix *Index) SyncNotes() (int, error)
  • func (ix *Index) SyncPending() (int, error)
• type SyncReport

Types

type Hit

type Hit struct {
	Slug, Title, Kind, Body string
	Score                   float64
	// FirstSeen is the note's date from its frontmatter, unix seconds, or zero
	// when it has none. Carried on the hit rather than looked up later because a
	// caller filtering "what did I do last month" needs it for every candidate,
	// and a per-hit query to answer that would be the retrieval path's slowest
	// step for the sake of one integer already sitting in the row.
	FirstSeen int64
	// Via is set when the note was pulled in as a graph neighbour rather than
	// matched directly, so the UI can show why it is in context.
	Via string
}

Hit is one retrieved note.

type Index

type Index struct {
	Vault string
	DB    *sql.DB
}

func Open

func Open(vaultDir string) (*Index, error)

func (*Index) Ask

func (ix *Index) Ask(p *provider.Provider, embedModel, chatModel, question string, k, budget int) (string, []Hit, error)

Ask is retrieval plus generation. Context is capped by character budget rather than token count so this stays honest on a 4k-context local model.

func (*Index) Close

func (ix *Index) Close() error

func (*Index) EdgeCount

func (ix *Index) EdgeCount() (int, error)

func (*Index) EmbedPending

func (ix *Index) EmbedPending(p *provider.Provider, model string, batch int) (int, error)

EmbedPending embeds every note that lacks a vector, in batches — a round trip per note makes a 2000-note vault take minutes instead of seconds.

func (*Index) Expand

func (ix *Index) Expand(hits []Hit, minConf float64, limit int) ([]Hit, error)

Expand pulls in notes linked from the top hits. This is the payoff of keeping a graph at all: asking about a project surfaces the people on it even when their notes share no vocabulary with the question.

func (*Index) HitBySlug

func (ix *Index) HitBySlug(slug string) (Hit, bool)

func (*Index) HybridSearch

func (ix *Index) HybridSearch(p *provider.Provider, model, query string, k int) ([]Hit, error)

HybridSearch fuses vector and lexical rankings with reciprocal rank fusion.

RRF scores each result by 1/(rank+k) summed across both lists, which needs no score normalisation between two incomparable scales (cosine distance vs bm25) — it only trusts the ordering each method is confident about. k=60 is the value from the original RRF paper and is not sensitive.

func (*Index) LexicalSearch

func (ix *Index) LexicalSearch(query string, k int) ([]Hit, error)

LexicalSearch is the FTS arm on its own, as full Hits. It is the fallback for a machine with no embedding model: worse than hybrid, but it needs no runtime and no vectors, so retrieval still works on a fresh checkout.

func (*Index) NoteCount

func (ix *Index) NoteCount() (int, error)

func (*Index) Search

func (ix *Index) Search(p *provider.Provider, model, query string, k int) ([]Hit, error)

func (*Index) Sync

func (ix *Index) Sync() (SyncReport, error)

Sync reads every note in the vault and reconciles it against the cache. Only notes whose content hash changed are re-parsed and marked for re-embedding.

func (*Index) SyncMemories

func (ix *Index) SyncMemories(p *provider.Provider, embedModel string) (int, error)

SyncMemories reconciles the memory store against its markdown in the vault.

The mirror of Sync, for the other half of what the system knows. Notes are rebuilt from files on every index; memories were not, which made "delete the database and reindex" true for one half and quietly destructive for the other. Running both from one command is what makes the guarantee whole.

Returns how many memories the vault held.

func (*Index) SyncNotes

func (ix *Index) SyncNotes() (int, error)

SyncNotes restores working notes the vault holds and the index does not.

The third half of the same guarantee. Checkpoints were always safe because they are read from markdown; memories became safe when Import landed; notes were the last thing that existed only in the database, so `rm -rf .brain` silently threw away exactly the in-flight work note_progress exists to keep.

Returns how many it restored.

func (*Index) SyncPending

func (ix *Index) SyncPending() (int, error)

SyncPending restores the memory review queue from the vault.

The fourth thing that only the database knew. A memory an agent proposed sits in quarantine until a person accepts it, and quarantine's whole point is that it is *not* written into memories/<kind>.md — so the queue lived in .brain/index.db alone, and the documented-safe rebuild threw away proposals nobody had reviewed yet, with `brain doctor` afterwards reporting "nothing pending" as though they had been.

Must run after SyncMemories: a proposal accepted since the last flush is an active memory now, and Import has to have restored it before the queue is allowed to have an opinion about that id.

Returns how many proposals it put back.

type SyncReport

type SyncReport struct {
	Added, Updated, Removed, Unchanged int
	// Skipped counts files the walk found but did not index: symlinks, and
	// files that could not be read. They are reported rather than silently
	// dropped, because "the index is missing a note" is otherwise invisible.
	Skipped int
}