memory

package
v1.117.2 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: May 15, 2026 License: Apache-2.0 Imports: 15 Imported by: 0

Documentation

Overview

* ChatCLI - Long-term memory: HyDE (Hypothetical Document Embeddings) — Phase 3a. * * The classic HyDE technique synthesizes a brief "hypothetical answer" * for the user's query and uses it as additional retrieval signal. In * the embeddings setting that signal becomes a vector; in our keyword * setting it becomes additional keywords merged with the user-derived * ones, so FactIndex.Search casts a wider, more semantic net. * * Phase 3b (vector embeddings) plugs into the same Augment call but * also writes vectors to a small SQLite-backed store; the keyword * path stays as a no-cost fallback. * * No coupling to llm/client: callers pass an AugmenterFunc closure so * memory stays free of provider imports (avoiding an import cycle).

* ChatCLI - Long-term memory: vector store for HyDE Phase 3b. * * Pure-Go cosine similarity over a JSON-persisted map of fact_id → * embedding. Deliberately not SQLite/sqlite-vec to keep CGO out of * the build matrix; for the typical chatcli memory size (hundreds of * facts) a linear scan on float32 vectors is microsecond-cheap. * * The store is dimension-locked: switching providers (e.g. voyage 1024 * → openai 1536) is rejected with an error so cosine math stays sound. * Operators clear the file (~/.chatcli/memory/vector_index.json) when * they want to migrate.

Index

Constants

View Source 
const DefaultHyDEPrompt = `` /* 544-byte string literal not displayed */

DefaultHyDEPrompt is the bilingual hypothesis-generation prompt used when HyDEConfig.Prompt is empty. Bilingual because chatcli memory stores facts in either locale and we want the keyword bag to cover both vocabularies.

View Source 
const EnhancedExtractionPrompt = `` /* 1809-byte string literal not displayed */

EnhancedExtractionPrompt is the updated prompt for the memory worker.

View Source 
const LastHypothesisSize = 600

LastHypothesisSize is unused at runtime — kept as documentation of the typical hypothesis budget. Hypothesis bodies are intentionally short (2-4 sentences) so the keyword bag stays focused.

Variables

This section is empty.

Functions

func ExtractKeywords 

func ExtractKeywords(messages []string) []string

ExtractKeywords extracts keywords from conversation messages for hint-based retrieval.

Types

type AugmenterFunc added in v1.106.0 

type AugmenterFunc func(ctx context.Context, query string) (string, error)

AugmenterFunc is the small callback shape the HyDE flow needs to ask an LLM for a hypothetical answer. Returning ("", nil) is treated as "skip augmentation" — the original hints are returned untouched.

type Compactor 

type Compactor struct {
	// contains filtered or unexported fields
}

Compactor handles LLM-based memory consolidation and cleanup.

func NewCompactor 

func NewCompactor(facts *FactIndex, daily *DailyNoteStore, config Config, memDir string, logger *zap.Logger) *Compactor

NewCompactor creates a new memory compactor.

func (*Compactor) CleanupDailyNotes 

func (c *Compactor) CleanupDailyNotes() (int, error)

CleanupDailyNotes removes old daily notes.

func (*Compactor) NeedsCompaction 

func (c *Compactor) NeedsCompaction() bool

NeedsCompaction checks if compaction should run.

func (*Compactor) RegenerateMemoryMD 

func (c *Compactor) RegenerateMemoryMD()

RegenerateMemoryMD is the public version for external callers.

func (*Compactor) RunScoreBased 

func (c *Compactor) RunScoreBased() error

RunScoreBased performs score-based compaction (no LLM needed).

func (*Compactor) RunWithLLM 

func (c *Compactor) RunWithLLM(ctx context.Context, sendPrompt func(ctx context.Context, prompt string) (string, error)) error

RunWithLLM performs LLM-assisted compaction.

type Config 

type Config struct {
	MaxMemoryMDSize    int     `json:"max_memory_md_size"`    // bytes, default 32KB
	DailyNoteRetention int     `json:"daily_note_retention"`  // days, default 30
	MaxFactsCount      int     `json:"max_facts_count"`       // default 500
	CompactionInterval int     `json:"compaction_interval_h"` // hours, default 24
	RetrievalBudget    int     `json:"retrieval_budget"`      // max chars for system prompt, default 4000
	DecayHalfLifeDays  float64 `json:"decay_half_life_days"`  // default 30
}

Config holds tunable parameters for the memory system.

func DefaultConfig 

func DefaultConfig() Config

DefaultConfig returns sensible defaults.

type DailyNote 

type DailyNote struct {
	Date    time.Time
	Path    string
	Content string
}

DailyNote represents a single day's note (same as parent package).

type DailyNoteStore 

type DailyNoteStore struct {
	// contains filtered or unexported fields
}

DailyNoteStore manages daily note files.

func NewDailyNoteStore 

func NewDailyNoteStore(memoryDir string, logger *zap.Logger) *DailyNoteStore

NewDailyNoteStore creates a new daily note store.

func (*DailyNoteStore) Cleanup 

func (d *DailyNoteStore) Cleanup(retentionDays int) (int, error)

Cleanup deletes daily notes older than retentionDays. Returns the number of files deleted.

func (*DailyNoteStore) GetRecentDailyNotes 

func (d *DailyNoteStore) GetRecentDailyNotes(days int) []DailyNote

GetRecentDailyNotes returns the last N days of notes.

func (*DailyNoteStore) TodayNotePath 

func (d *DailyNoteStore) TodayNotePath() string

TodayNotePath returns the path for today's note.

func (*DailyNoteStore) WriteDailyNote 

func (d *DailyNoteStore) WriteDailyNote(entry string) error

WriteDailyNote appends to today's daily note.

type ErrorPattern 

type ErrorPattern struct {
	Pattern    string    `json:"pattern"`
	Count      int       `json:"count"`
	Resolution string    `json:"resolution,omitempty"`
	LastSeen   time.Time `json:"last_seen"`
}

ErrorPattern records a recurring error and how it was resolved.

type Fact 

type Fact struct {
	ID            string    `json:"id"`
	Content       string    `json:"content"`
	Category      string    `json:"category"` // architecture, pattern, preference, gotcha, project, personal
	Tags          []string  `json:"tags,omitempty"`
	CreatedAt     time.Time `json:"created_at"`
	LastAccessed  time.Time `json:"last_accessed"`
	AccessCount   int       `json:"access_count"`
	Score         float64   `json:"score"`
	Source        string    `json:"source,omitempty"`
	SourceProject string    `json:"source_project,omitempty"` // project path where this fact was learned
}

Fact is a single unit of long-term memory with scoring metadata.

type FactIndex 

type FactIndex struct {
	// contains filtered or unexported fields
}

FactIndex manages scored long-term memory facts with JSON persistence.

func NewFactIndex 

func NewFactIndex(memoryDir string, config Config, logger *zap.Logger) *FactIndex

NewFactIndex creates a new fact index.

func (*FactIndex) AddFact 

func (fi *FactIndex) AddFact(content, category string, tags []string) bool

AddFact adds a new fact, deduplicating by content hash. Returns true if the fact was actually added (not a duplicate).

func (*FactIndex) AddFactWithSource 

func (fi *FactIndex) AddFactWithSource(content, category string, tags []string, sourceProject string) bool

AddFactWithSource adds a new fact with source project annotation. sourceProject is the workspace directory where the fact was learned (may be empty).

func (*FactIndex) ArchiveFacts 

func (fi *FactIndex) ArchiveFacts(facts []*Fact, archivePath string) error

ArchiveFacts moves low-scoring facts to an archive file and removes them from the index.

func (*FactIndex) Count 

func (fi *FactIndex) Count() int

Count returns the number of stored facts.

func (*FactIndex) GenerateMarkdown 

func (fi *FactIndex) GenerateMarkdown(maxSize int) string

GenerateMarkdown renders the top facts as human-readable markdown grouped by category.

func (*FactIndex) GetAll 

func (fi *FactIndex) GetAll() []*Fact

GetAll returns all facts sorted by score (descending).

func (*FactIndex) GetArchiveCandidates 

func (fi *FactIndex) GetArchiveCandidates(threshold float64) []*Fact

GetArchiveCandidates returns facts with score below threshold.

func (*FactIndex) GetByCategory 

func (fi *FactIndex) GetByCategory(category string) []*Fact

GetByCategory returns facts filtered by category.

func (*FactIndex) GetByID added in v1.106.0 

func (fi *FactIndex) GetByID(id string) (*Fact, bool)

GetByID returns a single fact by its content-hash id. The boolean signals presence so callers can distinguish "not stored" from a zero-value fact. Used by the HyDE retriever to lift cosine hits back into the keyword scorer.

func (*FactIndex) MarkAccessed 

func (fi *FactIndex) MarkAccessed(ids []string)

MarkAccessed updates access metadata for retrieved facts.

func (*FactIndex) RemoveFact 

func (fi *FactIndex) RemoveFact(id string) bool

RemoveFact removes a fact by ID.

func (*FactIndex) ReplaceFacts 

func (fi *FactIndex) ReplaceFacts(facts []*Fact)

ReplaceFacts replaces the entire fact set (used by compaction).

func (*FactIndex) Search 

func (fi *FactIndex) Search(keywords []string) []*Fact

Search returns facts matching keywords, scored by relevance. keywords are matched against content and tags (case-insensitive).

type HyDEAugmenter added in v1.106.0 

type HyDEAugmenter struct {
	// contains filtered or unexported fields
}

HyDEAugmenter generates a hypothetical answer to a query and merges keywords extracted from it with the caller's original hint set.

func NewHyDEAugmenter added in v1.106.0 

func NewHyDEAugmenter(cfg HyDEConfig, llm AugmenterFunc, logger *zap.Logger) *HyDEAugmenter

NewHyDEAugmenter constructs an augmenter. nil logger upgrades to a no-op so callers never have to nil-check.

func (*HyDEAugmenter) Augment added in v1.106.0 

func (h *HyDEAugmenter) Augment(ctx context.Context, query string, originalHints []string) []string

Augment returns the union of originalHints and keywords extracted from a freshly generated hypothesis. When HyDE is disabled, the LLM callback is nil, the query is empty, or the LLM fails, the original hints are returned unchanged so retrieval keeps working.

type HyDEConfig added in v1.106.0 

type HyDEConfig struct {
	Enabled     bool
	NumKeywords int    // upper bound on keywords pulled from the hypothesis (0 = use default)
	Prompt      string // optional override for the hypothesis system prompt
}

HyDEConfig controls hypothesis generation and keyword extraction.

type InteractionEvent 

type InteractionEvent struct {
	Timestamp time.Time
	Command   string
	Feature   string // chat, agent, coder, plugin
	Duration  time.Duration
	Error     string
}

InteractionEvent records a single interaction for stats tracking.

type LLMClient 

type LLMClient interface {
	SendPrompt(ctx interface{}, prompt string, history interface{}, maxTokens int) (string, error)
}

LLMClient is the interface the memory system needs from the LLM. Matches the subset of client.LLMClient used by memory operations.

type Manager 

type Manager struct {
	Profile  *UserProfileStore
	Facts    *FactIndex
	Topics   *TopicTracker
	Projects *ProjectTracker
	Patterns *PatternDetector
	Daily    *DailyNoteStore
	// contains filtered or unexported fields
}

Manager is the central orchestrator for the memory system. It owns all sub-stores and provides the same API surface that ContextBuilder and memoryWorker currently use, plus new capabilities.

func NewManager 

func NewManager(memoryDir string, config Config, logger *zap.Logger) *Manager

NewManager creates a new memory manager. The memoryDir should be the base memory directory (e.g., ~/.chatcli/memory/).

func (*Manager) AppendLongTerm 

func (m *Manager) AppendLongTerm(entry string) error

AppendLongTerm adds new content to long-term memory.

func (*Manager) AttachVectorIndex added in v1.106.0 

func (m *Manager) AttachVectorIndex(v *VectorIndex)

AttachVectorIndex wires a VectorIndex into the manager. Pass nil to detach (returns retrieval to keyword-only mode). Safe to call multiple times — only the most recent index is used.

func (*Manager) CleanupDailyNotes 

func (m *Manager) CleanupDailyNotes() (int, error)

CleanupDailyNotes removes old daily notes.

func (*Manager) EnsureDirectories 

func (m *Manager) EnsureDirectories() error

EnsureDirectories creates the memory directory structure.

func (*Manager) FormatExistingContext 

func (m *Manager) FormatExistingContext() string

FormatExistingContext builds the context section for the extraction prompt, including existing memory to avoid duplication.

func (*Manager) FormatStats 

func (m *Manager) FormatStats() string

FormatStats returns a formatted string of memory system statistics.

func (*Manager) GetConfig 

func (m *Manager) GetConfig() Config

GetConfig returns the current config.

func (*Manager) GetMemoryContext 

func (m *Manager) GetMemoryContext() string

GetMemoryContext returns memory context for the system prompt. This is the backward-compatible version that dumps relevant context.

func (*Manager) GetRecentDailyNotes 

func (m *Manager) GetRecentDailyNotes(days int) []DailyNote

GetRecentDailyNotes delegates to the daily note store.

func (*Manager) GetRelevantContext 

func (m *Manager) GetRelevantContext(hints []string) string

GetRelevantContext returns memory context tailored to conversation hints.

func (*Manager) GetRelevantContextWithHyDE added in v1.106.0 

func (m *Manager) GetRelevantContextWithHyDE(ctx context.Context, query string, hints []string, augmenter *HyDEAugmenter) string

GetRelevantContextWithHyDE runs the full HyDE retrieval (Phase 3a + Phase 3b) when an augmenter and/or vector index are configured. Falls back to plain Retrieve(hints) when neither is provided so the no-regression contract holds.

query is the raw user question used to seed the hypothesis. Pass "" to skip augmentation entirely.

func (*Manager) MemoryDir 

func (m *Manager) MemoryDir() string

MemoryDir returns the base memory directory path.

func (*Manager) NeedsCompaction 

func (m *Manager) NeedsCompaction() bool

NeedsCompaction checks if compaction should run.

func (*Manager) ProcessExtraction 

func (m *Manager) ProcessExtraction(response string)

ProcessExtraction processes the output from the memory extraction LLM. This replaces the old AppendLongTerm + WriteDailyNote pattern with structured extraction that populates profile, topics, and projects.

func (*Manager) ReadLongTerm 

func (m *Manager) ReadLongTerm() string

ReadLongTerm returns the rendered MEMORY.md content.

func (*Manager) RecordInteraction 

func (m *Manager) RecordInteraction(event InteractionEvent)

RecordInteraction records a usage event.

func (*Manager) RunCompaction 

func (m *Manager) RunCompaction(ctx context.Context, sendPrompt func(ctx context.Context, prompt string) (string, error)) error

RunCompaction runs memory compaction (LLM-assisted or score-based).

func (*Manager) SetWorkspaceDir 

func (m *Manager) SetWorkspaceDir(dir string)

SetWorkspaceDir sets the current session's workspace directory. Facts created during this session will be annotated with this path. The retriever also uses this to disambiguate facts from other projects.

func (*Manager) Stats 

func (m *Manager) Stats() map[string]interface{}

Stats returns a summary of memory system state.

func (*Manager) TodayNotePath 

func (m *Manager) TodayNotePath() string

TodayNotePath delegates to the daily note store.

func (*Manager) VectorIndex added in v1.106.0 

func (m *Manager) VectorIndex() *VectorIndex

VectorIndex returns the attached vector index (may be nil).

func (*Manager) WorkspaceDir 

func (m *Manager) WorkspaceDir() string

WorkspaceDir returns the current session's workspace directory.

func (*Manager) WriteDailyNote 

func (m *Manager) WriteDailyNote(entry string) error

WriteDailyNote delegates to the daily note store.

func (*Manager) WriteLongTerm 

func (m *Manager) WriteLongTerm(content string) error

WriteLongTerm replaces all long-term memory with new content. This parses the content into facts.

type Migration 

type Migration struct {
	// contains filtered or unexported fields
}

Migration handles one-time migration from flat MEMORY.md to structured format.

func NewMigration 

func NewMigration(memDir string, facts *FactIndex, logger *zap.Logger) *Migration

NewMigration creates a new migration helper.

func (*Migration) NeedsMigration 

func (m *Migration) NeedsMigration() bool

NeedsMigration checks if legacy MEMORY.md exists but no memory_index.json.

func (*Migration) RunHeuristic 

func (m *Migration) RunHeuristic() error

RunHeuristic performs a heuristic migration (no LLM needed). Splits MEMORY.md by lines and creates one fact per meaningful line.

func (*Migration) RunWithLLM 

func (m *Migration) RunWithLLM(sendPrompt func(prompt string) (string, error)) error

RunWithLLM performs LLM-assisted migration for better categorization.

type PatternDetector 

type PatternDetector struct {
	// contains filtered or unexported fields
}

PatternDetector tracks usage patterns, common errors, and skill evolution.

func NewPatternDetector 

func NewPatternDetector(memoryDir string, logger *zap.Logger) *PatternDetector

NewPatternDetector creates a new pattern detector.

func (*PatternDetector) FormatForPrompt 

func (pd *PatternDetector) FormatForPrompt() string

FormatForPrompt returns a concise usage summary.

func (*PatternDetector) GetPeakHours 

func (pd *PatternDetector) GetPeakHours(n int) []int

GetPeakHours returns the top N hours of activity.

func (*PatternDetector) GetStats 

func (pd *PatternDetector) GetStats() UsageStats

GetStats returns a copy of the current stats.

func (*PatternDetector) GetTopCommands 

func (pd *PatternDetector) GetTopCommands(n int) []string

GetTopCommands returns the N most used commands.

func (*PatternDetector) RecordInteraction 

func (pd *PatternDetector) RecordInteraction(event InteractionEvent)

RecordInteraction records a single interaction event.

func (*PatternDetector) RecordSessionEnd 

func (pd *PatternDetector) RecordSessionEnd(duration time.Duration)

RecordSessionEnd updates average session duration.

func (*PatternDetector) RecordSessionStart 

func (pd *PatternDetector) RecordSessionStart()

RecordSessionStart increments session count and updates avg duration.

type Project 

type Project struct {
	Name         string            `json:"name"`
	Path         string            `json:"path,omitempty"`
	Description  string            `json:"description,omitempty"`
	Status       string            `json:"status"` // active, paused, completed
	Technologies []string          `json:"technologies,omitempty"`
	KeyFiles     []string          `json:"key_files,omitempty"`
	LastActive   time.Time         `json:"last_active"`
	Priority     int               `json:"priority"`
	Metadata     map[string]string `json:"metadata,omitempty"`
}

Project tracks an active project with context.

type ProjectTracker 

type ProjectTracker struct {
	// contains filtered or unexported fields
}

ProjectTracker tracks active projects with context.

func NewProjectTracker 

func NewProjectTracker(memoryDir string, logger *zap.Logger) *ProjectTracker

NewProjectTracker creates a new project tracker.

func (*ProjectTracker) FormatForPrompt 

func (pt *ProjectTracker) FormatForPrompt() string

FormatForPrompt returns a summary of active projects for the system prompt.

func (*ProjectTracker) GetActive 

func (pt *ProjectTracker) GetActive() []Project

GetActive returns active projects sorted by last activity.

func (*ProjectTracker) GetAll 

func (pt *ProjectTracker) GetAll() []Project

GetAll returns all projects.

func (*ProjectTracker) Touch 

func (pt *ProjectTracker) Touch(name string)

Touch updates the LastActive timestamp for a project.

func (*ProjectTracker) Upsert 

func (pt *ProjectTracker) Upsert(updates map[string]string) bool

Upsert creates or updates a project.

type RelevanceRetriever 

type RelevanceRetriever struct {
	// contains filtered or unexported fields
}

RelevanceRetriever selects the most relevant memories for the current conversation.

func NewRelevanceRetriever 

func NewRelevanceRetriever(
	facts *FactIndex,
	profile *UserProfileStore,
	topics *TopicTracker,
	projects *ProjectTracker,
	patterns *PatternDetector,
	daily *DailyNoteStore,
	config Config,
) *RelevanceRetriever

NewRelevanceRetriever creates a new retriever.

func (*RelevanceRetriever) Retrieve 

func (r *RelevanceRetriever) Retrieve(hints []string) string

Retrieve returns memory context tailored to the current conversation. hints are extracted from recent messages (keywords, topics mentioned).

func (*RelevanceRetriever) RetrieveAll 

func (r *RelevanceRetriever) RetrieveAll() string

RetrieveAll returns the full memory dump (used for /memory longterm).

func (*RelevanceRetriever) RetrieveWithHyDE added in v1.106.0 

func (r *RelevanceRetriever) RetrieveWithHyDE(ctx context.Context, query string, hints []string, augmenter *HyDEAugmenter, vectors *VectorIndex) string

RetrieveWithHyDE runs the full HyDE retrieval path — Phase 3a (hypothesis-based keyword expansion) and Phase 3b (vector cosine search) when both are wired. When augmenter and vectors are nil or disabled, the call is byte-identical to a plain Retrieve(hints) — the no-regression contract.

query is the raw user question used to seed the hypothesis. Pass an empty string to skip augmentation entirely (rare; callers usually have at least one user message to feed in).

func (*RelevanceRetriever) SetWorkspaceDir 

func (r *RelevanceRetriever) SetWorkspaceDir(dir string)

SetWorkspaceDir updates the current workspace directory for disambiguation.

type Topic 

type Topic struct {
	Name         string    `json:"name"`
	Mentions     int       `json:"mentions"`
	FirstSeen    time.Time `json:"first_seen"`
	LastSeen     time.Time `json:"last_seen"`
	RelatedFacts []string  `json:"related_fact_ids,omitempty"`
}

Topic tracks a recurring subject across conversations.

type TopicTracker 

type TopicTracker struct {
	// contains filtered or unexported fields
}

TopicTracker tracks recurring topics across conversations.

func NewTopicTracker 

func NewTopicTracker(memoryDir string, logger *zap.Logger) *TopicTracker

NewTopicTracker creates a new topic tracker.

func (*TopicTracker) FormatForPrompt 

func (tt *TopicTracker) FormatForPrompt(limit int) string

FormatForPrompt returns a summary of active topics for the system prompt.

func (*TopicTracker) GetAll 

func (tt *TopicTracker) GetAll() []Topic

GetAll returns all topics.

func (*TopicTracker) GetTopTopics 

func (tt *TopicTracker) GetTopTopics(limit int) []Topic

GetTopTopics returns the most active topics (by recency-weighted mentions).

func (*TopicTracker) LinkFact 

func (tt *TopicTracker) LinkFact(topicName string, factID string)

LinkFact associates a fact ID with a topic.

func (*TopicTracker) Record 

func (tt *TopicTracker) Record(topicNames []string)

Record records one or more topic mentions.

type UsageStats 

type UsageStats struct {
	SessionCount     int            `json:"session_count"`
	TotalMessages    int            `json:"total_messages"`
	AvgSessionSecs   float64        `json:"avg_session_secs"`
	HourDistribution [24]int        `json:"hour_distribution"`
	CommandFrequency map[string]int `json:"command_frequency"`
	FeatureUsage     map[string]int `json:"feature_usage"`
	CommonErrors     []ErrorPattern `json:"common_errors,omitempty"`
	LastSession      time.Time      `json:"last_session"`
}

UsageStats tracks interaction patterns.

type UserProfile 

type UserProfile struct {
	Name           string            `json:"name,omitempty"`
	Role           string            `json:"role,omitempty"`
	ExpertiseLevel string            `json:"expertise_level,omitempty"` // beginner, intermediate, expert
	PreferredLang  string            `json:"preferred_language,omitempty"`
	CommStyle      string            `json:"communication_style,omitempty"`
	TopCommands    map[string]int    `json:"top_commands,omitempty"`
	Preferences    map[string]string `json:"preferences,omitempty"`
	LastUpdated    time.Time         `json:"last_updated"`
}

UserProfile tracks who the user is and how they interact.

type UserProfileStore 

type UserProfileStore struct {
	// contains filtered or unexported fields
}

UserProfileStore manages the user profile on disk.

func NewUserProfileStore 

func NewUserProfileStore(memoryDir string, logger *zap.Logger) *UserProfileStore

NewUserProfileStore creates a new profile store and loads existing data.

func (*UserProfileStore) FormatForPrompt 

func (ps *UserProfileStore) FormatForPrompt() string

FormatForPrompt returns a concise summary for system prompt injection.

func (*UserProfileStore) Get 

func (ps *UserProfileStore) Get() UserProfile

Get returns a copy of the current profile.

func (*UserProfileStore) RecordCommand 

func (ps *UserProfileStore) RecordCommand(cmd string)

RecordCommand increments command usage counter.

func (*UserProfileStore) Update 

func (ps *UserProfileStore) Update(updates map[string]string) bool

Update applies partial updates to the profile. Only non-empty fields in the update are applied.

type VectorEntry added in v1.106.0 

type VectorEntry struct {
	FactID    string    `json:"fact_id"`
	Vector    []float32 `json:"vector"`
	Dimension int       `json:"dim"`
	Provider  string    `json:"provider"`
}

VectorEntry pairs a fact id with its persisted embedding. Stored as the disk shape; in-memory the index keeps a parallel float32 slice to avoid per-call allocation.

type VectorIndex added in v1.106.0 

type VectorIndex struct {
	// contains filtered or unexported fields
}

VectorIndex is the cosine-similarity store backing HyDE vector search.

func NewVectorIndex added in v1.106.0 

func NewVectorIndex(memoryDir string, provider embedding.Provider, logger *zap.Logger) *VectorIndex

NewVectorIndex constructs the index. provider may be nil/Null — the index then becomes a no-op (Search returns nothing, Upsert silently skips). When provider is real, the on-disk file is loaded if present and dimension-checked against provider.Dimension.

func (*VectorIndex) BackfillFacts added in v1.106.0 

func (v *VectorIndex) BackfillFacts(ctx context.Context, items map[string]string) error

BackfillFacts computes embeddings for the supplied facts (id,text) pairs and stores them. Errors from the provider are returned but any successfully-embedded entries are still persisted, so a partial failure leaves the store usable.

func (*VectorIndex) Count added in v1.106.0 

func (v *VectorIndex) Count() int

Count returns the number of vectors currently stored.

func (*VectorIndex) EmbedQuery added in v1.106.0 

func (v *VectorIndex) EmbedQuery(ctx context.Context, query string) ([]float32, error)

EmbedQuery delegates to the provider so the retriever can ask for the query vector once and pass it to SimilarFacts.

func (*VectorIndex) Enabled added in v1.106.0 

func (v *VectorIndex) Enabled() bool

Enabled reports whether the index has a real provider behind it. Callers use this to short-circuit cheap paths without reflection.

func (*VectorIndex) Forget added in v1.106.0 

func (v *VectorIndex) Forget(factIDs ...string)

Forget removes a fact's vector. Called by the compactor when a fact is archived so we never serve cosine matches against a deleted note.

func (*VectorIndex) MissingFor added in v1.106.0 

func (v *VectorIndex) MissingFor(factIDs []string) []string

MissingFor returns the subset of factIDs that have no vector stored yet. The HyDE retriever feeds this to BackfillFacts so embeddings are computed lazily as facts surface.

func (*VectorIndex) ProviderName added in v1.106.0 

func (v *VectorIndex) ProviderName() string

ProviderName surfaces the configured provider name (or "null") for /config quality. Read-only — does not mutate the index.

func (*VectorIndex) SimilarFacts added in v1.106.0 

func (v *VectorIndex) SimilarFacts(query []float32, k int) []string

SimilarFacts ranks stored entries by cosine similarity to a query vector, returning the top-k fact ids. k <= 0 returns nothing.

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.