Documentation
¶
Overview ¶
Package memory is the assistant's persistent memory.
The vault holds what you write down; this holds what the assistant *learns* about you across conversations — your preferences, the people you work with, standing context and priorities — the things a good human assistant just knows and a stateless one forgets the moment a session ends. Memories are embedded and recalled semantically, so the relevant ones surface into a later conversation without you repeating yourself. That continuity is what turns a chatbot into a second brain.
Index ¶
- Constants
- Variables
- func Accept(db *sql.DB, id int64) error
- func Consolidate(db *sql.DB, rt *router.Router) (merged int, superseded int, err error)
- func Count(db *sql.DB) (int, error)
- func Decay(db *sql.DB, now int64) (int, error)
- func EffectiveSalience(m Memory, now int64) float64
- func Exclude(db *sql.DB, id int64) error
- func Export(db *sql.DB, dir string) error
- func ExportKind(db *sql.DB, dir string, kind Kind) error
- func Forget(db *sql.DB, id int64) error
- func ForgetBySource(db *sql.DB, source string) (int, error)
- func Fuse(query string, qVec []float32, cands []Candidate) []float64
- func HybridRank(query string, qVec []float32, cands []Candidate, k int) []string
- func Import(db *sql.DB, p *provider.Provider, embedModel, dir string) (int, error)
- func ImportPending(db *sql.DB, dir string) (int, error)
- func Init(db *sql.DB) error
- func Learn(db *sql.DB, rt *router.Router, exchange, source string) (int, error)
- func LearnedBetween(db *sql.DB, since, until int64) (int, error)
- func PendingCount(db *sql.DB) (int, error)
- func Pin(db *sql.DB, id int64) error
- func Reconcile(db *sql.DB, kind Kind) error
- func Reject(db *sql.DB, id int64) error
- func Render(mems []Memory) string
- func SetVault(db *sql.DB, dir string)
- func UnattributedCount(db *sql.DB) (int, error)
- func Unpin(db *sql.DB, id int64) error
- type BenchResult
- type Candidate
- type DiffEntry
- type DiffResult
- type GraphEdge
- type GraphNode
- type HealthReport
- type Kind
- type LogEntry
- type MemGraph
- type Memory
- func All(db *sql.DB) ([]Memory, error)
- func AllInProject(db *sql.DB, project string) ([]Memory, error)
- func Pending(db *sql.DB) ([]Memory, error)
- func Pinned(db *sql.DB, project string) ([]Memory, error)
- func Recall(db *sql.DB, p *provider.Provider, embedModel, query string, k int) ([]Memory, error)
- func RecallInProject(db *sql.DB, p *provider.Provider, embedModel, query, project string, k int) ([]Memory, error)
- func RecallProcedures(db *sql.DB, p *provider.Provider, embedModel, query string, k int) ([]Memory, error)
- func Surface(db *sql.DB, kinds []Kind, n int) ([]Memory, error)
- type PipelineCase
- type PipelineReport
- type Receipt
Constants ¶
const ( EvCreated = "created" // a new memory was stored EvReinforced = "reinforced" // a re-statement corroborated an existing memory EvSuperseded = "superseded" // a newer memory replaced this one (ref_id is the newer) EvMerged = "merged" // a duplicate was folded into this one (ref_id is the dropped) EvForgotten = "forgotten" // the memory was deleted EvUpdated = "updated" // an attribute (confidence, project, salience) changed EvQuarantined = "quarantined" // a machine proposed this and it is waiting for review EvAccepted = "accepted" // a quarantined memory was reviewed and released into active memory EvRejected = "rejected" // a quarantined memory was reviewed and discarded )
Event names for the log. Kept as plain strings so the log stays readable and new event types never require a migration.
const ( PinNone = 0 // ranked normally PinAlways = 1 // always included in a context pack, budget permitting — see spendMemories PinNever = 2 // excluded from recall entirely; still on record, still listed by All )
Pin states. PinNone is the default: ranked normally, neither forced in nor held out. Deliberately two states rather than a bool, because "always show me this" and "never show me this" are different user intents, not opposites of the same one — a memory can need overriding in either direction, and a single flag can only say one of them.
const DedupThreshold = 0.87
DedupThreshold is how close a new memory must be to an existing one to be treated as the same fact. High, so genuinely distinct facts about the same subject are kept; only near-restatements collapse.
The threshold alone was not enough — records that share a sentence frame and differ only in a number sit above it. It is now one of two conditions, the other being sameFact.
const Dir = "memories"
Dir is where memories live inside the vault.
const HalfLifeDays = 90.0
HalfLifeDays is how long an untouched memory takes to lose half its salience. Long, because a preference stated once is meant to persist — but not forever if it is never relevant again.
const MemSimilarityThreshold = 0.60
MemSimilarityThreshold gates "similar" edges. High enough that the graph shows genuine clusters, not a cobweb.
const OutcomeNoop = "noop"
OutcomeNoop means nothing was stored — there was nothing to store.
const PendingFile = "pending.md"
PendingFile is the review queue's name inside the memory directory. Not one of the kinds, on purpose: Import iterates `kinds`, so this file is invisible to the path that decides what counts as known.
const Schema = `` /* 1747-byte string literal not displayed */
const ( // StaleAgeDays: a memory older than this, never recalled, and faded in // effective salience is stale — still true, perhaps, but no longer earning // its keep. StaleAgeDays = 60 )
Variables ¶
var Ks = []int{1, 3, 5, 10}
Ks are the recall@k thresholds reported in one pass.
Functions ¶
func Accept ¶ added in v0.1.3
Accept releases a quarantined memory into active memory: it becomes recallable and gets written to the vault, same as anything Store creates directly. Rejects an id that is not actually pending, rather than silently no-op-ing on a typo or a double-accept.
func Consolidate ¶
Consolidate merges near-duplicate memories and supersedes ones a newer memory updates, so the store holds each fact once and holds its current version.
For each highly-similar pair, the model classifies the relation:
- duplicate → keep the more-salient, fold in salience, supersede the other
- update → the newer memory replaces the older; the older is superseded (this is what the knowledge-update case needs: "I moved to Boston" supersedes "I live in NYC")
- distinct → leave both
Only pairs above the similarity gate are shown to the model, so the number of calls stays proportional to genuine overlap, not to the store size squared.
func Decay ¶
Decay lowers the stored salience of memories that have gone unused, so the store's own sense of importance drifts toward what is actually exercised. Idempotent enough to run periodically; the floor stops anything vanishing.
func EffectiveSalience ¶
EffectiveSalience is a memory's stored salience decayed by how long since it was last useful, then lifted by how often it has proven useful. This is what ranking should trust, not the raw stored number: memory that earns its keep through recall stays prominent; memory that never helps recedes.
func Exclude ¶ added in v0.1.3
Exclude marks a memory never-include: it is dropped from recall entirely, but stays on record rather than being deleted. For the case Forget is too blunt for — a memory that is not wrong, just not something that should surface (something sensitive, something the user would rather manage by hand) — where the honest action is "stop showing me this", not "erase it".
func ExportKind ¶
ExportKind writes one kind's memories to memories/<kind>.md.
Superseded memories are left out. They are history, not knowledge — the memory_log already records that they existed and what replaced them, and carrying them here would put values the user has already corrected back into a file the user reads.
Quarantined memories are left out too, and for a stronger reason: they are not yet knowledge at all. A machine proposed them and nobody has said yes. Writing them to the file the user reads as "what the assistant knows" would make quarantine theatre — reviewable in principle, already believed in practice. They start appearing here the moment Accept clears the flag.
func ForgetBySource ¶ added in v0.1.3
ForgetBySource removes every memory learned a particular way and reports how many went. It returns the count rather than nothing so a caller can tell "I removed forty" from "there was nothing there".
This is what makes a bulk seeding reversible. `brain bootstrap` writes several memories at once from git history, and a user who dislikes the result should not have to forget them one id at a time — nor should they have to guess which ids were the machine's and which were theirs. Source is the record of how a fact arrived, so it is the honest handle for undoing a whole arrival.
Each row goes through Forget, so the timeline records every removal individually and the vault files are flushed. A bulk DELETE would be faster and would leave the vault holding notes the database no longer knows about.
func Fuse ¶
Fuse returns a reciprocal-rank-fusion score per candidate (aligned to cands), combining vector similarity and BM25 lexical rankings, normalised to 0..1 by the top score so callers can blend it with other signals like salience.
func HybridRank ¶
HybridRank fuses vector and BM25 rankings, returning candidate IDs best-first.
func Import ¶
Import rebuilds the store from the vault. This is what makes `rm -rf .brain` survivable.
The files win. Where a memory exists in both places the text, confidence and provenance are taken from markdown, because the whole point is that a person can open the file, correct a fact, and have the correction stick. Memories present in the cache but absent from the file are deleted — that is how deleting a line forgets something.
Vectors are not stored in the file; anything without one is re-embedded here, which is the same work `brain index` already does for notes.
func ImportPending ¶ added in v0.1.3
ImportPending restores the review queue from the vault, and is what makes deleting the cache survivable for proposals as well as for memories.
Ordering matters: this runs after Import, so an id the queue claims is one Import has already had its chance to restore as an active memory. If a proposal was accepted between the last flush and now, the row exists and is no longer quarantined — re-quarantining it would un-accept a decision the user already made, so an existing row is left exactly as it is.
A damaged file is refused rather than acted on, for the reason Import gives: a truncated queue read as authoritative would silently reject every proposal the missing tail held.
func Learn ¶
Learn extracts durable memories from a conversation excerpt and stores the ones worth keeping. Returns how many were newly stored. Runs at T1 — a narrow classification, not reasoning.
func LearnedBetween ¶
LearnedBetween counts memories first created in [since, until] (unix seconds), read from the append-only log so the count is stable even after some of those memories were later superseded or forgotten. Used for growth over time.
func PendingCount ¶ added in v0.1.3
PendingCount is the number `brain doctor` surfaces (see internal/health). The whole point of quarantine is that nothing sits there unseen, and a count nobody ever looks at is exactly that — so this has to be cheap enough to run on every health check, which is why it is a COUNT rather than len(Pending()).
func Pin ¶ added in v0.1.3
Pin marks a memory always-include: a context pack carries it regardless of relevance score. For the handful of things that are always worth having on hand — the standing instruction, the fact that governs everything else — and that a ranking model might otherwise rate as unremarkable on any given query.
func Reconcile ¶ added in v0.1.3
Reconcile adopts whatever the user did to one kind's file by hand, before anything overwrites it.
This closes a data loss, and the file itself is what made it one. Every memories/<kind>.md says, in its own header, "Edit any line to correct it. Delete a line to forget it." Users did — and then the next thing any agent stored called flush, which regenerates the whole file from the index, and the correction was reverted or the deleted line came back. Silently, with the instruction still printed at the top. A memory someone deliberately deleted reappearing is the worst version of that: they deleted it because they did not want an agent repeating it.
So every mutating path reconciles first and writes second. Callers do this at the *start* of their work rather than inside flush, and that ordering is the whole design: by the time flush runs, the row it is about to export is already in the index but not yet in the file, and nothing downstream could tell that apart from a line the user had just deleted.
Conservative by construction, sharing every guard Import has:
- a missing file says nothing about intent. A sync client mid-restore or a vault that has never been exported both look like this, and treating it as "the user deleted everything" is the failure this must never cause.
- a truncated file is an interrupted write, not an edit, and acting on it would delete every memory the missing tail held.
- quarantined memories are never in the file (see ExportKind), so their absence is not a deletion.
A provider is not required. An edited line keeps its old vector — COALESCE in upsert — so search is briefly stale for that one memory until `brain index` re-embeds it. Stale ranking is a small, self-correcting cost; a lost edit is permanent.
func Reject ¶ added in v0.1.3
Reject discards a quarantined memory outright. Unlike Forget, this only works on something still pending: rejecting an already-active memory is not what this call means, and forcing the caller through Forget for that keeps the two operations from being silently interchangeable.
func Render ¶
Context renders the recalled memories as a compact block for injection into a prompt — what the assistant "already knows" about the user, so it does not ask again and can act on preferences unprompted.
func SetVault ¶
SetVault tells one store where its durable copy belongs. An empty dir unbinds it, which disables writing through — what tests that only care about the cache want, and what index.Close does so a closed handle leaves nothing behind.
func UnattributedCount ¶ added in v0.1.3
UnattributedCount reports how many log lines carry no project at all. These are pre-migration events for memories that were forgotten before the backfill could reach them; they belong to some project, and there is no longer any way to say which. A project timeline that quietly omits them is accurate about what it shows and incomplete about the past, and a caller that says so is more trustworthy than one that does not.
Types ¶
type BenchResult ¶
type BenchResult struct {
Category string
N int
HitsAt map[int]int // k -> number of questions whose evidence was in the top k
}
BenchResult is the outcome for one category (or overall): hits at each k.
func RunLongMemEval ¶
func RunLongMemEval(p *provider.Provider, embedModel, path string, limit int, hybrid bool, progress func(done, total int)) ([]BenchResult, error)
RunLongMemEval evaluates retrieval recall at several k thresholds (Ks) over up to `limit` instances of a LongMemEval file, in one embedding pass.
func (BenchResult) RecallAt ¶
func (r BenchResult) RecallAt(k int) float64
RecallAt returns recall@k for this result.
type DiffEntry ¶
type DiffEntry struct {
MemID int64 `json:"mem_id"`
Event string `json:"event"`
Text string `json:"text"` // snapshot from the log at the time of the change
TS int64 `json:"ts"`
}
DiffEntry is one change to one memory within the window.
type DiffResult ¶
type DiffResult struct {
Subject string `json:"subject,omitempty"`
Since int64 `json:"since"`
Until int64 `json:"until"`
Added []DiffEntry `json:"added"` // newly learned facts
Removed []DiffEntry `json:"removed"` // superseded or forgotten
Corroborated []DiffEntry `json:"corroborated"` // re-stated, merged, or updated — belief firmed
}
DiffResult is what changed between two moments.
func Diff ¶
Diff computes what changed in the store between since and until (unix seconds, inclusive). If subject is non-empty, it narrows to changes whose snapshot text mentions it, case-insensitively — "what changed about Sarah?". Matching on the snapshot (not the live memories table) is deliberate: it still catches facts that were forgotten or superseded inside the window.
func (DiffResult) Empty ¶
func (d DiffResult) Empty() bool
type GraphEdge ¶
type GraphEdge struct {
Src string `json:"src"`
Dst string `json:"dst"`
Rel string `json:"rel"` // mentions | similar | supersedes
Weight float64 `json:"weight"`
}
GraphEdge connects two nodes.
type GraphNode ¶
type GraphNode struct {
ID string `json:"id"` // "m<id>" for a memory, the note slug for a note
Type string `json:"type"` // "memory" | "note"
Label string `json:"label"`
Kind string `json:"kind"`
Confidence float64 `json:"confidence,omitempty"`
Salience float64 `json:"salience,omitempty"`
Project string `json:"project,omitempty"`
Superseded bool `json:"superseded,omitempty"`
Degree int `json:"degree"`
}
GraphNode is a memory or a note in the memory graph.
type HealthReport ¶
type HealthReport struct {
Total int `json:"total"`
Duplicates int `json:"duplicates"` // memories that are part of a near-duplicate pair
DuplicatePairs [][2]int64 `json:"duplicate_pairs"` // a sample, for `consolidate` to act on
Stale int `json:"stale"`
StaleIDs []int64 `json:"stale_ids"` // a sample
Orphans int `json:"orphans"` // linked to no note (informational)
LowConfidence int `json:"low_confidence"`
Score float64 `json:"score"` // 0..1, fraction with no defect flag
}
HealthReport is the computed diagnostic.
type Kind ¶
type Kind string
Kind categorises a memory so it can be weighted and surfaced appropriately.
const ( // Preference: how the user likes things done ("prefers short emails", // "no meetings before 10am"). The heart of the "emotional intelligence" a // secretary is expected to have. Preference Kind = "preference" // Person: a relationship fact ("Sarah is the CFO", "Alex is sensitive // about deadlines"). Person Kind = "person" // Fact: a durable fact about the user or their work. Fact Kind = "fact" // Context: standing situational context ("launching in Q4", "hiring two // engineers") — what lets the assistant anticipate. Context Kind = "context" // Procedure: how to do something in this codebase, with a trap — what // goes wrong if you don't. The mirror of a recorded dead end: negative // knowledge stops an agent repeating a mistake, this stops it repeating // the discovery that the mistake was avoidable. Never surfaced by // ordinary recall or in a context pack (see recallScoped, Pinned) — its // only reader is before_you_try, via RecallProcedures. See // internal/procedure for the record shape and retrieval ranking. Procedure Kind = "procedure" )
type LogEntry ¶
type LogEntry struct {
ID int64 `json:"id"`
TS int64 `json:"ts"`
MemID int64 `json:"mem_id"`
Event string `json:"event"`
Detail string `json:"detail"`
RefID int64 `json:"ref_id"`
// Project is the memory's project as it stood when the event happened,
// which is not always its project now — a memory can be re-scoped, and the
// timeline of a project should show what was true at the time rather than
// retroactively rewriting itself.
Project string `json:"project,omitempty"`
}
LogEntry is one line of the timeline.
func History ¶
History returns the timeline for a single memory, oldest first — its whole life, from first learned to now.
func Timeline ¶
Timeline returns the most recent log entries, newest first, up to limit (limit <= 0 means all).
func TimelineInProject ¶ added in v0.1.3
TimelineInProject is Timeline narrowed to one project: what changed in what the assistant knows about *this* work, newest first.
It is a separate function rather than a filter the caller applies afterwards because the limit has to be applied after the narrowing, not before. Asking for "the last 20 events in kestrel" and getting two, because the other eighteen of the global last-20 belonged to other projects, is the kind of wrong that looks like an empty project rather than a bad query.
Unattributed lines — written before the log carried a project, by a memory since forgotten — match nothing and are simply absent. UnattributedCount reports how many there are so a caller can say the timeline is partial instead of implying it is complete.
type MemGraph ¶
MemGraph is the whole thing.
func BuildGraph ¶
BuildGraph assembles the memory graph. withSimilarity adds the (view-time, never-stored) similarity edges; they are the densest, so they are opt-in.
type Memory ¶
type Memory struct {
ID int64 `json:"id"`
Text string `json:"text"`
Kind Kind `json:"kind"`
Salience float64 `json:"salience"` // 0..1, how much it matters
// Confidence is how sure we are the fact is true and current, as distinct
// from salience (how much it matters). A preference you stated by hand is
// near-certain; one the model inferred from a passing remark is not.
// Corroboration raises it; being superseded or going stale lowers it.
Confidence float64 `json:"confidence"` // 0..1
// Project scopes a memory to one piece of work, so an assistant helping with
// ÉlyséeBot recalls ÉlyséeBot's context and not the whole life. Empty means
// global — it applies everywhere.
Project string `json:"project,omitempty"`
Source string `json:"source"` // where it was learned (conversation, manual, mcp)
// Agent names which coding agent learned this — "claude-code", "cursor",
// "codex" — when the MCP host said so at handshake. Empty means either the
// fact predates this field or arrived somewhere that isn't one agent among
// several (the CLI, a dream pass). Never asked of the model: see
// internal/mcpserver's clientInfoFromInitialize, which is the only writer.
Agent string `json:"agent,omitempty"`
Created int64 `json:"created"`
LastUsed int64 `json:"last_used"`
Uses int `json:"uses"`
SupersededBy int64 `json:"superseded_by,omitempty"` // id of the memory that replaced this one
// Quarantined means a machine proposed this and nobody has looked at it yet.
// A quarantined memory is a real row — same table, same shape — it is just
// invisible to Recall/All until a human accepts it. See quarantine.go.
Quarantined bool `json:"quarantined,omitempty"`
Score float64 `json:"score"` // set on recall: relevance to the query
// Pin overrides ranking entirely: PinAlways forces inclusion in a context
// pack regardless of relevance score, PinNever removes a memory from
// recall while leaving it on record. See PinAlways and PinNever.
Pin int `json:"pin,omitempty"`
// contains filtered or unexported fields
}
Memory is one remembered thing.
func All ¶
All returns every active memory, most salient first — the fallback and the CLI view. Superseded memories are excluded (they are history, not truth).
func AllInProject ¶
AllInProject is All narrowed to one project's memories plus the global ones, which is what a project-scoped view wants: the work's own facts, and the standing truths that apply to every project. An empty project means no narrowing at all, so this degrades to All.
func Pending ¶ added in v0.1.3
Pending returns quarantined memories awaiting review, oldest first — a backlog should be worked in the order it arrived, not have the newest arrival jump the queue.
It carries Agent, which the review queue needs more than active memory does: the question in front of a reviewer is whether to believe a proposal, and which agent proposed it is part of the answer.
func Pinned ¶ added in v0.1.3
Pinned returns every active PinAlways memory in scope for a project — its own plus the global ones, the same scoping AllInProject uses. This is the list contextpack must include regardless of relevance score; ranking never gets a vote on whether a pinned memory appears; only the budget does, and only in the sense of how much of it fits — see spendMemories.
kind != Procedure is not optional here the way it is a design choice elsewhere: recallScoped's exclusion only stops a procedure from being ranked into the pack, but Pinned is contextpack's separate, unconditional path around ranking entirely (see recallScoped's own comment on why no special case lives there for PinAlways). Without this line, pinning a procedure would be the one way to put instruction-shaped vault content in front of a model with no boundary line in sight — the opposite of "procedures never enter the pack."
func Recall ¶
Recall returns the memories most relevant to a query, by embedding similarity. The used memories have their reinforcement bumped, so what proves useful stays fresh — memory that is exercised persists, memory that never helps fades in the ranking.
func RecallInProject ¶
func RecallInProject(db *sql.DB, p *provider.Provider, embedModel, query, project string, k int) ([]Memory, error)
RecallInProject is Recall narrowed to a project's memory (plus global memory).
func RecallProcedures ¶ added in v0.3.9
func RecallProcedures(db *sql.DB, p *provider.Provider, embedModel, query string, k int) ([]Memory, error)
RecallProcedures returns the vault's procedure-kind memories bearing on a query, unscoped by project — deliberately, mirroring deadend.Collect, which searches every project on purpose because the procedure that would help was quite possibly recorded on a different project than the one being asked about. internal/procedure.Check takes it from here to rank and flag cross-project hits; this function's only job is "every stored procedure", the way Collect's is "every recorded dead end".
type PipelineCase ¶
type PipelineCase struct {
Name string
Exchange string // the conversation to learn from
Probe string // a later question that should recall the fact
Expect []string // any of these substrings appearing in a recalled memory counts as a hit
}
PipelineCase is one conversation and what should be learnable from it.
type PipelineReport ¶
type PipelineReport struct {
Cases int
Learned int // memories stored
RecallHits int // probes that recalled the expected fact
Duplicates int // re-learning the same case stored nothing new (good)
LearnTime time.Duration // total extraction time
RecallTime time.Duration // total recall time
MemoryCount int // final store size
Details []string // per-case notes
}
PipelineReport is the outcome of a pipeline run.
func RunPipeline ¶
func RunPipeline(db *sql.DB, rt *router.Router, cases []PipelineCase) (PipelineReport, error)
RunPipeline learns from each case, then probes recall, over a fresh store. It also re-learns every case once to confirm dedup holds (no growth).
func (PipelineReport) Accuracy ¶
func (r PipelineReport) Accuracy() float64
type Receipt ¶
type Receipt struct {
Outcome string `json:"outcome"` // EvCreated, EvReinforced, EvQuarantined, or OutcomeNoop
ID int64 `json:"id"`
// Ref is the memory that was reinforced, when the store folded into an
// existing one instead of adding a twin.
Ref int64 `json:"ref,omitempty"`
}
A Receipt says what actually happened to a memory that was stored.
"Saved" is not one outcome. The same call can create a memory, corroborate one that already existed, land in quarantine awaiting review, or do nothing — and a caller that cannot tell those apart will report all four as success. That matters most across MCP, where an agent tells the user it remembered something: it should be able to say whether it learned a new fact, confirmed an old one, or just queued something for the user to look at.
func Store ¶
Store embeds a memory and saves it, deduplicated on its normalised text so the same fact learned twice does not accumulate. The receipt distinguishes a new memory from a corroborated one.
Source Files