agentmemory

package module
v0.0.10 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Oct 2, 2026 License: MIT Imports: 17 Imported by: 2

README

agentmemory

Memory for Go agents over Open Responses: what the agent keeps between sessions about the user, the project and its own past work, in a scoped store with an append-only journal, changed only through tools built with agenttool, and rendered into the instructions with a manifest of what the model was shown. A product wires the block and the tools into its agentturn config; the loop is never imported here.

  • The root module depends on openresponses, agenttool and the standard library.
  • filestore is the reference store: one Markdown file per entry, which a person can open, edit, diff and commit. It imports the root package and the standard library.
  • sqlite is a nested module on modernc.org/sqlite with full-text search.
  • storetest is the conformance suite every store passes.

Install

go get github.com/ChristopherDavenport/agentmemory
go get github.com/ChristopherDavenport/agentmemory/sqlite   # optional

Go 1.25 or later.

The shape

An Entry is one remembered thing: a kebab-case Name unique within a Scope (products define the scopes; user, project and session are the conventional three), Markdown Content bounded by the store's limit (4 KiB by default), and Meta such as a description. A Store holds entries and a journal of every change:

type Store interface {
	Get(ctx context.Context, scope Scope, name string) (*Entry, error)
	List(ctx context.Context, scope Scope) ([]Entry, error)
	Put(ctx context.Context, e Entry, opts ...PutOption) (*Change, error)
	Forget(ctx context.Context, scope Scope, name string) (*Change, error)
	Search(ctx context.Context, scopes []Scope, query string, limit int) ([]Entry, error)
	Journal(ctx context.Context, after uint64) iter.Seq2[Change, error]
	MaxEntryBytes() int
}

Put creates or replaces the whole entry and is the store's only write, so the journal is a list of full states, each with its hash. It returns the journal record it appended, so a write can be recorded beside the call that made it. IfHash(h) makes a Put conditional on the stored hash, "" meaning the entry must not exist, so a write built on a stale read fails with ErrConflict instead of clobbering. Forget writes a tombstone that keeps the last content. Every Change carries a Seq that orders the journal within the store, the Session that wrote it, from WithSession(ctx, id), and two hashes: Replaced, the state the write landed on, which chains the records, and Prev, the state it was built on, which the caller names with IfHash or, without making it a precondition, with BasedOn. A record whose Prev is not its Replaced is a write composed from a state another writer had already replaced, and LostUpdates(ctx, store, after) lists them, so an auditor can say what a session discarded. Content over the bound is refused with a SizeError naming the attempted size, the limit and what the entry holds now.

Wiring

mem, err := filestore.Open(filepath.Join(home, "memory"))
scopes := []agentmemory.Scope{"user", "project"}

block, manifest, err := agentmemory.Render(ctx, mem, scopes)
var recorded *agentmemory.Manifest // the last manifest this session wrote
var shown atomic.Pointer[agentmemory.Manifest]
shown.Store(&manifest)
memTools := agentmemory.Tools(mem, scopes,
	// memory_save names what the model was shown as its base.
	agentmemory.WithRendered(func() agentmemory.Manifest { return *shown.Load() }))
cfg := agentturn.Config{
	Model:        client,
	Instructions: prompt + "\n\n" + block + "\n\n" + agentmemory.Usage(),
	Tools:        append(tools, memTools...),
	// The freshest state each turn. Not a Transform: a Transform
	// cannot reach the instructions and what it injects is not
	// recorded.
	BeforeModelCall: func(ctx context.Context, req *openresponses.Request) error {
		b, m, err := agentmemory.Render(ctx, mem, scopes)
		if err != nil {
			return err
		}
		req.Instructions = prompt + "\n\n" + b + "\n\n" + agentmemory.Usage()
		shown.Store(&m)
		// Record the manifest when it has moved. The render is a pure
		// function of the store and the bounds, so most turns produce
		// the manifest the turn before produced, and the recorder
		// compares nothing for an annotation. After the first, only
		// what moved: a delta on the one before.
		if recorded != nil && m.Hash() == recorded.Hash() {
			return nil
		}
		ns, data := m.Record() // agentmemory:render, and the JSON
		if recorded != nil {
			ns, data = m.RecordSince(*recorded)
		}
		recorded = &m
		return rec.Annotate(ctx, ns, json.RawMessage(data))
	},
}
ctx = agentmemory.WithSession(ctx, sessionID) // attributes the journal

The module never imports the loop or the session format, so the call that writes the entry is the product's; the namespace and the bytes are the module's, through ManifestNS and Manifest.Record, so a reader of the session recognises the entry without knowing the product. Each omission in the manifest carries its scope, name, size and reason, so the record says what the model was not given and why: OmitBudget for an entry the block's bound left out, and OmitBlock, which a product writes itself, for every entry of a turn on which it had no room for the block at all. Manifest.RecordSince writes a later manifest as a delta on the one the session last recorded, the entries that moved and a keep for each run that did not, so under a memory past its bound a write costs the entry it touched rather than every entry again; ApplyManifestRecord folds a record of either form onto the manifest in force. Where two agents with their own memories take turns in one session, the manifest in force at a hand-back is the other agent's, which shares nothing with this one's, so each agent records on its own last manifest instead when that is the smaller delta, and a reader folds with a ManifestFold, which resolves a delta on any of the last ManifestFoldDepth distinct manifests in force. ApplyManifestRecord refuses such a delta, so a writer uses one only once its session's readers fold with ManifestFold (v0.0.9). A writer that has folded the session's path and kept no manifest of its own, after a restart, records fold.Record(m): the smallest of the whole record and the delta on each manifest the fold holds.

Render produces the block: a title, one section per scope, one heading per entry with its size and the limit and its description, then the content verbatim, and last a line with the counts and the budget. MaxTotalBytes (32 KiB by default) bounds the block itself, headings, descriptions and that last line included, and the line reports the block's own size. The block never comes out over it: a bound under one byte, or under what the title, that line and a heading per scope take, is refused with ErrBudget, so a product sharing one budget among instruction layers leaves memory out when its share reaches zero. The line is last because every write changes it, and the instructions are the prefix a provider caches: a write keeps every entry before the one it touched in the cached prefix. An entry whose rendered form does not fit is skipped and the next is still considered, so one large entry cannot hide the small ones after it; what was left out is listed under its scope so the model knows what memory_search can fetch. Every scope shares the one bound, in the order given, so a full scope starves the scopes after it; WithScopeMaxBytes(scope, n) caps what one scope's entries may take inside the bound, so a growing scope of facts about the environment cannot push the user's preferences out of the block. The output is determined by the store's state and the bounds alone, so an unchanged store renders the same bytes, and the Manifest lists what the block held and omitted, by scope, name, hash, size and, for an omission, the reason, for the session's provenance.

# Memory

## user

### style (28 of 4096 bytes) — How the user likes answers

Short answers.

Code in Go.

### timezone (13 of 4096 bytes)

Europe/London

Entries: 2 shown, 0 omitted. Block: 250 of 32768 bytes (32518 free). Entry limit: 4096 bytes.

RenderParts returns the same block as its parts, Part{ID, Text}, which JoinParts joins with one blank line, PartSeparator, the rule agentsession joins instructions parts by. The title is memory, each scope heading memory/<scope>, each entry memory/<scope>/<name> (PartID), a scope's omission line memory/<scope>:omitted, and the last line memory:summary. A product that records its instructions as parts, through agentturn/session's WithInstructionsParts, gives agentsession the memory parts beside its own, and a write to one entry is recorded as that entry's part and the last line, the rest by hash; the manifest's omissions, named with PartID, are its omitted parts.

The tools

Tools(store, scopes) returns four tools restricted to the scopes the product allows. The list is in each tool's schema as an enum on scope, not only in its description, and with more than one scope scope is required, so a call that omits it is an error the model can read rather than a write into whichever scope the product listed first. With one scope there is nothing to choose and the argument may be left out. A scope outside the list is an error the model sees.

WithReadScopes(scopes...) adds scopes the model may read and not write, such as project rules the product renders for the model to follow: memory_search names them in its enum and searches them when a call names no scope, so what the block omitted from them is reachable, and the three writers refuse them with scope <name> is read-only, which their descriptions state.

The tools carry agenttool.Annotations, so a host can approve the search without asking: memory_search is read-only, memory_save and memory_forget destructive, memory_patch neither, and none is open-world.

tool arguments does
memory_save scope, name, content, meta create, or replace whole
memory_patch scope, name, old_text, new_text replace one exact occurrence
memory_forget scope, name write a tombstone
memory_search query, scopes, limit find entries the block omits

Each write's result carries the journal record it produced as WriteRecord in agenttool.Result.Details, which the model never sees and a recorder writes beside the call under WriteNS (agentmemory:write), so a session says which write produced the memory and with what sequence number, hash and session.

memory_save replaces the entry, but a call that leaves meta out keeps the metadata the entry has rather than deleting the description the block and the index show; {} clears it, and the result says which happened, what it replaced and what the write was built on. Its write is anchored with BasedOn, so two channels that save one entry from one state leave a journal LostUpdates can report. The model composed the content from the block, not from the tool's read, so a product passes WithRendered with the manifest of the block it last sent: the save is then based on the hash the block showed, a write another session made after the render is a lost update the journal reports, and the result tells the model the entry had changed.

memory_patch is the tool the model is told to prefer for an edit: the call is the size of the change, and the edit is anchored in the stored text, so an edit whose anchor another session removed fails out loud and two sessions patching one entry keep both edits; the tool reads, patches and puts with IfHash, retrying on a conflict. memory_search returns five entries and 16 KiB by default, because a tool result stays in the transcript where the block is replaced each turn.

Stores

filestore.Open(dir) keeps one directory per scope, one <name>.md per entry with a frontmatter of name, updated and the meta, an INDEX.md per scope, and one journal.jsonl at the root with the cursor .state.json beside it:

memory/
  journal.jsonl
  .state.json
  user/
    INDEX.md
    style.md

Writes are atomic and serialised by a lock file that names its holder, is taken over when the holder is dead on the same host, and is waited for only so long (ErrLocked, LockHolder, BreakLock). The takeover is exclusive: one writer of the several that find a dead holder's lock removes it, so no two writers hold the store and take the same sequence number. Reconcile journals what a person changed by hand, reading the journal from the cursor in .state.json rather than whole, so a turn does not pay for the store's whole history. It still opens every entry file to notice an edit, so its cost grows with the number of entries, a few milliseconds at five hundred, which a product that reconciles before every render pays on every turn. And a write records the person's version of the entry it is about to replace, under Source: "reconciled" and no session, because once the agent has written, the file and the journal agree again and no later Reconcile could tell that anything was there. The directory is a format a second program may read and append to: docs/filestore-format.md specifies the layout, the entry file, the journal record, how a record is numbered and chained, the lock, and how a torn tail is read, and a test holds the package to the document's examples. sqlite.Open(path) keeps the same rows and the same journal lines in one database and searches an FTS5 index in relevance order. NewMemStore() is the in-memory store. All three pass storetest.Run.

Development

make check    # gofmt, tidy, vet, deps, staticcheck, govulncheck, race tests, every module

Render has golden fixtures under testdata/render/; regenerate with go test . -update and review the diff. See CONTRIBUTING.md.

License

MIT. See LICENSE.

Documentation

Overview

Package agentmemory is memory for Go agents over Open Responses: what the agent keeps between sessions about the user, the project and its own past work, in a scoped store with an append-only journal, changed only through tools and rendered into the instructions with a manifest for the session's provenance.

An Entry is one remembered thing: a kebab-case name unique within a Scope, Markdown content bounded by the store's limit, and metadata such as a description. A Store holds entries and a journal of every change; NewMemStore is the in-memory one, filestore the reference one a person can open, edit and commit, and sqlite the one with full-text search. Tools returns the four tools through which the model reaches a store, so every write is a function call in the transcript. Render turns the entries of the scopes a product names into the block it puts in agentturn's Config.Instructions, bounded in total, and the Manifest of what the block held.

The package depends on openresponses, agenttool and the standard library. The agent loop is never imported; a product wires the block and the tools into its configuration.

Index

Constants

View Source 
const (
	// MaxNameBytes bounds a scope or a name.
	MaxNameBytes = 64
	// MaxMetaBytes bounds the meta of one entry, keys and values
	// together, separately from the content bound.
	MaxMetaBytes = 1 << 10
)

Limits on names and metadata that every store enforces.

View Source 
const (
	// OmitBudget is an entry whose rendered form did not fit in what
	// was left of the block's bound. Entries after it may still fit,
	// so the block holds what it can and the model is told the rest by
	// name.
	OmitBudget = "budget"
	// OmitBlock is an entry left out because the product left the whole
	// block out: it had no room for memory on this call, as when its
	// share of an instruction budget reached zero or [Render] refused
	// the bound with [ErrBudget]. Render never writes it; a product does,
	// in the manifest it records for a turn the model saw no memory on,
	// with every entry under Omitted and none under Entries, so a reader
	// of the session tells a block dropped whole from a block that held
	// nothing because the store did.
	OmitBlock = "block"
)

Reasons an entry is left out of the block, carried on ManifestEntry.Reason.

View Source 
const (
	// TitlePartID is the block's first part, its title, which no write
	// changes. Every ID starts "memory", so a product that records its
	// own parts beside these gives its own some other prefix.
	TitlePartID = "memory"
	// SummaryPartID is the block's last part: the counts, the block's
	// size and the entry limit, which every write changes.
	SummaryPartID = "memory:summary"
)

The IDs of the block's fixed parts.

View Source 
const (
	SaveTool   = "memory_save"
	PatchTool  = "memory_patch"
	ForgetTool = "memory_forget"
	SearchTool = "memory_search"
)

The tools' names.

View Source 
const (
	// DefaultSearchLimit is how many entries a search returns when the
	// call names no limit.
	DefaultSearchLimit = 5
	// DefaultSearchBytes bounds the content one search result shows;
	// matches past it are listed by name.
	DefaultSearchBytes = 16 << 10
)

Defaults for memory_search. A search result is appended to the transcript once and stays for the rest of the session, unlike the rendered block, so both are low against the render budget.

View Source 
const DefaultMaxEntryBytes = 4 << 10

DefaultMaxEntryBytes is the per-entry bound every store starts with.

View Source 
const DefaultMaxTotalBytes = 32 << 10

DefaultMaxTotalBytes is the bound Render starts with: the whole block, summary and headings included.

View Source 
const ManifestFoldDepth = 8

ManifestFoldDepth is how many distinct manifests a ManifestFold resolves a delta's base against: the one in force and those in force before it, most recent first. A writer records a delta on a manifest other than the one in force only when it is among the last ManifestFoldDepth distinct manifests on the session's path, and otherwise on the one in force.

View Source 
const ManifestNS = "agentmemory:render"

ManifestNS is the namespace a Manifest is recorded under, so a reader of a session recognises one without knowing the product that wrote it. See Manifest.Record.

A record under it is a whole manifest, from Manifest.Record, or a delta, from Manifest.RecordSince, whose base is the hash of a manifest earlier on the path: the one in force, or, since v0.0.9, any of the last ManifestFoldDepth distinct manifests in force. A reader before v0.0.9 folds with ApplyManifestRecord and refuses a delta of the second kind, so a writer records one only once the session's readers fold with ManifestFold.

View Source 
const MetaDescription = "description"

MetaDescription is the meta key the index and the rendered block show beside an entry.

View Source 
const PartSeparator = "\n\n"

PartSeparator joins the parts RenderParts returns into the block Render returns: one blank line. It is agentsession's PartSeparator, the rule a session's instructions parts are joined by, stated here because this module does not import the session format; a product that records the block's parts as instructions parts, beside parts of its own, gets from the format's join exactly the string Render gives.

View Source 
const (
	// SourceReconciled is a state the store found in place of the one
	// it last wrote: a person edited, added or removed an entry outside
	// the store, and the store recorded what it found before writing
	// over it. Session is empty on such a record, since nobody in a
	// session made it.
	SourceReconciled = "reconciled"
)

Sources a change can have beside a write through the store, carried on Change.Source.

View Source 
const WriteNS = "agentmemory:write"

WriteNS is the namespace a memory write is recorded under, so a reader of a session recognises one without knowing the product that wrote it. It is WriteRecord's agenttool.Recordable namespace.

Variables

View Source 
var (
	// ErrNotFound is returned for an entry that is not live.
	ErrNotFound = errors.New("agentmemory: no such entry")
	// ErrInvalid is returned for a scope, name, content or meta that
	// breaks the rules in [CheckEntry].
	ErrInvalid = errors.New("agentmemory: invalid entry")
	// ErrTooLarge is returned for content over the store's bound; the
	// error is a [SizeError].
	ErrTooLarge = errors.New("agentmemory: entry over the size limit")
	// ErrConflict is returned by a conditional Put whose precondition
	// fails; the error is a [ConflictError].
	ErrConflict = errors.New("agentmemory: stored entry differs from the one expected")
)

Errors a Store returns, each wrapped with the detail of the case.

View Source 
var ErrBudget = errors.New("agentmemory: render bound is too small for the block")

ErrBudget is returned by Render and RenderParts for a bound the block cannot meet: one under one byte, or one under what the block holds whatever it shows, its title, its summary and a heading per scope; or for a scope cap under zero. The block is never returned over its bound; a product that gets ErrBudget has no room for memory on this call and leaves the block out.

View Source 
var ErrManifestBase = errors.New("agentmemory: manifest delta is not based on a manifest the reader holds")

ErrManifestBase is returned by ApplyManifestRecord for a delta whose base is not the manifest it was given, and by ManifestFold.Apply for one whose base is none the fold holds: the reader missed a record, or the delta belongs to another session.

Functions

func CheckEntry 

func CheckEntry(e Entry, limit int, stored *Entry) error

CheckEntry reports what a store refuses: an invalid scope or name, empty content or content that is not UTF-8, a meta key that is not a name or is reserved ("name", "updated"), a meta value with a line break or surrounding space, meta over MaxMetaBytes together, each as ErrInvalid; and content over limit as a SizeError, with stored's size when the entry exists. Every store calls it in Put, so the rules are the same everywhere.

func Hash 

func Hash(content string) string

Hash returns "sha256:" and the hex digest of content, the form Entry.Hash and Change.Prev carry.

func JoinParts added in v0.0.5 

func JoinParts(parts []Part) string

JoinParts returns the block the parts compose: their texts joined with PartSeparator, in order.

func Match 

func Match(e Entry, query string) bool

Match reports whether e matches query as the reference stores search: every whitespace-separated word of query appears, ignoring case, in the entry's name, one of its meta values or its content. An empty query matches every entry.

func PartID added in v0.0.5 

func PartID(scope Scope, name string) string

PartID is the ID of the part that holds an entry, "memory/" and the scope and the name, and the ID a product gives an omitted entry of the Manifest when it records what the block left out beside the parts it holds.

func RenderParts added in v0.0.5 

func RenderParts(ctx context.Context, s Store, scopes []Scope, opts ...RenderOption) ([]Part, Manifest, error)

RenderParts returns the in-context block as its parts, and the Manifest of what it holds. In order: the title; for each scope in the order given, its heading, then each entry that fits in list order, its heading carrying its size, the store's limit and its description, then its content; after a scope's entries, the line naming the ones left out; and last, the summary: the counts, the block's own size against the bound, and the entry limit.

The summary comes last because it is the one part every write changes, and the block is the instructions, the request's prefix, which every prompt cache in use caches by prefix: a write keeps every part before the entry it touched in the cached prefix. Nothing in a part depends on whether it is last.

The bound is on the joined block, not on the content it holds: the title, the summary, the scope headings, the per-entry headings with their descriptions, the list of what was left out and the separators are all counted, because they are all sent to the model, and the summary reports the block's own size so what the model reads is what the window pays. An entry whose part does not fit in what is left of the bound, or of its scope's cap under WithScopeMaxBytes, is skipped, recorded in Manifest.Omitted with OmitBudget and listed by name after its scope's entries so the model knows what memory_search can fetch; entries after it are still considered, so one large entry cannot hide the small ones that sort after it. Order is never changed: the block holds the entries in list order, whichever were skipped.

The output is determined by the store's state and the bounds alone, so an unchanged store renders the same parts and costs nothing in the session, and the manifest's hashes are the hashes of the content the block shows. A product records the manifest when Manifest.Hash differs from the last one it recorded, under ManifestNS; see Manifest.Record. The summary's own width is reserved before the entries are placed, at the widest the counts could be, so a block can come out a few bytes under the bound; it never comes out over it. The title, the summary and one heading per scope are always written, so a bound too small for those, or under one, is refused with ErrBudget and no block.

Content is not transformed, except that one trailing newline is dropped, since the separator after the part supplies it. A heading inside an entry's content at level one to three would read as structure, so the tool descriptions ask the model for level four or none.

func SessionFrom 

func SessionFrom(ctx context.Context) string

SessionFrom returns the session ID attached to ctx, or "".

func Tools 

func Tools(store Store, scopes []Scope, opts ...ToolOption) []agenttool.Tool

Tools returns the four tools through which the model reaches store, restricted to the scopes the product allows: memory_save, memory_patch, memory_forget and memory_search. A call that names a scope outside the list is an error the model sees, and the list is in the schema as an enum, not only in the description; a call that omits the scope is an error too, unless the product allows one scope and there is nothing to choose. Every write is a function call in the transcript, and the store's journal records it under the session on the context, see WithSession. A write's result carries that journal record as WriteRecord in its Details, which a recorder writes beside the call under WriteNS and the model never sees. Tools panics with no scopes, since a tool set that can reach nothing is a programming error.

memory_search is annotated read-only, memory_save and memory_forget destructive, since a save replaces the entry whole, and memory_patch neither; none is open-world. See agenttool.Annotations.

memory_search claims agenttool.ReplaySafe, so a harness resuming a session runs again a search a crash cut off. The writers claim nothing and read as agenttool.ReplayUnknown: none of them is safe to run twice as it stands. A save run again is recorded as a second write and, under WithRendered, based on the block rather than on its own first run, so LostUpdates reports the session's own write as lost; a patch run again fails because old_text is gone, or edits twice when new_text contains it; a forget run again removes whatever another session saved under the name in between.

WithReadScopes adds scopes memory_search reaches and the writers refuse, and WithRendered anchors memory_save to the block the model read.

The schema check is the tools' own, added with agenttool.Wrap; a host that unwraps a tool and executes what is inside skips it.

func Usage 

func Usage() string

Usage is one paragraph telling the model how to use the memory tools over the block Render produced. A product appends it to its instructions when it offers Tools.

func ValidName 

func ValidName(s string) bool

ValidName reports whether s is kebab-case: lowercase ASCII letters and digits in groups joined by single hyphens, between 1 and MaxNameBytes bytes. Scopes, names and meta keys all follow it, so a name is a file name, a directory name and a frontmatter key without escaping.

func ValidScope 

func ValidScope(s Scope) bool

ValidScope reports whether s is a well-formed scope: see ValidName.

func WithSession 

func WithSession(ctx context.Context, id string) context.Context

WithSession attaches the session ID that writes through ctx, so a store records it on each change. One store serves many sessions at once, so the attribution travels on the context rather than the store; a product sets it on the context it runs the agent with.

Types

type Change 

type Change struct {
	// Seq is the record's position in the store's journal, monotonic
	// within one store and the cursor [Store.Journal] takes. It
	// promises nothing across stores.
	Seq uint64 `json:"seq"`
	// Entry is the entry after the change; a tombstone carries the
	// last content with Deleted set.
	Entry Entry `json:"entry"`
	// Prev is the hash of the content the write was built on: the one
	// the caller anchored it to with [IfHash] or [BasedOn], or, when it
	// anchored the write to nothing, the hash the store held. "" for a
	// create, and for a write built on an entry that did not exist.
	//
	// It is the writer's claim about what it was editing, which is what
	// makes a fork visible: two records with one Prev and different
	// hashes are two writes built on one read, and the later of them
	// lost the earlier. Chaining is Replaced's job.
	Prev string `json:"prev,omitempty"`
	// Replaced is the hash of the content the store held when the
	// change landed, "" for a create. Records chain through it: the
	// Replaced of each names the Entry.Hash of the record before it for
	// that entry, and a record whose Prev is not its Replaced is a write
	// built on something else. See [LostUpdates].
	Replaced string `json:"replaced,omitempty"`
	// Session is the session that wrote the change, from
	// [WithSession], or "" for a person or an unattributed caller.
	Session string `json:"session,omitempty"`
	// Source says where a change the store did not write came from,
	// "" for a write through the store. [SourceReconciled] is a state
	// the store found rather than wrote, such as a person's edit to a
	// file, which a store records so that nothing it overwrites is
	// lost.
	Source string `json:"source,omitempty"`
	// At is when the change was made, for display and the record. It
	// never orders records; Seq does.
	At time.Time `json:"at"`
}

Change is one journal record: the entry as it was after the change, what it replaced, and who made it.

type ConflictError 

type ConflictError struct {
	Scope      Scope
	Name       string
	Want, Have string
}

ConflictError is the ErrConflict a conditional Put returns: the hash the caller expected and the one the store holds, "" for no entry on either side.

func (*ConflictError) Error 

func (e *ConflictError) Error() string

func (*ConflictError) Is 

func (e *ConflictError) Is(target error) bool

Is reports ErrConflict.

type Entry 

type Entry struct {
	Scope Scope `json:"scope"`
	// Name is kebab-case and unique within the scope.
	Name string `json:"name"`
	// Content is Markdown, bounded by the store's limit.
	Content string `json:"content"`
	// Meta is what the product and the model add beside the content:
	// the description the index and the block show, a type, anything
	// else. Keys are kebab-case; values are one line each.
	Meta map[string]string `json:"meta,omitempty"`
	// Hash is "sha256:" and the hex digest of Content. A store sets it.
	Hash string `json:"hash"`
	// Updated is when the store last wrote the entry. A store sets it.
	Updated time.Time `json:"updated"`
	// Deleted marks a tombstone in the journal; Content is then the
	// last content the entry held. A store never returns one from Get
	// or List.
	Deleted bool `json:"deleted,omitempty"`
}

Entry is one remembered thing.

func (Entry) Description 

func (e Entry) Description() string

Description returns the entry's description, or "".

func (Entry) Size 

func (e Entry) Size() int

Size returns the content's size in bytes, the number the bound applies to.

type LostUpdate added in v0.0.2 

type LostUpdate struct {
	// Change is the record whose base is not what it replaced.
	Change Change
	// Over is the hash the journal left for that entry just before this
	// record, "" when the entry did not exist there.
	Over string
}

LostUpdate is one journal record that was not built on the state it replaced. The write landed, so the store is consistent; what it overwrote is only in the journal, and an auditor reads this to find out what a session may have discarded.

Two shapes reach it. When Change.Replaced is Over, another writer's record landed between the read this write was built on and the write itself, and the content that record left is not in the entry any more: a lost update. When Replaced is not Over, the entry changed outside the journal, which for filestore means a person edited the file before the store noticed; filestore's Reconcile records that as a change of its own, so it appears as a record rather than a gap.

func LostUpdates added in v0.0.2 

func LostUpdates(ctx context.Context, s Store, after uint64) ([]LostUpdate, error)

LostUpdates reads s's journal from after and returns, in order, the records whose Change.Prev is not the state the journal left for that entry: the writes that were built on something else. A store whose writers all anchor their writes, as memory_patch does with IfHash and memory_save with BasedOn, returns none.

The cursor is Store.Journal's: 0 reads the whole journal, and a later number reads from there, in which case the first record seen for an entry has no predecessor in the window and is not reported. The read stops at the first error the journal yields and returns what it has with it.

type Manifest 

type Manifest struct {
	// Entries are the included entries in block order.
	Entries []ManifestEntry `json:"entries"`
	// Omitted are the entries the block's bound left out, in the order
	// they would have appeared, each with the reason.
	Omitted []ManifestEntry `json:"omitted,omitempty"`
}

Manifest lists what Render put in the block, for the session's provenance: a product records it beside the turn, in a custom entry under ManifestNS, so a later reader knows which memories the model had and which it did not.

The render is a pure function of the store and the bounds, so most turns produce the manifest the turn before produced. Manifest.Hash is what a product compares to record it only when it has changed, which is what the recorder already does for the block itself, and Manifest.Record is the namespace and the bytes to hand to the recorder when it has.

func ApplyManifestRecord added in v0.0.7 

func ApplyManifestRecord(m Manifest, data []byte) (Manifest, error)

ApplyManifestRecord returns the manifest in force after the record data, the bytes of an entry under ManifestNS, given m, the one in force before it. A whole record, from Manifest.Record, is the manifest it holds, whatever m is. A delta, from Manifest.RecordSince, is folded onto m: one whose base is not m's hash is refused with ErrManifestBase, and one whose keeps run past m, or whose result does not hash as the delta says, is refused as malformed.

A delta may be based on an earlier manifest on the path than the one in force, as a writer handed back to after a handoff records one; a reader of a session more than one agent records into folds with a ManifestFold, which resolves those.

func Render 

func Render(ctx context.Context, s Store, scopes []Scope, opts ...RenderOption) (string, Manifest, error)

Render returns the in-context block: RenderParts joined with PartSeparator. A product that records its instructions as one string uses it; one that records parts uses RenderParts.

func (Manifest) Hash added in v0.0.2 

func (m Manifest) Hash() string

Hash is the manifest's identity: "sha256:" and the hex digest of the entries it holds and the ones it left out, in order, each by scope, name, content hash, size and, for an omission, reason. Two renders that showed the model the same memories hash the same, so a product records the manifest when the hash differs from the last one it recorded and writes nothing when the render has not moved.

func (Manifest) Record added in v0.0.2 

func (m Manifest) Record() (ns string, data []byte)

Record returns the namespace and the JSON of the manifest, for a product to hand to its session recorder: this module never imports the loop or the session format, so the call that writes the entry is the product's, and the namespace and the bytes are this module's. 

if h := man.Hash(); h != last {
	last = h
	ns, data := man.Record()
	err := rec.Annotate(ctx, ns, json.RawMessage(data))
}

A manifest is strings and numbers, so encoding it cannot fail; a caller that wants an error of its own marshals the value itself.

A session that already holds a manifest records the next one with Manifest.RecordSince, which writes what moved rather than every entry again.

func (Manifest) RecordSince added in v0.0.7 

func (m Manifest) RecordSince(prev Manifest) (ns string, data []byte)

RecordSince returns the namespace and the JSON of the manifest as a delta on prev, the last manifest the product recorded in this session: a write moves one entry's hash, and under a memory at its bound the whole record repeats hundreds of entries that did not move to say so.

The delta is an object with base, prev's Manifest.Hash; hash, this manifest's; and entries and omitted, each the list with every run of prev's entries that stays in place and unchanged written as {"keep":n}, and every other entry written whole. An entry prev holds and this manifest does not is neither kept nor written. A reader folds it onto the manifest in force with ApplyManifestRecord; the member base is what tells it from a whole record.

It returns the whole record, as Manifest.Record does, when that is no larger than the delta, which it is for a first manifest, and when either manifest names one entry twice in a list, since a keep could not say which of the two it means.

The product keeps prev per session, and records a session's first manifest whole. prev is the manifest in force on the session's path, which every reader resolves, or, where more than one agent records into the session, the agent's own last manifest when it is among the last ManifestFoldDepth distinct manifests in force and the delta on it is the smaller: a reader resolves that base with a ManifestFold, and ApplyManifestRecord refuses it.

type ManifestEntry 

type ManifestEntry struct {
	Scope Scope  `json:"scope"`
	Name  string `json:"name"`
	Hash  string `json:"hash"`
	Bytes int    `json:"bytes"`
	// Reason is why the entry was left out, one of the Omit constants,
	// and empty for an entry the block holds.
	Reason string `json:"reason,omitempty"`
}

ManifestEntry names one entry by scope and name, with the hash and size of the content the block held or left out, and, for an omitted one, why. The fields are what a session needs to say what the model was shown and what it was not.

type ManifestFold added in v0.0.9 

type ManifestFold struct {
	// contains filtered or unexported fields
}

ManifestFold reads the records under ManifestNS on a session's path, in order, and keeps the manifest in force. Unlike ApplyManifestRecord it resolves a delta whose base is any of the last ManifestFoldDepth distinct manifests in force, not only the current one: when two agents with their own memories take turns in one session, each records on its own last manifest rather than the other agent's, which shares nothing with it, and writes what moved rather than every entry at each hand-back.

The zero value is ready to use and holds the empty manifest. A fold is not safe for concurrent use.

func (*ManifestFold) Apply added in v0.0.9 

func (f *ManifestFold) Apply(data []byte) error

Apply folds the record data, the bytes of an entry under ManifestNS. A whole record becomes the manifest in force. A delta whose base is none of the manifests the fold holds is refused with ErrManifestBase, and a malformed one as ApplyManifestRecord refuses it; a refused record leaves the fold as it was.

func (*ManifestFold) Manifest added in v0.0.9 

func (f *ManifestFold) Manifest() Manifest

Manifest returns the manifest in force: the empty one before any record is folded.

func (*ManifestFold) Record added in v0.0.10 

func (f *ManifestFold) Record(m Manifest) (ns string, data []byte)

Record returns the namespace and the JSON of m for a session whose readers fold with a ManifestFold: the smallest of the whole record and the delta on each manifest the fold holds, every one of which such a reader resolves. A writer that has folded the session's path so far, as one taking a session up after a restart has, calls it in place of Manifest.RecordSince on a manifest it kept itself, which it has not got; the fold holds its last manifest, when that is among the last ManifestFoldDepth distinct manifests in force, and the record is the delta on it when that is the smaller. Equal sizes prefer the manifest in force, whose delta ApplyManifestRecord resolves as well. An empty fold gives the whole record.

The fold is the reader's state and this writes nothing into it: the caller folds the record it then writes, as every reader of the session does.

type MemOption 

type MemOption func(*MemStore)

MemOption configures NewMemStore.

func WithClock 

func WithClock(now func() time.Time) MemOption

WithClock sets the clock that stamps Updated and At, for tests that want a fixed one.

func WithMaxEntryBytes 

func WithMaxEntryBytes(n int) MemOption

WithMaxEntryBytes sets the content bound; the default is DefaultMaxEntryBytes.

type MemStore 

type MemStore struct {
	// contains filtered or unexported fields
}

MemStore is the in-memory Store: the reference for the contract, the store for tests, and a store for a product whose memory need not outlive the process. It is safe for concurrent use.

func NewMemStore 

func NewMemStore(opts ...MemOption) *MemStore

NewMemStore returns an empty in-memory store.

func (*MemStore) Forget 

func (m *MemStore) Forget(ctx context.Context, scope Scope, name string) (*Change, error)

Forget implements Store.

func (*MemStore) Get 

func (m *MemStore) Get(_ context.Context, scope Scope, name string) (*Entry, error)

Get implements Store.

func (*MemStore) Journal 

func (m *MemStore) Journal(_ context.Context, after uint64) iter.Seq2[Change, error]

Journal implements Store.

func (*MemStore) List 

func (m *MemStore) List(_ context.Context, scope Scope) ([]Entry, error)

List implements Store.

func (*MemStore) MaxEntryBytes 

func (m *MemStore) MaxEntryBytes() int

MaxEntryBytes implements Store.

func (*MemStore) Put 

func (m *MemStore) Put(ctx context.Context, e Entry, opts ...PutOption) (*Change, error)

Put implements Store.

func (*MemStore) Search 

func (m *MemStore) Search(_ context.Context, scopes []Scope, query string, limit int) ([]Entry, error)

Search implements Store with Match, listing by scope order then name.

type Part added in v0.0.5 

type Part struct {
	// ID names the piece: [TitlePartID] and [SummaryPartID] for the
	// block's first and last lines, "memory/<scope>" for a scope's
	// heading, [PartID] for an entry, and "memory/<scope>:omitted" for
	// the line naming what a scope left out. Scopes and names are
	// kebab-case, so no two pieces of one block can share an ID.
	ID string `json:"id"`
	// Text is the piece, without the blank line that separates it from
	// the next.
	Text string `json:"text"`
}

Part is one piece of the rendered block, with an ID that is the same across renders for as long as the piece exists, so a product that records its instructions as parts records a write as a change to the entry it touched and names the others by hash. RenderParts returns the block as parts, in order, and JoinParts gives back the block.

type PutOption 

type PutOption func(*PutOptions)

PutOption configures one Put.

func BasedOn added in v0.0.2 

func BasedOn(h string) PutOption

BasedOn records the hash the write was built on without making it a precondition: the Put still lands on whatever the store holds, and the journal record's Prev is h, so a later reader can see that two writes were built on one read and that the second lost the first. A caller that wants the write refused instead uses IfHash, which records the same base.

It is for a writer that reads, composes and writes whole, as memory_save does: an unanchored write leaves nothing to tell a lost update from an ordinary edit, because a store fills Prev from its own value when the caller claims none.

func IfHash 

func IfHash(h string) PutOption

IfHash makes the Put conditional: it fails with ErrConflict when the stored content hash is not h, so a write built on a stale read fails instead of clobbering. "" means the entry must not exist.

type PutOptions 

type PutOptions struct {
	// IfHash is the hash the stored entry must have when Conditional
	// is set, "" meaning the entry must not exist.
	IfHash      string
	Conditional bool
	// Base is the hash the write was built on when the caller named one
	// through [BasedOn] and HasBase says it did. [IfHash] names one
	// too, and enforces it.
	Base    string
	HasBase bool
}

PutOptions is the resolved form of a Put's options, for a Store to read through ResolvePutOptions.

func ResolvePutOptions 

func ResolvePutOptions(opts ...PutOption) PutOptions

ResolvePutOptions applies the options to a zero PutOptions.

func (PutOptions) BaseFor added in v0.0.2 

func (o PutOptions) BaseFor(stored *Entry) string

BaseFor returns the hash a store records as the write's Change.Prev: the base the caller named through IfHash or BasedOn, or the hash of the entry the store holds when it named none, "" for a create. A store calls it under its write lock, with the live entry or nil.

func (PutOptions) Check 

func (o PutOptions) Check(scope Scope, name string, stored *Entry) error

Check reports an ErrConflict when the options are conditional and stored, the live entry or nil for none, does not satisfy them. A store calls it under its write lock, after reading the entry.

type RenderOption 

type RenderOption func(*renderOptions)

RenderOption configures Render.

func WithMaxTotalBytes 

func WithMaxTotalBytes(n int) RenderOption

WithMaxTotalBytes sets the block's bound; without it the bound is DefaultMaxTotalBytes. A bound under one holds no block, and Render refuses it with ErrBudget rather than reading it as the default: a caller dividing a budget among layers reaches zero or less exactly when there is no room.

func WithScopeMaxBytes added in v0.0.9 

func WithScopeMaxBytes(scope Scope, n int) RenderOption

WithScopeMaxBytes caps what one scope's entries may take of the block, inside its bound: the bytes the scope's entries and the line naming its omissions add, separators included, and not its heading, which the block holds whatever it shows. The scope packs against the smaller of its cap and what the scopes before it left, and an entry the cap leaves out is OmitBudget, as one the bound leaves out is.

A scope without a cap gets what is left, so with one bound for every scope the scope rendered last gets what the others did not take, and a full scope early in the order starves it. Capping the early scope, or each, keeps room for the later ones; a cap of zero shows none of the scope's entries and names them, room permitting. A cap under zero is refused with ErrBudget. A cap on a scope the render does not list does nothing, and the last cap given for a scope is the one that holds.

type Scope 

type Scope string

Scope says whose memory an entry is. Products define the values; "user", "project" and "session" are the conventional three. A scope has the same grammar as a name: see ValidName.

type SizeError 

type SizeError struct {
	Scope Scope
	Name  string
	// Size is the attempted content size and Limit the bound.
	Size, Limit int
	// Stored is the size the live entry holds, or -1 when there is
	// none.
	Stored int
}

SizeError is the ErrTooLarge a Put returns, with the numbers the model needs to trim: the attempted size, the limit, and what the entry holds now.

func (*SizeError) Error 

func (e *SizeError) Error() string

func (*SizeError) Is 

func (e *SizeError) Is(target error) bool

Is reports ErrTooLarge.

type Store 

type Store interface {
	// Get returns one live entry, or [ErrNotFound].
	Get(ctx context.Context, scope Scope, name string) (*Entry, error)
	// List returns the live entries of a scope by name. An unknown
	// scope lists nothing.
	List(ctx context.Context, scope Scope) ([]Entry, error)
	// Put creates or replaces the whole entry, content and meta, and
	// appends the new state to the journal. It refuses an invalid
	// entry with [ErrInvalid], content over [Store.MaxEntryBytes] with
	// [ErrTooLarge], and, under [IfHash], a stored hash other than the
	// one expected with [ErrConflict]. Hash and Updated on e are
	// ignored and set by the store.
	//
	// It returns the record it appended, the caller's own copy, so a
	// write can be recorded beside the call that made it without
	// reading the journal back. A failed Put returns nil and writes
	// nothing.
	Put(ctx context.Context, e Entry, opts ...PutOption) (*Change, error)
	// Forget writes a tombstone: the entry leaves Get and List, and
	// the journal keeps its last content, and returns that record. A
	// missing entry is [ErrNotFound].
	Forget(ctx context.Context, scope Scope, name string) (*Change, error)
	// Search returns the live entries of the scopes that match query,
	// at most limit of them when limit is positive. What matches and
	// in what order is the store's; the reference stores use [Match]
	// and list by scope then name, and sqlite ranks by full-text
	// relevance.
	Search(ctx context.Context, scopes []Scope, query string, limit int) ([]Entry, error)
	// Journal yields every change with Seq greater than after, in
	// order; 0 reads from the beginning. A reader that stops early may
	// resume with the last Seq it saw.
	Journal(ctx context.Context, after uint64) iter.Seq2[Change, error]
	// MaxEntryBytes is the content size Put refuses to exceed.
	MaxEntryBytes() int
}

Store holds entries and the journal of every change to them. Every implementation passes storetest.

type ToolOption 

type ToolOption func(*toolOptions)

ToolOption configures Tools.

func WithReadScopes added in v0.0.5 

func WithReadScopes(scopes ...Scope) ToolOption

WithReadScopes gives memory_search scopes the model may read and not write, such as project rules a product renders into the block for the model to follow and never edit. They join the scopes memory_search names in its schema and searches when a call names none, so an entry the block omitted from them is reachable as the block says it is; memory_save, memory_patch and memory_forget refuse them with "scope <name> is read-only", which their descriptions state. Render is unchanged: a product renders the read scopes beside the others.

A read scope must be kebab-case, given once, and not also one of the scopes passed to Tools; Tools panics otherwise.

func WithRendered added in v0.0.5 

func WithRendered(fn func() Manifest) ToolOption

WithRendered tells memory_save what the model was shown: fn returns the Manifest of the render the model is composing from, which is the last one the product put in the instructions. A save of an entry that manifest holds names the hash the block showed as the write's base, through BasedOn, so a write another session made after the render and this save replaced is a record LostUpdates reports, rather than a base the save claimed by reading it. An entry the manifest does not hold, a create or one the block omitted, is based on the save's own read, as it is without the option. The result line says which base the write took, and says so to the model when the entry changed after the render.

fn is called on every save, from the goroutine running the call, so it must be safe to call while the product renders the next turn. A nil fn is the same as no option. The manifest must be the one the model is reading now: a product that re-renders before every model call, in BeforeModelCall, keeps it so; one that renders once per turn has a second save of an entry in that turn based on the block rather than on the model's own first save, and reported as a lost update. memory_patch is unchanged: its edit is anchored in the stored text, so it merges with a concurrent write or fails when the anchor is gone.

func WithSearchBytes 

func WithSearchBytes(n int) ToolOption

WithSearchBytes sets the most content one memory_search result shows; the default is DefaultSearchBytes, and a value under one means the default.

func WithSearchLimit 

func WithSearchLimit(n int) ToolOption

WithSearchLimit sets the number of entries memory_search returns when the call names no limit; the default is DefaultSearchLimit, and a value under one means the default.

type WriteRecord added in v0.0.2 

type WriteRecord struct {
	// Tool is the tool that made the write.
	Tool string `json:"tool"`
	// Change is the record the store appended.
	Change Change `json:"change"`
}

WriteRecord is what a memory tool knows and the line it returns to the model does not: the journal record the write produced, with the sequence number it took, the hashes it was built on and replaced, and the session it was attributed to. The tools set it as the Details of their agenttool.Result, where it is never sent to the model, and a recorder that does not know the type writes it beside the call as a namespaced entry, which is what joins a session to the store's journal without re-reading a journal that may since have been compacted or moved.

func (WriteRecord) RecordNS added in v0.0.2 

func (WriteRecord) RecordNS() string

RecordNS implements agenttool.Recordable.

Source Files

View all Source files

Directories

Path Synopsis
Package filestore is the reference agentmemory.Store: one directory per scope, one Markdown file per entry with a frontmatter, an INDEX.md per scope listing every live entry, and one append-only journal.jsonl at the root.
 Package filestore is the reference agentmemory.Store: one directory per scope, one Markdown file per entry with a frontmatter, an INDEX.md per scope listing every live entry, and one append-only journal.jsonl at the root.
Package storetest is the conformance suite for agentmemory.Store implementations.
 Package storetest is the conformance suite for agentmemory.Store implementations.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.