memory

package module
v1.5.12 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Sep 12, 2026 License: MIT Imports: 24 Imported by: 0

README

memory

ci Go Reference

Hierarchical agent memory for Go — a portable library for durable, searchable memory entries with optional vector search and temporal APIs.

This is a library kernel (posture: embeddable filesystem palace), not a memory SaaS and not an agent runtime. It is also not MemPalace / mempalace (an unrelated Python project). Module: github.com/iome-sh/memory

Features

  • File-backed store — atomic writes, tiers (working / contextual / semantic / archival), caller-managed best-effort version snapshots (overwrite does not auto-increment)
  • Hybrid search — keyword + optional dense/sparse vectors (Qdrant) and multi-factor re-ranking
  • Temporal APIs — session/time filters, multi-session (SessionIDs / conv: grouping), event-time timelines, as-of fact listing, supersession helpers
  • Multi-hop retrieval — lightweight entity-graph expansion with hop-distance ranking
  • Pluggable embeddings — deterministic hash default for tests; production ONNX via hugot (pure-Go GoMLX or optional ORT). PersistEmbeddings default off; hash vectors are never stored
  • Compaction hooks — kernel primitives for recency/compaction pipelines
  • Benchmarks — LongMemEval-oriented tooling under cmd/ and scripts/

Install

go get github.com/iome-sh/memory@latest
# or pin a release: go get github.com/iome-sh/memory@v1.5.12

Requires the Go version in go.mod. CI uses GOTOOLCHAIN=auto.

Supported topology

One process per palace root. Multi-process writers on a shared BaseDir are unsupported — that is the product contract, not a defect to hide. In-process writeMu serializes the two shared files (relations/entity-graph.json, indexes/event-time.json). Per-entry JSON uses CreateTemp + chmod 0600 + Rename (the rename is the ingest ack). Path isolation is not cloud tenancy. Operators can collect last-write-wins evidence with scripts/two_process_writer_probe.sh; flock is not shipped.

Quick start (TTFH-shaped)

The first worked path is the walking skeleton: ingest three RCA-shaped turns, retrieve in the same process, list facts-as-of, and print source_hint. Hash embedder · no Qdrant · no cloud palace. This is not a chatbot “favourite colour” demo.

git clone https://github.com/iome-sh/memory.git
cd memory
go run ./examples/ttfh_rca

store := memory.NewPalaceStore("./data/ttfh-palace")
session := "inc-webhook-5xx"

_ = store.IngestTurn(memory.MemoryEntry{
	SessionID: session,
	Content: memory.MemoryContent{
		Summary: "PagerDuty page: webhook ingress 5xx",
		Full:    "On-call: webhook ingress returned 5xx. Start RCA from the signed delivery, not the dashboard chrome.",
		Tags:    []string{"pagerduty"},
	},
	ExtractedFacts: []string{"PagerDuty page fired for webhook ingress 5xx"},
})
// …two more RCA turns (HMAC 200 ≠ consume receipt; CreateConsumer mode NULL)…

hits := store.SearchMemoryWithOptions("hmac consume receipt", memory.SearchMemoryOptions{
	SessionID: session,
	Limit:     10,
})
for _, h := range hits {
	fmt.Println(h.Content.Summary, h.Provenance.SourceHint) // private
}

facts := store.ListFactsAsOf(memory.FactsAsOfOptions{
	SessionID: session,
	Limit:     10,
})
for _, f := range facts {
	fmt.Println(f.Content.Summary, f.Provenance.SourceHint)
}

Full program: examples/ttfh_rca. Operator page: docs/TTFH.md. Host path (optional): iomesh-tui v1.3.6 + iomesh-memory-mcp v0.4.1 — /memory ingest three RCA turns, then /memory digest --require-sources mesh,private (cite-both or explicit miss). Cost-max: hash embedder, no Qdrant, no cloud palace, optional Ollama via the TUI.

If PalaceConfig.BaseDir (or NewPalaceStore's argument) is empty, the store uses .palace under the process working directory (DefaultPalaceBaseDir). Prefer an explicit path in applications. This is a local filesystem root — not a leftover .ossa product path and not a hosted palace.

This package is a local filesystem library, not a cloud multi-tenant service. It does not implement mesh X-IOMesh-Org headers. Isolation is the directory you pass as BaseDir (or OS isolation around that directory).

Optional semantic embeddings
embedFn, err := memory.NewGONNXEmbeddingFuncFromEnv()
if err != nil {
	panic(err)
}
store := memory.NewPalaceStoreWithConfig(memory.PalaceConfig{
	BaseDir:       "./data/palace",
	EmbeddingFunc: embedFn,
})
Variable Purpose
MEMORY_ONNX_MODEL_PATH Hugot model directory or .onnx file
MEMORY_HUGOT_BACKEND go (default pure-Go), ort, or auto
MEMORY_ORT_LIBRARY_DIR Directory containing ONNX Runtime shared library
MEMORY_ORT_CUDA 1 to enable CUDA EP (Linux ORT builds)
MEMORY_EMBEDDING_STRICT true to disable hash fallback on inference errors

Default ONNX export is BGE-small-en-v1.5 (384 dimensions). When using Qdrant with that model, set collection EmbeddingDim to 384.

PersistEmbeddings defaults off. When on, only a non-hash EmbeddingModel (e.g. bge-small-en-v1.5) is stored on entry JSON. Hash embeddings (GenerateSimpleEmbedding, empty or "hash" model) are never persisted as stored vectors / QueryVec. Embed miss is not ingest failure; JSON rename remains the ack. usearch, ORT, and Qdrant stay optional.

Download helper (fail-soft; BGE is optional):

go run ./scripts/download_onnx_model.go
export MEMORY_ONNX_MODEL_PATH="$(go run ./scripts/download_onnx_model.go)"

Hugging Face 401/404 for KnightsAnalytics/bge-small-en-v1.5 is expected — that repo is not published (valid token still 404, not a login miss). The helper then downloads public BAAI/bge-small-en-v1.5 (onnx/model.onnx) — no Hugging Face login required. HF_TOKEN is optional; on 401/403 with a token the helper retries unauthenticated so a bad token cannot block public BGE. Layout: testdata/models/BAAI_bge-small-en-v1.5/ (gitignored; ~127 MB; not vendored). If that fetch fails, in-tree MiniLM is the local 384-d fallback — not the official V1 BGE pin. Hash-overlap unpublished. If MiniLM is missing too, stdout is empty and TTFH cost-max stays the hash embedder. os.Exit(1) only for mkdir failures. Generate/judge need OPENAI_API_KEY (unrelated to HF). Auth matrix: docs/LONGMEMEVAL_BASELINE.md. Locked mixed n=12: same page (make longmemeval-baseline) — not a README number, not official V1.

Optional Qdrant
podman run -d --name qdrant \
  -p 6333:6333 -p 6334:6334 \
  -v qdrant_storage:/qdrant/storage:z \
  qdrant/qdrant

store := memory.NewPalaceStoreWithConfig(memory.PalaceConfig{
	BaseDir:          "./data/palace",
	VectorURL:        "http://localhost:6333",
	VectorCollection: "memory_collection",
	EmbeddingFunc:    embedFn, // recommended for semantic recall
})

Unit tests run without Podman/Qdrant. Integration helpers start a temporary container when available; set PODMAN_QDRANT_SKIP=1 to force skip.

When to use this kernel

Stars and vendor LongMemEval scores are a category error here.

Job Use
Inspectable local ops record (JSON files, cat/diff/cite) in Go, no required DB or extract LLM This kernel
Chatbot personalization API / drop-in memory SaaS Mem0
Dual-clock temporal knowledge graph (Neo4j / FalkorDB / Neptune) Graphiti / Zep
Agent runtime that edits its own memory blocks Letta
Documents/tables → company knowledge graph Cognee
Already on LangGraph, want a Python library LangMem
Coding-agent session compressor / verbatim IDE store claude-mem, MemPalace (unrelated Python project — name collision only)

Naming: MemPalace / mempalace is a different project. Industry roundups that list “MemPalace” next to Mem0 are not describing this repository.

API overview

Area Entry points
Store NewPalaceStore, NewPalaceStoreWithConfig, Write, Read, …
Search SearchMemory, SearchMemoryWithOptions
Timeline ListMemoryWithOptions
As-of facts ListFactsAsOf, ParseValidityWindow, EntryValidAt
Supersession SupersedeEntityFacts, WriteAndSupersede
Multi-hop MultiHopRetrieve, ExpandRelatedEntities, ExpandRelatedEntitiesHops
Vectors NewVectorStore, collection create/upsert helpers
Embeddings GenerateSimpleEmbedding, NewGONNXEmbeddingFunc, NewGONNXEmbeddingFuncFromEnv
Search options
from := time.Now().Add(-24 * time.Hour)
results := store.SearchMemoryWithOptions("project goals", memory.SearchMemoryOptions{
	SessionID:      "sess-abc",
	TimeFrom:       &from,
	Limit:          10,
	ReRankTemporal: true,
})
Field Effect
SessionID Keep entries with matching session
TimeFrom / TimeTo Inclusive event-time window
Limit Cap results (default 10 for search)
Tier Optional tier filter
IncludeArchival When Tier is nil, also walk Archival (default retrieve skips it)
QueryVec Dense re-rank when non-empty; keyword token hits stay ahead of Limit
ReRankTemporal Sort by relevance after keyword/vector path; keyword hits stay ahead of Limit

Default retrieve tiers are Working + Contextual + Semantic (Archival skipped), matching ListMemoryWithOptions. Archival is included when IncludeArchival is set, Tier is Archival, or the default-tier keyword hit set is empty (low-confidence fallback; not a numeric score cutoff).

Timeline list
timeline := store.ListMemoryWithOptions(memory.ListMemoryOptions{
	SessionID: "sess-abc",
	TimeFrom:  &from,
	TagPrefix: "subject:",
	Limit:     50,
})

Filters apply before Limit. Default limit is 50 when ≤ 0. Listing uses a best-effort in-memory meta index plus an optional durable snapshot (indexes/event-time.json). A clean index is patched on Write / unlink instead of walking every tier JSON; a new process skips re-parse when the stamp matches. FS Palace remains source of truth. DisableMetaIndex / DisableDurableIndex opt out. Btree/tag secondary indexes remain residual.

Full reference: pkg.go.dev/github.com/iome-sh/memory.

Development

git clone https://github.com/iome-sh/memory.git
cd memory
go mod download

make check   # fmt-check + vet + test
make ci      # + govulncheck + build
make test
make test-race   # optional

Optional last-write-wins evidence (not a lock): make two-process-writer-probe / scripts/two_process_writer_probe.sh. Multi-process writers remain unsupported. Probe ≠ flock; flock is not shipped. Not part of make ci / make test.

See CONTRIBUTING.md for the contributor guide and SECURITY.md for reporting vulnerabilities.

LongMemEval tooling (optional)

Methodology card: docs/LONGMEMEVAL.md. No official number is published. Official V1 is upstream evaluate_qa.py + judge gpt-4o-2024-08-06 against longmemeval_oracle.json (ONNX, mixed-type, session_id on retrieve). Makefile default judge gpt-4o-mini is a cheap local path — not official V1.

Offline overlap smoke/bench (no OpenAI). Printed aggregate recall is top-k gold-answer string overlap (judge-free). It is not official V1 and not V2 LAFS Gain. Hash embeddings are the no-dep default; do not publish hash overlap as a leaderboard number.

make longmemeval-smoke
make longmemeval-recall-gate
make longmemeval-bench
make longmemeval-v2-bench   # official V2 file layout; does not vendor the 7GB snapshot
make longmemeval-v1-card    # methodology card; SKIP if oracle missing (exit 0); not make ci

Official V1 scored QA: make longmemeval-judge (needs OPENAI_API_KEY). Official judge pin is gpt-4o-2024-08-06; Makefile default gpt-4o-mini is a cheap local path — not official V1. Methodology card (no published score): make longmemeval-v1-card — optional, not part of make ci; missing oracle is SKIP (exit 0). In-repo subset is 3 single-session-user items, not mixed official V1. Official V2 scored runs use the upstream harness with a fixed Qwen3.5-9B reader and GPT-5.2 judge — this kernel only loads V2 files and exposes Insert/Query. Full dataset / judge flows need extra deps and keys; see comments in Makefile and scripts/.

--limit N on scripts/longmemeval_qa_generate.py is dataset prefix order. Official V1 starts with temporal-reasoning, so a small n is not a mixed V1 score. Use --sample mixed (or LONGMEMEVAL_QA_SAMPLE=mixed) for a stratified slice and print the type histogram. Prefix-n is not overall V1. overlap ≠ gpt-4o ≠ V2 LAFS.

/retrieve accepts session_id (official generate passes conv_id / question_id). Shared-palace QA without it is other-session dominated. Hypothesis JSONL keeps question_date, retrieve snippets, and embed_mode for audit. Hash default. Not a leaderboard submit.

Haystack dates accept official cleaned 2006/01/02 (Mon) 15:04 as well as RFC3339.

Documentation

Document Description
CHANGELOG.md Release notes
RELEASING.md How maintainers tag module versions; support / version policy for consumers
SECURITY.md Security policy and supported-versions table
CONTRIBUTING.md Development workflow
CODE_OF_CONDUCT.md Community standards
SUPPORT.md How to get help; scope (library kernel) and related host
docs/temporal-memory-kernel-roadmap.md Temporal API roadmap (shipped vs T1–T5 next TODOs)
docs/TTFH.md Operator TTFH walking skeleton
docs/LONGMEMEVAL.md LongMemEval methodology card (no published official number)
docs/OPEN_SOURCE_AUDIT.md Maintainer OSS process residual (not a product spec)
Repository Role
iomesh-memory-mcp Lean MCP host binary for this kernel
iomesh-tui Multi-provider agent TUI/CLI (optional mesh hooks)
iomesh-client-sdk-go Official Go client for I/O Mesh
iomesh-client-sdk-python Official Python client for I/O Mesh (Beta / pre-1.0)

This module is a library (tags for go get). Binary packaging, SBOM, and cosign apply to host tools such as iomesh-memory-mcp — see RELEASING.md.

License

MIT · NOTICE

Documentation

Index

Constants

View Source 
const (
	// MiniLMEmbeddingDim is the output size for all-MiniLM-L6-v2 (and KnightsAnalytics ONNX export).
	MiniLMEmbeddingDim = 384
	// MiniLMMaxSeqTokens is the BERT position limit for all-MiniLM-L6-v2 ONNX exports.
	MiniLMMaxSeqTokens = 512

	// DefaultHashEmbeddingDim is the dimension used by GenerateSimpleEmbedding when dim <= 0.
	DefaultHashEmbeddingDim = 768
	// EnvONNXModelPath points at a hugot model directory (tokenizer + model.onnx) or a single .onnx file.
	EnvONNXModelPath = "MEMORY_ONNX_MODEL_PATH"
	// EnvEmbeddingStrict disables hash fallback when ONNX inference fails (recommended in production).
	EnvEmbeddingStrict = "MEMORY_EMBEDDING_STRICT"
)
View Source 
const (
	// EnvHugotBackend selects the hugot inference backend: go (default), ort, or auto.
	EnvHugotBackend = "MEMORY_HUGOT_BACKEND"
	// EnvORTLibraryDir is the directory containing libonnxruntime.so/.dylib (ORT builds only).
	EnvORTLibraryDir = "MEMORY_ORT_LIBRARY_DIR"
	// EnvORTCuda enables the CUDA execution provider when set to 1/true.
	EnvORTCuda = "MEMORY_ORT_CUDA"
	// EnvORTCoreML enables the CoreML execution provider when set to 1/true.
	EnvORTCoreML = "MEMORY_ORT_COREML"
	// EnvORTCudaDeviceID selects the CUDA device (default 0).
	EnvORTCudaDeviceID = "MEMORY_ORT_CUDA_DEVICE_ID"
)
View Source 
const (
	// DefaultONNXModelHF is the hugot-style ONNX export id. KnightsAnalytics has
	// not published this repo (HF 404 with auth). Download falls back to BAAI.
	DefaultONNXModelHF = "KnightsAnalytics/bge-small-en-v1.5"
	// BAAIONNXModelHF is the public BGE-small-en-v1.5 source (onnx/model.onnx).
	// Reshaped into hugot layout (root model.onnx) as testdata/models/BAAI_bge-small-en-v1.5.
	BAAIONNXModelHF = "BAAI/bge-small-en-v1.5"
	// LegacyONNXModelHF is the previous default (384-d MiniLM).
	LegacyONNXModelHF = "KnightsAnalytics/all-MiniLM-L6-v2"
	// BGESmallEmbeddingDim is the output width for BAAI/bge-small-en-v1.5 ONNX exports.
	BGESmallEmbeddingDim = 384
	// EnvEmbeddingModelHF overrides the Hugging Face model id used for documentation and download scripts.
	EnvEmbeddingModelHF = "MEMORY_EMBEDDING_MODEL"
)
View Source 
const ConvTagPrefix = "conv:"

ConvTagPrefix groups haystack sessions that belong to one conversation/palace retrieve key (T1). Retrieve with SessionID=conv_id matches entries tagged conv:<conv_id> even when MemoryEntry.SessionID is the inner haystack session.

View Source 
const DefaultPalaceBaseDir = ".palace"

DefaultPalaceBaseDir is the local palace root used when PalaceConfig.BaseDir (or NewPalaceStore's argument) is empty. Relative to the process working directory. Not a leftover product path and not a hosted palace.

View Source 
const SourceHintPrivate = "private"

SourceHintPrivate is the default local-palace ingest class. It maps to the TUI ClassifyDigestSourceHint private bucket (private, palace, local, …).

View Source 
const SourceHintTagPrefix = "source_hint:"

SourceHintTagPrefix is the Content.Tags / TemporalTags prefix for an observable source class (source_hint:private, source_hint:mesh, …).

Variables

View Source 
var DefaultCompactionConfig = CompactionConfig{
	Tier2Strategy:         StrategyPatternExtraction,
	Tier3Strategy:         StrategyCorePrinciple,
	TemporalWindowSize:    12,
	SimilarityThreshold:   0.75,
	DataSim:               0.7,
	DataCount:             5,
	ProtectHighScoreFacts: true,
	FactScoreThreshold:    0.90,
}

Functions

func AssembleCountEvidence added in v1.5.12 

func AssembleCountEvidence(query string, facts []MemoryEntry) string

AssembleCountEvidence lists unique matching fact snippets for a count query (named-pattern first, then first-person noun overlap). Deduped, not persisted. Clothing count queries diversify by action+object (dry-clean / return / pick-up × boot / blazer / generic) so a compound "return … pick them up" is two bullets and dry-clean survives when the query only says pick/return/store. Empty when the query is not a count question or no facts match.

func CalculateRecencyBoost 

func CalculateRecencyBoost(deltaHours float64) float64

CalculateRecencyBoost returns recency boost factor

func CalculateRelevanceScore 

func CalculateRelevanceScore(entry MemoryEntry) float64

CalculateRelevanceScore combines score impact, recency, temporal decay, usage

func CalculateTemporalDecay 

func CalculateTemporalDecay(entry MemoryEntry) float64

CalculateTemporalDecay implements H-Mem style forgetting curve

func ClassifyIngestSourceHint added in v1.5.10 

func ClassifyIngestSourceHint(hint string) string

ClassifyIngestSourceHint maps a source hint or tag to mesh, private, or "". Aligned with TUI ClassifyDigestSourceHint cite-both buckets. Host process labels (mcp_memory_ingest_turn, source:iomesh-memory-mcp) are not a class. Catalog/grant/external stay host concerns and return "".

func ConvTag added in v1.5.12 

func ConvTag(id string) string

ConvTag returns conv:<id>, or empty when id is blank.

func CosineSimilarity 

func CosineSimilarity(a, b []float32) float64

CosineSimilarity helper

func DefaultEmbeddingModelFromEnv 

func DefaultEmbeddingModelFromEnv() string

DefaultEmbeddingModelFromEnv returns the configured HF model id or DefaultONNXModelHF.

func DefaultONNXModelCacheDirName 

func DefaultONNXModelCacheDirName() string

DefaultONNXModelCacheDirName is the testdata/models subdirectory for the default ONNX export.

func EntryEntityKeys 

func EntryEntityKeys(e MemoryEntry) []string

EntryEntityKeys extracts entity-like keys from an entry for multi-hop matching. Sources:

  • TemporalTags with "entity:" prefix → value after prefix (e.g. entity:person:alice → person:alice)
  • TemporalTags with "subject:" prefix → full tag (subject:auth)
  • Content.Tags with entity:/subject: prefixes (same rules)
  • Relations.RelatedConcepts (trimmed non-empty strings as-is)

Duplicates are removed; order is stable (first-seen).

func EntryHasTag 

func EntryHasTag(e MemoryEntry, tag string) bool

EntryHasTag reports whether e has an exact tag match in TemporalTags or Content.Tags.

func EntryHasTagPrefix 

func EntryHasTagPrefix(e MemoryEntry, prefix string) bool

EntryHasTagPrefix reports whether e has a TemporalTags or Content.Tags entry with the given prefix.

func EntryValidAt 

func EntryValidAt(e MemoryEntry, asOf time.Time) bool

EntryValidAt reports whether e is considered valid at asOf.

Rules (bi-temporal lite — validity window on entries, not full dual clocks):

  1. Zero asOf is treated as time.Now().UTC().
  2. When valid_from / valid_until tags are present: - if valid_from is set and asOf.Before(from) → false - if valid_until is set and !asOf.Before(until) → false (valid_until is an exclusive end: asOf == until is invalid)
  3. When NO validity tags at all: fall back to "known by asOf" — valid if entryEventTime is zero OR !entryEventTime.After(asOf) (entry exists / was recorded by asOf).

This is not a full temporal knowledge graph (no transaction time, no edge validity). Hosts that write validity tags get windowed facts; untagged entries remain historically "known once recorded."

func ExtractAtomicFacts 

func ExtractAtomicFacts(entry MemoryEntry) []string

ExtractAtomicFacts extracts high-value personal facts from a memory entry.

func FormatSourceHintTag added in v1.5.10 

func FormatSourceHintTag(hint string) string

FormatSourceHintTag returns source_hint:<hint> for a non-empty hint.

func GenerateMemoryID 

func GenerateMemoryID() string

GenerateMemoryID uses cuid2

func GenerateSimpleEmbedding 

func GenerateSimpleEmbedding(text string, dim int) []float32

GenerateSimpleEmbedding (deterministic hash-based fallback)

func HugotCacheDirName 

func HugotCacheDirName(hfModel string) string

HugotCacheDirName returns the local directory name hugot uses for a Hugging Face model id.

func MultiFactorScore 

func MultiFactorScore(entry MemoryEntry, queryVec []float32) float64

MultiFactorScore implements full H-Mem scoring

func ParseValidityWindow 

func ParseValidityWindow(e MemoryEntry) (from, until *time.Time)

ParseValidityWindow reads valid_from / valid_until from TemporalTags. Missing or unparseable bounds are open-ended (nil).

Tag format: "valid_from:<RFC3339>" and "valid_until:<RFC3339>" (time.RFC3339 / time.RFC3339Nano accepted via time.Parse).

func PopulateTemporalTags 

func PopulateTemporalTags(cycle int) []string

PopulateTemporalTags (cycle-aware)

func ResolveEmbeddingDim 

func ResolveEmbeddingDim(modelPath string) int

ResolveEmbeddingDim returns the Qdrant/Palace vector width for the active embedding backend. When an ONNX model path is configured, infers dimension from the model directory name; otherwise hash (768).

func ResolveEmbeddingDimFromEnv 

func ResolveEmbeddingDimFromEnv() int

ResolveEmbeddingDimFromEnv reads MEMORY_ONNX_MODEL_PATH and returns the matching dimension.

Types

type BatchCollectionError 

type BatchCollectionError struct {
	Failures []CollectionCreateResult
}

func (*BatchCollectionError) Error 

func (e *BatchCollectionError) Error() string

func (*BatchCollectionError) Is 

func (e *BatchCollectionError) Is(target error) bool

func (*BatchCollectionError) Unwrap 

func (e *BatchCollectionError) Unwrap() []error

type BatchEmbeddingFunc 

type BatchEmbeddingFunc func(texts []string, dim int) ([][]float32, error)

BatchEmbeddingFunc embeds multiple texts in one forward pass when supported (e.g. GONNXEmbedder).

type CollectionCreateResult 

type CollectionCreateResult struct {
	Name  string
	Error error
}

type CompactionAction 

type CompactionAction struct {
	Action    string
	TargetIDs []string
	Reason    string
}

type CompactionConfig 

type CompactionConfig struct {
	Tier2Strategy      CompactionStrategy `json:"tier2_strategy"`
	Tier3Strategy      CompactionStrategy `json:"tier3_strategy"`
	LastEvaluatedCycle int                `json:"last_evaluated_cycle"`
	AvgScoreBefore     float64            `json:"avg_score_before"`
	AvgScoreAfter      float64            `json:"avg_score_after"`
	Improvement        float64            `json:"improvement"`
	// H-Mem inspired
	TemporalWindowSize  int     `json:"temporal_window_size"` // beta-like (e.g. cycles or months)
	SimilarityThreshold float64 `json:"similarity_threshold"` // alpha
	// RecMem phase-transition (Phase 1)
	DataSim   float64 `json:"data_sim"`   // geometric similarity radius (default 0.7)
	DataCount int     `json:"data_count"` // critical recurrence count (default 5)
	// LongMemEval production hardening
	ProtectHighScoreFacts bool    `json:"protect_high_score_facts"` // default true
	FactScoreThreshold    float64 `json:"fact_score_threshold"`     // default 0.90
}

CompactionConfig (extended with RecMem phase-transition parameters)

type CompactionStrategy 

type CompactionStrategy string

CompactionStrategy types (from ossa, kept for compatibility) 

const (
	StrategySimpleSummary     CompactionStrategy = "simple_summary"
	StrategyPatternExtraction CompactionStrategy = "pattern_extraction"
	StrategyCorePrinciple     CompactionStrategy = "core_principle"
)

type EmbeddingFunc 

type EmbeddingFunc func(text string, dim int) []float32

EmbeddingFunc is injectable for semantic embeddings (Phase 5.1)

func NewGONNXEmbeddingFunc 

func NewGONNXEmbeddingFunc(modelPath string) (EmbeddingFunc, error)

NewGONNXEmbeddingFunc returns a Palace-compatible EmbeddingFunc backed by pure-Go ONNX. modelPath may be empty to use GenerateSimpleEmbedding (dev/tests). modelPath may be a hugot model directory or a direct path to a single .onnx file.

func NewGONNXEmbeddingFuncFromEnv 

func NewGONNXEmbeddingFuncFromEnv() (EmbeddingFunc, error)

NewGONNXEmbeddingFuncFromEnv loads MEMORY_ONNX_MODEL_PATH when set; otherwise hash embeddings.

type EntityGraph 

type EntityGraph struct {
	Entities map[string][]string `json:"entities"`
}

EntityGraph for H-Mem KG integration (richer relational graph)

type FactsAsOfOptions 

type FactsAsOfOptions struct {
	// AsOf is the validity instant (zero = Now UTC).
	AsOf time.Time
	// Query optional case-insensitive substring on Summary / Full / OriginalText.
	Query string
	// SessionID, when non-empty, keeps matching SessionID or conv:<SessionID> tags (T1).
	SessionID string
	// SessionIDs, when non-empty, is an any-of filter (union with SessionID).
	SessionIDs []string
	// Entity filters TemporalTags:
	//   - if Entity contains ':', exact match on "entity:<value>" (or Entity itself
	//     when it already has the "entity:" prefix);
	//   - otherwise any TemporalTag with prefix "entity:" that contains Entity.
	Entity string
	// Limit caps results (default 50 when <= 0).
	Limit int
	// Tier when non-nil: only that tier. When nil: Working+Contextual+Semantic
	// (exclude Archival unless IncludeArchival).
	Tier *MemoryTier
	// IncludeArchival, when true and Tier==nil, also includes Archival.
	IncludeArchival bool
}

FactsAsOfOptions configures as-of validity listing (K4 first slice / s616).

Filters apply before Limit so many invalid-at-asOf entries do not underfill.

type GONNXEmbedder 

type GONNXEmbedder struct {
	// contains filtered or unexported fields
}

GONNXEmbedder runs sentence embeddings via hugot FeatureExtractionPipeline. Call Close when done to release the hugot session.

func NewGONNXEmbedder 

func NewGONNXEmbedder(opts GONNXOptions) (*GONNXEmbedder, error)

NewGONNXEmbedder loads a hugot-compatible ONNX model directory and starts a feature-extraction pipeline.

func (*GONNXEmbedder) Backend 

func (e *GONNXEmbedder) Backend() string

Backend returns the active hugot backend label (gomlx, ort+cpu, ort+cuda:0, ort+coreml).

func (*GONNXEmbedder) BatchFunc 

func (e *GONNXEmbedder) BatchFunc() BatchEmbeddingFunc

BatchFunc returns a Palace-compatible batch embedding closure for SearchMemory batch scoring.

func (*GONNXEmbedder) Close 

func (e *GONNXEmbedder) Close() error

Close releases hugot session resources.

func (*GONNXEmbedder) Dimension 

func (e *GONNXEmbedder) Dimension() int

Dimension returns the native embedding width produced by the loaded ONNX model.

func (*GONNXEmbedder) Embed 

func (e *GONNXEmbedder) Embed(text string) ([]float32, error)

Embed returns an L2-normalized embedding for text.

func (*GONNXEmbedder) EmbedBatch 

func (e *GONNXEmbedder) EmbedBatch(texts []string) ([][]float32, error)

EmbedBatch returns L2-normalized embeddings for all texts in one pipeline forward pass.

func (*GONNXEmbedder) Func 

func (e *GONNXEmbedder) Func() EmbeddingFunc

Func returns an EmbeddingFunc that preserves the PalaceConfig injectable signature. The returned closure keeps the embedder alive; callers should not discard the embedder until all embedding work is finished (or use Func() only via NewGONNXEmbeddingFunc).

type GONNXOptions 

type GONNXOptions struct {
	ModelPath string
	// Strict disables silent fallback to GenerateSimpleEmbedding on inference errors.
	Strict bool
	// HugotBackend overrides MEMORY_HUGOT_BACKEND for this embedder (go, ort, auto).
	HugotBackend HugotBackendConfig
}

GONNXOptions configures a hugot ONNX embedder (GoMLX by default; ORT when built with -tags ORT).

type HugotBackendConfig 

type HugotBackendConfig struct {
	Backend    string // go, ort, auto
	LibraryDir string
	Cuda       bool
	CoreML     bool
	DeviceID   string
}

HugotBackendConfig holds resolved hugot session options from the environment.

func ResolveHugotBackendFromEnv 

func ResolveHugotBackendFromEnv() HugotBackendConfig

ResolveHugotBackendFromEnv reads MEMORY_HUGOT_BACKEND and ORT accelerator env vars.

func (HugotBackendConfig) HugotBackendLabel 

func (c HugotBackendConfig) HugotBackendLabel() string

HugotBackendLabel returns a log-friendly backend description.

type ListMemoryOptions 

type ListMemoryOptions struct {
	SessionID string
	// SessionIDs, when non-empty, is an any-of filter (union with SessionID).
	// conv:<id> tags match the same way as SearchMemoryWithOptions (T1).
	SessionIDs []string
	// TimeFrom / TimeTo filter by entry event time (see entryEventTime). Both inclusive when set.
	TimeFrom *time.Time
	TimeTo   *time.Time
	// Tag exact-matches any of TemporalTags or Content.Tags.
	Tag string
	// TagPrefix matches tag strings with strings.HasPrefix (e.g. "subject:", "session_seq:").
	TagPrefix string
	// Query optional case-insensitive substring on Summary / Full / OriginalText.
	Query string
	// Limit caps results (default 50 when <= 0 — timeline-friendly; not search's 10).
	Limit int
	// Tier when non-nil: only that tier. When nil: Working+Contextual+Semantic
	// (exclude Archival by default to match the public MCP host timeline).
	Tier *MemoryTier
	// IncludeArchival, when true and Tier==nil, also includes Archival.
	IncludeArchival bool
	// Ascending: false = newest first (default); true = oldest first.
	Ascending bool
}

ListMemoryOptions configures event-time ordered listing with optional filters (K2 / s611).

Filters are applied before Limit so windowed/session timelines do not underfill when many out-of-scope entries exist (same underfill class as SearchMemoryWithOptions K1).

Complexity: FS Palace is source of truth. When the best-effort meta index is enabled (default), ListMemoryWithOptions filters on lightweight entryMeta and loads full JSON only for survivors (s1066). Write/unlink patches a clean index in place (#63). A durable snapshot at indexes/event-time.json (#44) avoids re-parsing every tier JSON on a fresh process when the stamp matches. With DisableMetaIndex, falls back to O(n) full entry scan.

type MemoryContent 

type MemoryContent struct {
	Summary        string    `json:"summary"`
	Full           string    `json:"full,omitempty"`
	Tags           []string  `json:"tags,omitempty"`
	Embedding      []float32 `json:"embedding,omitempty"`
	EmbeddingModel string    `json:"embedding_model,omitempty"`
	EmbeddingDim   int       `json:"embedding_dim,omitempty"`
}

MemoryContent holds summary/full/tags and optional persisted ONNX vectors. Embedding fields are omitted unless PersistEmbeddings is on and the model is a non-hash id. Hash vectors are never stored (kernel #45).

type MemoryEntry 

type MemoryEntry struct {
	ID           string     `json:"id"`
	Type         string     `json:"type"`
	Tier         MemoryTier `json:"tier"`
	Version      int        `json:"version"`
	CreatedAt    time.Time  `json:"created_at"`
	UpdatedAt    time.Time  `json:"updated_at"`
	Cycle        int        `json:"cycle"`
	TemporalTags []string   `json:"temporal_tags,omitempty"`
	AccessCount  int        `json:"access_count"`
	LastAccessed time.Time  `json:"last_accessed,omitempty"`

	// === LongMemEval / turn-level granularity extensions (production implementation) ===
	TurnID         string    `json:"turn_id,omitempty"`         // explicit round/turn identifier
	SessionID      string    `json:"session_id,omitempty"`      // multi-session grouping
	Timestamp      time.Time `json:"timestamp,omitempty"`       // precise event time for temporal reasoning
	ExtractedFacts []string  `json:"extracted_facts,omitempty"` // fact-augmented for better recall
	Keyphrases     []string  `json:"keyphrases,omitempty"`      // keyphrase expansion for indexing
	OriginalText   string    `json:"original_text,omitempty"`   // raw turn text for provenance

	Content    MemoryContent    `json:"content"`
	Provenance MemoryProvenance `json:"provenance"`
	Metrics    MemoryMetrics    `json:"metrics"`
	Relations  MemoryRelations  `json:"relations"`
}

MemoryEntry is the core unit stored in the Palace. Extended for LongMemEval production readiness: explicit turn/session granularity + fact-augmented indexing.

type MemoryMetrics 

type MemoryMetrics struct {
	ScoreImpact  float64   `json:"score_impact,omitempty"`
	UsageCount   int       `json:"usage_count"`
	LastAccessed time.Time `json:"last_accessed,omitempty"`
}

MemoryMetrics for scoring and access

type MemoryProvenance 

type MemoryProvenance struct {
	SourceCycle int    `json:"source_cycle"`
	SourceStep  string `json:"source_step,omitempty"`
	// SourceHint is the cite-both source class (private|mesh|…). IngestTurn
	// defaults this to "private" on the local-palace path when the caller
	// does not already supply a classifiable hint or tag. Distinct from
	// SourceStep (process name such as mcp_memory_ingest_turn).
	SourceHint string   `json:"source_hint,omitempty"`
	ParentIDs  []string `json:"parent_ids,omitempty"`
	ToolCalls  []string `json:"tool_calls,omitempty"`
}

MemoryProvenance tracks origin

type MemoryRelations 

type MemoryRelations struct {
	ImprovesUpon    []string `json:"improves_upon,omitempty"`
	RelatedConcepts []string `json:"related_concepts,omitempty"`
	Backlinks       []string `json:"backlinks,omitempty"`
}

MemoryRelations for graph links

type MemoryStats 

type MemoryStats struct {
	WorkingCount    int
	ContextualCount int
	ArchivalCount   int
	SemanticCount   int // Phase 3
	TotalEntries    int
	LastCompaction  time.Time
}

MemoryStats provides observability metrics

type MemoryTier 

type MemoryTier int

MemoryTier defines the three-tier hierarchical memory (inspired by ossa Palace + H-Mem ideas) 

const (
	TierWorking    MemoryTier = 1
	TierContextual MemoryTier = 2
	TierArchival   MemoryTier = 3
	// TierSemantic is used for high-fidelity atomic facts protected by RecMem Phase 3
	TierSemantic MemoryTier = 4
)

type MultiHopOptions 

type MultiHopOptions struct {
	// SeedEntity is the starting entity key (normalized). Prefer exact graph node keys.
	SeedEntity string
	// SeedQuery optional: if set, run SearchMemoryWithOptions with Query then derive
	// entity seeds from top hits' TemporalTags entity:* and RelatedConcepts
	// (combined with SeedEntity when both are set).
	SeedQuery string
	// MaxHops default 2; clamped to 1..4. Hop 0 is the seed itself.
	MaxHops int
	// Limit default 20; applied AFTER expansion + entry collect + filters.
	Limit int
	// SessionID, when non-empty, keeps matching SessionID or conv:<SessionID> tags (T1).
	SessionID string
	// SessionIDs, when non-empty, is an any-of filter (union with SessionID).
	SessionIDs []string
	// AsOf optional EntryValidAt filter (before Limit).
	AsOf *time.Time
	// Tier when non-nil: only that tier. When nil: Working+Contextual+Semantic
	// (exclude Archival unless IncludeArchival).
	Tier *MemoryTier
	// IncludeArchival, when true and Tier==nil, also includes Archival.
	IncludeArchival bool
	// QueryVec optional pass-through when seeding via SeedQuery search.
	QueryVec []float32
	// PreferShorterHops ranks by minimum BFS hop distance from seed (lower first),
	// then event time descending within the same hop. Default true (nil or true).
	// Set to a false pointer to opt out and use legacy seed-match-first sort.
	PreferShorterHops *bool
}

MultiHopOptions configures associative / multi-hop lite retrieval over the EntityGraph plus entry entity tags (A2 first slice / s619; hop ranking s1067; residual pin s1278).

BFS over GetRelatedEntities and entry collect via TemporalTags / Content.Tags / RelatedConcepts. Not a full Zep / Graphiti knowledge graph. Hop-distance ranking is path-aware ranking lite (prefer shorter BFS hops), not typed-edge / embedding-guided path scoring. Not a full graph RAG.

type PalaceConfig 

type PalaceConfig struct {
	// BaseDir is the local palace root. Empty uses DefaultPalaceBaseDir (".palace").
	BaseDir            string
	MaxWorkingEntries  int
	MaxWorkingAgeHours int
	CompactionConfig   CompactionConfig
	EmbeddingFunc      EmbeddingFunc      `json:"-"` // pluggable (Phase 5.1)
	BatchEmbeddingFunc BatchEmbeddingFunc `json:"-"` // optional ONNX batch path (Phase 5.1)
	// PersistEmbeddings stores ONNX vectors on MemoryContent when true.
	// Default false: Write does not persist embeddings. Hash models
	// (EmbeddingModel empty or "hash") are never persisted even when true.
	PersistEmbeddings bool
	// EmbeddingModel is "hash" or empty for GenerateSimpleEmbedding; ONNX
	// callers set a model id (e.g. "bge-small-en-v1.5"). Persist and search
	// match on this string — not EmbeddingFunc pointer identity.
	EmbeddingModel string
	// EmbeddingDim is the expected vector width. 0 infers from EmbeddingFunc
	// output or DefaultHashEmbeddingDim.
	EmbeddingDim int
	// DisableMetaIndex forces ListMemoryWithOptions to use the full FS scan path
	// instead of the best-effort in-memory metadata index (K2 residual / s1066).
	// Default false (index enabled). Useful for parity tests.
	DisableMetaIndex bool
	// DisableDurableIndex skips load/save of indexes/event-time.json.
	// The in-memory meta index still runs unless DisableMetaIndex is set.
	// Default false (durable snapshot enabled). FS Palace remains source of truth.
	DisableDurableIndex bool
}

PalaceConfig for configurable PalaceStore creation (Phase 4.3)

type PalaceStore 

type PalaceStore struct {
	BaseDir string
	Config  PalaceConfig
	// contains filtered or unexported fields
}

PalaceStore provides file-backed hierarchical memory storage.

func NewPalaceStore 

func NewPalaceStore(baseDir string) *PalaceStore

NewPalaceStore is a convenience constructor (uses defaults). Empty baseDir defaults to DefaultPalaceBaseDir.

func NewPalaceStoreWithConfig 

func NewPalaceStoreWithConfig(cfg PalaceConfig) *PalaceStore

NewPalaceStoreWithConfig creates PalaceStore with full configuration (Phase 4.3). Empty BaseDir defaults to DefaultPalaceBaseDir (".palace") under the process cwd. EmbeddingModel is left empty (hash default) unless the caller set it. PersistEmbeddings stays false unless the caller set it.

func (*PalaceStore) AddEntityRelationship 

func (ps *PalaceStore) AddEntityRelationship(entity, related string)

AddEntityRelationship adds normalized entity links (H-Mem style). Ensures BaseDir/relations exists before write (safe if ensureDirs was skipped or the directory was removed). Shared graph rewrite is serialized on writeMu and written via temp+rename at 0600 (#86).

func (*PalaceStore) AutoRecMemCompaction 

func (ps *PalaceStore) AutoRecMemCompaction(generateFn func(prompt string) string, vectorCb VectorStoreCallback)

AutoRecMemCompaction is the production entry point for automatic RecMem formation.

func (*PalaceStore) EvictWorkingTier 

func (ps *PalaceStore) EvictWorkingTier(maxAgeHours int, maxCount int)

EvictWorkingTier performs age/size-based eviction from Working tier (Phase 4.2)

func (*PalaceStore) ExpandRelatedEntities 

func (ps *PalaceStore) ExpandRelatedEntities(seed string, maxHops int) []string

ExpandRelatedEntities BFS from seed over GetRelatedEntities up to maxHops (includes seed at hop 0). maxHops is clamped to 1..4.

Returns unique entity keys in BFS discovery order (seed first).

func (*PalaceStore) ExpandRelatedEntitiesHops 

func (ps *PalaceStore) ExpandRelatedEntitiesHops(seed string, maxHops int) map[string]int

ExpandRelatedEntitiesHops BFS from seed over GetRelatedEntities up to maxHops (includes seed at hop 0). maxHops is clamped to 1..4.

Returns entity key → minimum hop distance from seed. Empty seed → nil.

func (*PalaceStore) GetRelatedEntities 

func (ps *PalaceStore) GetRelatedEntities(entity string) []string

GetRelatedEntities returns related entities for graph traversal

func (*PalaceStore) GetStats 

func (ps *PalaceStore) GetStats() MemoryStats

GetStats returns observability metrics (Phase 1.2)

func (*PalaceStore) IngestTurn 

func (ps *PalaceStore) IngestTurn(turn MemoryEntry) error

IngestTurn is the primary production entry point for LongMemEval benchmark and ego online session processing. It writes the parent turn first, then each non-empty ExtractedFacts child as type turn_fact. When ExtractedFacts is empty, facts are auto-extracted from the turn text.

Child Content.Tags inherit the parent turn's Content.Tags (trimmed, de-duplicated) and always include the structural markers fact_augmented and from_turn. The kernel does not stamp longmemeval; callers that want that label (the LongMemEval harness under cmd/longmemeval-* and internal/longmemeval) pass it on the parent so children inherit it.

Local-palace IngestTurn stamps an observable private source class when the caller does not already provide a classifiable source hint or tag: Provenance.SourceHint is set to "private" and Content.Tags gains source_hint:private. Mesh-class hints (source_hint:mesh, source:mesh, …) stay distinct and are not overwritten. Host process labels such as mcp_memory_ingest_turn / source:iomesh-memory-mcp are not a cite-both class. Fact children inherit SourceHint and the source_hint:* tag.

Fact-augmented children get valid_from stamped when unset; child Write errors are returned.

Partial persist is the contract: a child Write error does not roll back the parent or earlier facts already written. A non-nil error does not mean nothing persisted. Not all-or-nothing.

func (*PalaceStore) InvalidateMetaIndex 

func (ps *PalaceStore) InvalidateMetaIndex()

InvalidateMetaIndex is the exported test/debug hook for force-rebuild.

func (*PalaceStore) ListEntriesInTier 

func (ps *PalaceStore) ListEntriesInTier(tier MemoryTier) []MemoryEntry

ListEntriesInTier returns all entries in the given tier, sorted by relevance score (descending).

func (*PalaceStore) ListFactsAsOf 

func (ps *PalaceStore) ListFactsAsOf(opts FactsAsOfOptions) []MemoryEntry

ListFactsAsOf lists entries valid at AsOf with optional filters. Filters apply before Limit (underfill class, same as K1/K2).

Order of operations: collect candidates → session → entity → query → EntryValidAt(asOf) → sort (Semantic first, then event time desc) → limit.

Default tiers when Tier == nil: Working + Contextual + Semantic (+ Archival if IncludeArchival).

Bi-temporal lite (validity window tags). Not full Graphiti dual clocks + graph. FS Palace remains O(n) over tier files.

func (*PalaceStore) ListMemoryWithOptions 

func (ps *PalaceStore) ListMemoryWithOptions(opts ListMemoryOptions) []MemoryEntry

ListMemoryWithOptions returns entries ordered by event time with optional filters applied before Limit (K2 / s611 timeline surface; s1066 meta index when enabled).

Order of operations: collect candidates → session → time → tag filters → query substring → sort by entryEventTime → limit.

func (*PalaceStore) Load 

func (ps *PalaceStore) Load(id string, tier MemoryTier) (MemoryEntry, bool)

Load retrieves by ID and tier. Production: no side-effect prints, clean bool return.

func (*PalaceStore) MetaIndexLen 

func (ps *PalaceStore) MetaIndexLen() int

MetaIndexLen returns the number of entries currently cached in the meta index (0 if dirty/unbuilt). Intended for tests and observability — not a product API.

func (*PalaceStore) MetaIndexRebuilds 

func (ps *PalaceStore) MetaIndexRebuilds() uint64

MetaIndexRebuilds returns how many full tier-JSON walks rebuildMetaIndexLocked has run on this store. Tests use this to prove Write/unlink patched in place. Not a product API.

func (*PalaceStore) MultiHopRetrieve 

func (ps *PalaceStore) MultiHopRetrieve(opts MultiHopOptions) []MemoryEntry

MultiHopRetrieve performs multi-hop lite associative retrieval:

  1. Resolve seeds (SeedEntity and/or entities from SeedQuery search hits)
  2. ExpandRelatedEntitiesHops for each seed (min hop across seeds)
  3. Collect entries from default tiers matching any expanded entity
  4. Optional AsOf + SessionID filters BEFORE Limit
  5. Sort: lower min hop first (default), then event time desc within hop (legacy: seed-match first when PreferShorterHops is explicitly false)
  6. Limit

Default MaxHops=2 (clamped 1..4), Limit=20. Default tiers: Working + Contextual + Semantic (+ Archival if IncludeArchival). PreferShorterHops defaults true (path-aware ranking lite / s1067).

func (*PalaceStore) PerformCompaction 

func (ps *PalaceStore) PerformCompaction(
	targetTier MemoryTier,
	cfg CompactionConfig,
	generateFn func(prompt string) string,
	vectorCallback VectorStoreCallback,
) error

PerformCompaction runs agent-managed compaction with H-Mem temporal window + alpha constraints. Now respects LongMemEval fact protection and turn granularity. Product Write errors from SUMMARIZE / MERGE / CREATE_CORE_PRINCIPLE / ARCHIVE are returned.

func (*PalaceStore) PromoteToContextual 

func (ps *PalaceStore) PromoteToContextual(threshold float64)

PromoteToContextual promotes high-relevance Working entries (Phase 4.2)

func (*PalaceStore) ReadWithChainOfNote 

func (ps *PalaceStore) ReadWithChainOfNote(retrieved []MemoryEntry, query string) (string, error)

ReadWithChainOfNote formats retrieved MemoryEntry items into a Chain-of-Note style prompt.

func (*PalaceStore) SearchMemory 

func (ps *PalaceStore) SearchMemory(query string, tier *MemoryTier, limit int, vec []float32) []MemoryEntry

SearchMemory provides hybrid retrieval (keyword + vector + temporal) - Phase 4.1. Thin wrapper over SearchMemoryWithOptions with no session/time filters and ReRankTemporal=false. When tier is nil, default tiers skip Archival (Working+Contextual+Semantic) unless the query has no keyword hits on those tiers (low-confidence fallback; #87).

func (*PalaceStore) SearchMemoryWithOptions 

func (ps *PalaceStore) SearchMemoryWithOptions(query string, opts SearchMemoryOptions) []MemoryEntry

SearchMemoryWithOptions provides hybrid retrieval with optional session, time-window, and temporal re-ranking. Uses the configured EmbeddingFunc for vector re-ranking. A persisted Content.Embedding is reused when EmbeddingModel and dim match the query vec / store config; otherwise the entry is re-embedded. Keyword hits still rank ahead of Limit.

Default tiers when Tier==nil match ListMemoryWithOptions: Working+Contextual+Semantic. Archival is included when IncludeArchival is set, or when the default-tier keyword hit set is empty (low-confidence fallback). That fallback is the empty keyword-hit set, not a numeric cosine cutoff (Memory P0: do not invent a threshold). Count queries (`how many` / `how much`) union matching turn_facts from the session/`conv:` candidate set, rank named-pattern facts above fallback chatter, then session-diversify before Limit.

func (*PalaceStore) SemanticRefine 

func (ps *PalaceStore) SemanticRefine(cluster []MemoryEntry) error

SemanticRefine (RecMem Phase 3) protects high-stake atomic facts from clusters. Products get SessionID / Timestamp from the parent and valid_from when unset.

func (*PalaceStore) SupersedeEntityFacts 

func (ps *PalaceStore) SupersedeEntityFacts(entityKey string, asOf time.Time) (int, error)

SupersedeEntityFacts finds entries matching entityKey (via EntryEntityKeys / entity: tags) that are still open at asOf (EntryValidAt), and writes valid_until = asOf (exclusive end matching existing FactsAsOf semantics). Does not delete entries.

Matching: normalize entityKey to lower-case trim; an entry matches when any key from EntryEntityKeys equals that normalized form (also lower-cased). Empty entityKey is a no-op (returns 0, nil).

Bi-temporal lite supersession — not automatic NLP contradiction detection, not full Zep dual-clock KG. Callers pass explicit entity keys.

Returns the count of updated entries.

func (*PalaceStore) Write 

func (ps *PalaceStore) Write(entry MemoryEntry) error

Write persists a MemoryEntry with an atomic write (temp file + rename).

Versioning is caller-managed and best-effort. Version==0 is stored as 1. archiveToVersions writes versions/memory-entries/<id>/v{Version}.json for the incoming entry; its errors are non-fatal and do not fail Write. A second Write of the same ID and Version overwrites both the live tier file and that snapshot. Callers who want history must increment Version themselves. This is not automatic overwrite versioning and not a hosted version store.

PersistEmbeddings default false strips embedding fields before marshal (including stuffed floats). Hash models (EmbeddingModel empty or "hash") never persist vectors, even when the flag is true (kernel #45). When the flag is on and EmbeddingModel is a non-hash id, the JSON rename is the ingest ack; embedding is then filled best-effort. Embed miss, empty output, or panic is not an ingest failure.

func (*PalaceStore) WriteAndSupersede 

func (ps *PalaceStore) WriteAndSupersede(entry MemoryEntry, supersedeKeys []string) error

WriteAndSupersede writes entry first (stamping valid_from=now when unset), then runs SupersedeEntityFacts for each supersedeKeys entry, excluding the newly written entry's ID so it is not closed by its own write.

asOf for supersession is the same UTC now used for valid_from stamping.

func (*PalaceStore) WriteLatent 

func (ps *PalaceStore) WriteLatent(entry MemoryEntry) error

WriteLatent stores entry in the subconscious latent buffer (RecMem Phase 1). No immediate promotion or LLM compaction is triggered.

Versioning is caller-managed and best-effort, same as Write: Version==0 is stored as 1; archiveToVersions errors are non-fatal; callers who want history must increment Version themselves. Overwrite does not auto-increment.

type SearchMemoryOptions 

type SearchMemoryOptions struct {
	// SessionID, when non-empty, keeps entries whose SessionID matches or that
	// carry tag conv:<SessionID> (T1: inner haystack sessions under one conv).
	SessionID string
	// SessionIDs, when non-empty, is an any-of filter (union with SessionID).
	SessionIDs []string
	// TimeFrom / TimeTo filter by entry event time (see entryEventTime).
	// Both bounds are inclusive when set.
	TimeFrom *time.Time
	TimeTo   *time.Time
	// AsOf, when non-nil, drops candidates where !EntryValidAt(e, *AsOf) before Limit
	// (K4 facts-as-of / validity window; see EntryValidAt).
	AsOf *time.Time
	// Limit caps the result set (default 10 when <= 0).
	Limit int
	// Tier, when non-nil, restricts candidates to that tier.
	Tier *MemoryTier
	// QueryVec, when non-empty, ranks candidates by cosine similarity.
	// Keyword hits (if the query has tokens of length >= 3) are kept ahead of
	// non-hits so hash embeddings cannot drop a literal match past Limit.
	QueryVec []float32
	// ReRankTemporal, when true, sorts results by CalculateRelevanceScore descending
	// after the keyword/vector path (before Limit). Keyword hits stay ahead of
	// non-hits so temporal re-rank cannot drop a literal match past Limit.
	ReRankTemporal bool
	// IncludeArchival, when true and Tier==nil, also includes Archival.
	// Default retrieve is Working+Contextual+Semantic (same as ListMemoryWithOptions).
	IncludeArchival bool
}

SearchMemoryOptions configures hybrid retrieval with optional session, time-window, as-of validity, and temporal re-ranking filters.

type SearchResult 

type SearchResult struct {
	ID      string
	Score   float64
	Payload map[string]interface{}
}

SearchResult represents a single vector search hit with score and payload.

type VectorStore 

type VectorStore struct {
	Client     *qdrant.Client
	Collection string
	Enabled    bool
}

VectorStore provides optional vector capabilities using official Qdrant Go client v1 Production implementation with full dense/sparse query support, proper error handling, and payload extraction.

func NewVectorStore 

func NewVectorStore(rawURL, collection string) *VectorStore

NewVectorStore initializes the official Qdrant client.

func (*VectorStore) BatchUpsert 

func (vs *VectorStore) BatchUpsert(points []*qdrant.PointStruct) error

BatchUpsert upserts multiple points at once

func (*VectorStore) CreateBatchSparseCollections 

func (vs *VectorStore) CreateBatchSparseCollections(names []string) error

func (*VectorStore) CreateBatchSparseCollectionsContext 

func (vs *VectorStore) CreateBatchSparseCollectionsContext(ctx context.Context, names []string) error

func (*VectorStore) CreateBatchSparseCollectionsWithConcurrency 

func (vs *VectorStore) CreateBatchSparseCollectionsWithConcurrency(names []string, concurrency int) error

func (*VectorStore) CreateBatchSparseCollectionsWithConcurrencyContext 

func (vs *VectorStore) CreateBatchSparseCollectionsWithConcurrencyContext(ctx context.Context, names []string, concurrency int) error

CreateBatchSparseCollectionsWithConcurrencyContext is the context-aware version with custom concurrency.

func (*VectorStore) CreateBatchSparseCollectionsWithResults 

func (vs *VectorStore) CreateBatchSparseCollectionsWithResults(ctx context.Context, names []string) ([]CollectionCreateResult, error)

func (*VectorStore) CreateBatchSparseCollectionsWithResultsAndConcurrency 

func (vs *VectorStore) CreateBatchSparseCollectionsWithResultsAndConcurrency(ctx context.Context, names []string, concurrency int) ([]CollectionCreateResult, error)

func (*VectorStore) CreateCollection 

func (vs *VectorStore) CreateCollection(dim int) error

CreateCollection creates a dense vector collection with cosine distance.

func (*VectorStore) CreateSparseCollection 

func (vs *VectorStore) CreateSparseCollection() error

CreateSparseCollection creates a sparse vector collection.

func (*VectorStore) SearchByText 

func (vs *VectorStore) SearchByText(text string, limit int, filter map[string]interface{}, withPayload bool) ([]SearchResult, error)

SearchByText performs text-based semantic search.

func (*VectorStore) SearchSimilar 

func (vs *VectorStore) SearchSimilar(queryVec []float32, limit int, filter map[string]interface{}, withPayload bool) ([]SearchResult, error)

SearchSimilar performs dense vector similarity search using the official Qdrant Query API.

func (*VectorStore) SearchSparse 

func (vs *VectorStore) SearchSparse(queryIndices []uint32, queryValues []float32, limit int, filter map[string]interface{}, withPayload bool) ([]SearchResult, error)

SearchSparse performs sparse vector similarity search using the official Qdrant Query API.

func (*VectorStore) StoreSparseVector 

func (vs *VectorStore) StoreSparseVector(id string, indices []uint32, values []float32, payload map[string]interface{}) error

StoreSparseVector upserts a sparse vector with payload

func (*VectorStore) StoreVector 

func (vs *VectorStore) StoreVector(id string, vec []float32, payload map[string]interface{}) error

StoreVector upserts a dense vector with payload

type VectorStoreCallback 

type VectorStoreCallback func(id string, vec []float32, payload map[string]interface{}) error

VectorStoreCallback allows optional vector integration (e.g. Qdrant)

Source Files

View all Source files

Directories

Path Synopsis
cmd
longmemeval-bench command
longmemeval-server command
longmemeval-v2-bench command
two-process-writer-probe command
Two-process writer probe child: one PalaceStore process against a shared root.
 Two-process writer probe child: one PalaceStore process against a shared root.
examples
ttfh_rca command
TTFH-shaped walking skeleton for the palace kernel.
 TTFH-shaped walking skeleton for the palace kernel.
internal
longmemeval
onnxdownload
Package onnxdownload is the fail-soft helper for scripts/download_onnx_model.go.
 Package onnxdownload is the fail-soft helper for scripts/download_onnx_model.go.
writerprobe
Package writerprobe is last-write-wins evidence for two OS processes on one palace root.
 Package writerprobe is last-write-wins evidence for two OS processes on one palace root.
Download the default KnightsAnalytics bge-small-en-v1.5 ONNX model for local Palace recall.
 Download the default KnightsAnalytics bge-small-en-v1.5 ONNX model for local Palace recall.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.