store

package
v0.10.0 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Aug 25, 2026 License: MIT Imports: 7 Imported by: 0

Documentation

Overview

FTS read surface: the two keyword-recall queries (flat hits and anchor rows) over messages_fts + messages + sessions, with dynamic filter composition. MATCH-expression building (OR-rewrite, sanitizing) stays in the retrieve layer — store receives the finished FTS5 expression and owns only the SQL.

Messages read surface: the typed queries over the messages table (and its sessions join) that the view / agentproto / semantic / cli layers previously issued as inline SQL. Ordering within a session is by message id (insertion order), NOT ts — ts can be non-monotonic, so id is the reliable ordering key.

Sessions read surface: the typed queries over the sessions table that the view / agentproto / retrieve / cli layers previously issued as inline SQL. Every method reproduces its consumer site's WHERE/ORDER BY/LIMIT semantics exactly, so moving a consumer onto them is behavior-preserving.

Corpus stats surface (D6): session counts and aggregate stats over one indexed project's db, moved verbatim from internal/index. These open their own read-only connection from a db path (they are whole-db aggregates, not per-connection reads).

Package store owns the SQLite schema (base + FTS + topic sidecar DDL, the schema-version gates) and the connection helpers for the per-scope index dbs. It sits at the bottom of the index seam: it imports no other internal package, so schema text and connection policy have a single, dependency-free home. Pure-Go via modernc.org/sqlite (no cgo).

Topic query surface: the topic_segment / topic_fts SQL, moved verbatim from internal/index so the sidecar's table/column names live beside its DDL (EnsureTopicSchema in store.go). Ingest orchestration stays in index.

Vectors surface (D4): the chunk_vec table's DDL and row-level SQL. The semantic package keeps hashing, embedding, cosine KNN, and RRF fusion — store owns only the SQL so table/column names live in one package. Vectors are keyed by (session_id, content_hash) so msg-id churn on reindex is harmless; the table sits under its OWN schema gate and is NEVER in the keyword Rebuild() drop list, so a keyword reindex can't nuke vectors.

Session-verdict surface: the per-session `routine` verdict + its floor|agent source, kept in the topic sidecar (session_verdict, gated by TopicSchemaVersion). A verdict is a surfacing signal only — never a truth or importance claim on the transcript (raw stays raw). Downstream consumers use it for sort-tiering and for the routine-fed delete plan.

Index

Constants

View Source 
const (
	VerdictSourceFloor = "floor"
	VerdictSourceAgent = "agent"
)

Verdict source values. `floor` = the deterministic math floor; `agent` = an LLM tagger's explicit call. `source` is load-bearing for the provenance-gated delete plan.

View Source 
const FTSSQL = `` /* 518-byte string literal not displayed */

FTSSQL is the FTS5 virtual table + sync triggers (contentful/inline + porter).

View Source 
const ROMmapSize = 1 << 28 // 256 MiB read-only mmap window

ROMmapSize is the memory-mapped I/O size for read-only connections.

View Source 
const Schema = `` /* 1252-byte string literal not displayed */

Schema is the base (non-FTS) DDL. The sessions provenance/retention columns (origin_machine/source_tool/source_path/only_copy_since) and the scope columns (project/cwd) are present here so a fresh or rebuilt db carries them from the start; an existing current-version db gets them via index's in-place migrateDurabilityColumns / migrateScopeColumns migrations.

project/cwd make a session's scope readable from the ROW rather than inferred from which per-project db file it lives in. Today that inference is exact (one db per project), which is precisely why the filename is load-bearing and a shared store is impossible; carrying the scope on the row is the prefactor that removes the dependency. origin_machine already answers "which machine", so the row triple is (project, cwd, origin_machine).

View Source 
const SchemaVersion = 4

SchemaVersion gates a full rebuild on mismatch. It is deliberately NOT bumped for the durable-retention columns (origin_machine/source_tool/source_path/ only_copy_since) nor for the scope columns (project/cwd): a bump forces Rebuild() to re-walk the live tree and re-prune every already-retained session, defeating retention on the first upgrade. Those columns are added in place by index's migrateDurabilityColumns / migrateScopeColumns instead.

View Source 
const TopicSchemaVersion = 2

TopicSchemaVersion gates the topic sidecar tables separately from the keyword schema — like VecSchemaVersion, it is its OWN gate and is NEVER in Schema/FTSSQL/dropSQL, so a keyword reindex can't nuke topic rows. Topic rows are keyed by the source-stable message uuid (start_uuid/end_uuid), so they re-map losslessly after a base reindex churns the integer msg ids.

v2 (tags-ride-the-archive): topic_segment gains origin_machine (per-machine attribution for the cross-machine LWW ingest) and a new session_verdict sidecar (the routine verdict + its floor|agent source) joins the gate. Bumping re-runs EnsureTopicSchema, which adds the column in place (PRAGMA-guarded ALTER) and creates session_verdict — NOT a base rebuild, so no transcript re-walk. Like the durability columns, existing NULL-origin rows are this machine's and backfill to MachineID().

View Source 
const TrigramBatchBoundSQL = `SELECT max(id) FROM (SELECT id FROM messages WHERE id > ? ORDER BY id LIMIT ?)`

TrigramBatchBoundSQL returns the highest messages.id in the next backfill batch — the upper bound of a half-open id window — or NULL when nothing is left to copy. Taking the bound first means the INSERT that follows is a plain range scan over the rowid index rather than a LIMIT whose row set depends on evaluation order. Args: the watermark, the batch size.

View Source 
const TrigramBatchFillSQL = `INSERT OR REPLACE INTO messages_fts_trigram(rowid, content) SELECT id, content FROM messages WHERE id > ? AND id <= ?`

TrigramBatchFillSQL copies one id window of messages into the substring index. Args: the watermark (exclusive), the batch bound (inclusive).

OR REPLACE makes the copy idempotent, which matters because an entry can already be there for a row this window covers: another process writing a message during the backfill fires the insert trigger for it. Re-writing an identical entry is a no-op in effect, whereas a plain INSERT would fail the whole migration on a rowid collision.

View Source 
const TrigramResetSQL = `DELETE FROM messages_fts_trigram`

TrigramResetSQL empties the substring index. It is the recovery path for a db whose index holds rows the backfill watermark cannot account for, where starting over is the only state that is knowably correct.

View Source 
const TrigramSQL = `` /* 617-byte string literal not displayed */

TrigramSQL is the SUBSTRING index: a second FTS5 table over the same content of the same messages, tokenized into overlapping three-character sequences, plus the triggers that keep it in step with messages exactly as the word index's triggers do.

It exists because an FTS5 table fixes one tokenizer, and a word tokenizer only ever matches on token boundaries. A query landing mid-token is not something messages_fts ranks badly — it is something messages_fts cannot answer at all. The two tables therefore answer disjoint question shapes, and both are needed.

The shape is copied verbatim from Hermes' schema — a plain fts5(content, tokenize='trigram'), no detail=none, no external content — because that is what a session-search tool with a large user base runs in production. Tuning it is something to do once there is evidence it is needed.

Unlike the word DDL above, every object here is IF NOT EXISTS: these arrive at an already-populated db through an additive migration (index.migrateTrigramIndex) as well as through a rebuild, so the DDL has to be safe to re-run against a db that already has some of it.

View Source 
const VecSchemaVersion = 1

VecSchemaVersion gates the chunk_vec table separately from the keyword schema.

View Source 
const VerdictRoutine = "routine"

VerdictRoutine is the only verdict kind today: the session is routine (trivial / low-signal). Kept as a named value rather than a bool so the column can carry future verdicts without a schema change (one schema).

Variables

This section is empty.

Functions

func CacheDir 

func CacheDir() string

CacheDir returns the session-search state dir (<cacheHome>/session-search), creating it. It holds the per-project index dbs, the tombstone sidecar, and the machine-id file — and is the discovery surface for orphaned-source dbs.

func ConnectRO 

func ConnectRO(dbp string) (*sql.DB, error)

ConnectRO opens dbp in read-only mode (file:<dbp>?mode=ro). Exported so sibling packages can reuse it. Configured with a 5s busy timeout and a 256MB mmap_size so hot queries serve directly from memory-mapped pages.

SINGLE-CONN DISCIPLINE: the pool is capped at ONE connection, so a caller MUST fully drain + close a result set (rows.Close) before issuing the next query on the same *sql.DB. Interleaving — opening a second query while rows from the first are still open — blocks forever waiting for a second connection (the view.Browse / semantic.VecKNN deadlock class).

func ConnectRW 

func ConnectRW(dbp string) (*sql.DB, error)

ConnectRW opens dbp read-write with WAL + a 10s busy timeout, single-writer. (10s is the unification of index's old 5s and cli's 10s timeouts.)

SINGLE-CONN DISCIPLINE: the pool is capped at ONE connection, so a caller MUST fully drain + close a result set (rows.Close) before issuing the next query on the same *sql.DB. Interleaving — opening a second query while rows from the first are still open — blocks forever waiting for a second connection (the view.Browse / semantic.VecKNN deadlock class).

func CountMessagesBetween added in v0.9.0 

func CountMessagesBetween(con *sql.DB, sid string, afterID, beforeID int) (int, error)

CountMessagesBetween counts ONE session's messages strictly between two row ids; beforeID <= 0 means no upper bound. Scoped by session_id on purpose: in a database holding every project, row ids run across all of them and a session that ran in two working directories has its rows split around other sessions' rows, so subtracting two ids counts strangers. [agentproto.Outline]

func CountSessions 

func CountSessions(dbp string) int

CountSessions opens dbp read-only and returns the session count, or -1 on error (callers must treat <0 as unknown).

func CountTopLevelSessions 

func CountTopLevelSessions(dbp string) int

CountTopLevelSessions returns the count of TOP-LEVEL sessions (is_subagent=0) — what a user means by "this project's sessions". Use this for display; the raw CountSessions above includes subagent threads and is internal bookkeeping. Returns -1 on error.

func DistinctProjects added in v0.9.0 

func DistinctProjects(con *sql.DB) ([]string, error)

DistinctProjects returns every project label present in the store, sorted. A caller holding a path pattern resolves it here — matching in Go — and passes the winners as Filter.Projects. Rows with no project (indexed before the scope columns existed) are left out, because there is no label to match a pattern against.

func EnsureTopicSchema 

func EnsureTopicSchema(con *sql.DB) error

EnsureTopicSchema creates the topic sidecar (its own gate, separate from the keyword schema) and stamps the topic_schema_version meta key. Idempotent. Mirrors EnsureVecSchema: every object is IF NOT EXISTS and lives outside the Rebuild() drop list, so a base reindex leaves it (and its rows) intact.

func EnsureVecSchema 

func EnsureVecSchema(con *sql.DB) error

EnsureVecSchema creates the chunk_vec table (its own gate, separate from the keyword schema) and stamps the vec_schema_version meta key. Idempotent.

func FirstUserMessages 

func FirstUserMessages(con *sql.DB, sid string, limit int) ([]string, error)

FirstUserMessages returns the contents of a session's first `limit` non-empty user messages, in id order — the browse-preview scan window. A NULL content reads as "". [view.sessionPreview]

func HasVectors 

func HasVectors(con *sql.DB) bool

HasVectors reports whether chunk_vec holds any rows. A missing table or read error reads as false. [semantic.HasVectors]

func InsertTopicSegments added in v0.10.0 

func InsertTopicSegments(con *sql.DB, segs []TopicSegment) error

InsertTopicSegments appends new topic segments to topic_segment. Segments are only ever added — prior segments are preserved.

func IsEffectivelyRoutine added in v0.6.0 

func IsEffectivelyRoutine(con *sql.DB, sessionID string) (bool, error)

IsEffectivelyRoutine resolves the cross-kind rule at read time: a session is effectively routine iff it carries a `routine` verdict AND has no real topic segment. "A real tag beats routine" — someone bothered to tag it, so it is not noise. Non-destructive and reversible: adding a real segment silently demotes the routine verdict without touching it; re-tagging reverses. The sort-tier surfacing reads exactly this.

func MergeVerdict added in v0.6.0 

func MergeVerdict(con *sql.DB, v Verdict) error

MergeVerdict is the cross-machine INGEST path for a verdict. Unlike segments (resolved by provenance authority), the verdict tie-break is FIXED by design — "verdict-vs-verdict tie = latest tagged_at wins" — so it is wall-clock LWW by design, not by default. This is low-stakes precisely because the only verdict kind is `routine`: when two machines both mark a session routine, the outcome is identical and the tie-break only selects which source/origin attribution to keep. The incoming row wins if strictly newer, or ties with a lexicographically-higher origin_machine (deterministic, skew is immaterial when both verdicts agree). Idempotent: an equal-or-older row is a no-op.

The cross-KIND rule — a real topic tag beats routine — is NOT applied here (destructively at ingest); it is resolved at READ time by IsEffectivelyRoutine, so it is order-independent and reversible by re-tag.

func MessageMeta 

func MessageMeta(con *sql.DB, msgID int) (iso, parent string, isSubagent bool, onlyCopySince float64, ok bool)

MessageMeta reads one message's ts_iso plus its session's parent_id, is_subagent, and only_copy_since (the vector-candidate existence check). A missing/churned row reads as ok=false; NULL fields read as their zero values. [semantic.VecKNN]

func MessageUUID 

func MessageUUID(con *sql.DB, msgID int) string

MessageUUID resolves a message rowid to its uuid. A missing row (or any read error, or a NULL uuid) reads as "". [agentproto.msgUUID]

func NewestHumanMessageID added in v0.9.0 

func NewestHumanMessageID(con *sql.DB, sid string) (int, error)

NewestHumanMessageID returns the id of the newest message in a session that a PERSON actually typed, or 0 when there is none.

Why a query and not a scan: an earlier version walked back a fixed window of trailing records looking for a human turn, and it silently found nothing. Measured on the live corpus, ~85% of role='user' rows are machinery — tool results, notifications, slash-command plumbing, reminder blocks — so in a working session the newest human turn can sit far behind the tail. It was 65 records back in the session that exposed this, against a 40-record window, and the caller degraded to "no turn found" without saying so. A predicate has no window to be wrong about.

The NOT LIKE list is the one the untagged-session census already validated on this corpus, plus the interruption marker. length>2 drops bare punctuation acks.

func ParentOf 

func ParentOf(con *sql.DB, sid string) string

ParentOf returns a session's parent_id, or "" when the session is missing, the parent is NULL/empty, or the read fails — the lineage walk treats all three identically as "root reached". [retrieve.LineageRoot]

func Rebuild 

func Rebuild(con *sql.DB) error

Rebuild drops and recreates the full schema + FTS, then stamps the version.

func ReplaceSessionSegments added in v0.6.0 

func ReplaceSessionSegments(con *sql.DB, sessionID string, segs []TopicSegment) error

ReplaceSessionSegments replaces a session's ENTIRE segment set atomically — DELETE the session's rows, then INSERT the incoming set — mirroring index.ReindexFile's per-session atomic replace. This is deliberately NOT a per-segment merge: two independent taggings of one session with different segment boundaries would interleave into a franken-set under per-key union. The session's tagging is ONE authored unit, so it is applied as one unit.

This is the primitive for BOTH authoring paths:

  • local tag-write (a re-tag REPLACES the prior set instead of stacking a second set beside it — segs carry an empty OriginMachine, stored NULL);
  • cross-machine ingest (segs carry the authoring machine's id).

For ingest, WHICH set wins is decided by the caller (archive ingest) via PROVENANCE AUTHORITY — the machine the session's transcript lives under owns its tags — NOT wall-clock: independent authorings differ in QUALITY, not freshness, and a clock cannot rank quality (and skews across machines). This function just applies the chosen set. Idempotent: replacing a set with an identical set is a no-op net of the FTS trigger churn, so re-ingesting the same files converges.

origin_machine is stored via NULLIF against the empty string, so an empty OriginMachine lands as NULL — the "this machine" sentinel the consolidated-store COALESCE depends on; a non-empty (foreign) origin passes through unchanged. An empty segs clears the session's segments (a caller does this only when authority says so).

func ResolveMessageUUID 

func ResolveMessageUUID(con *sql.DB, sid, uuidPrefix string, limit int) ([]int, error)

ResolveMessageUUID returns up to `limit` message ids in sid whose uuid has the given prefix, ordered by id. Callers pass limit=2 (the git-style ambiguity idiom: 0 matches = not found, 1 = resolved, 2 = ambiguous — never silently pick one). [agentproto.resolveUUID]

func RoutineSet added in v0.10.0 

func RoutineSet(con *sql.DB) (map[string]bool, error)

RoutineSet returns the set of all session IDs in con that are effectively routine (verdict="routine" AND no real topic segment). A missing sidecar table reads as no routine sessions; other database errors are propagated.

func RoutineVerdictSet added in v0.10.0 

func RoutineVerdictSet(con *sql.DB) (map[string]bool, error)

RoutineVerdictSet returns the set of all session IDs in con that carry a "routine" verdict in session_verdict.

func SessionHasRealSegments added in v0.6.0 

func SessionHasRealSegments(con *sql.DB, sessionID string) (bool, error)

SessionHasRealSegments reports whether a session carries at least one real topic segment (a non-empty topic) — the read-time signal that "a real tag beats routine" (a routine verdict is inert when real segments exist) and the cross-machine authority tie-break "real beats routine/empty". A missing topic table reads as false (no real tags).

func SessionIDsIn added in v0.6.0 

func SessionIDsIn(con *sql.DB) ([]string, error)

SessionIDsIn returns every session id present in a db's sessions table — the set the archive tag-ingest walks to attach pulled tags to their message-bearing session. A missing table reads as none (non-fatal).

func SessionMeta 

func SessionMeta(con *sql.DB, sid string) (lastTS float64, msgCount int, ok bool)

SessionMeta reads a session's last_ts + message_count. A missing row (or any read error) reads as ok=false. ISO formatting of lastTS stays caller-side. A NULL last_ts reads as 0. [agentproto.sessionMeta]

func SessionsByPrefix 

func SessionsByPrefix(con *sql.DB, prefix string, includeSubagents bool, limit int) ([]string, error)

SessionsByPrefix returns up to `limit` session ids with the given id prefix, ordered by id. includeSubagents=false adds is_subagent=0 (top-level only). Callers pass a small limit (2 for the git-style ambiguity guard, 3 for resume candidates) — enough rows to DETECT a collision without fetching the world. [agentproto.locateSession, cli.codexResumeHits]

func TaggedSessionIDs added in v0.6.0 

func TaggedSessionIDs(con *sql.DB) ([]string, error)

TaggedSessionIDs returns the distinct session ids that carry ANY tag — a topic segment or a verdict — in this db. The archive-export walk enumerates these to write one tag file per tagged session. Missing tables read as "none" (non-fatal): a db that predates the topic sidecar simply exports no tags.

func TopicForMessage 

func TopicForMessage(con *sql.DB, sessionID, msgUUID string) string

TopicForMessage returns the topic of the segment whose [start_uuid, end_uuid] range contains the given message uuid, for the non-fused (no embedder) path where there is no FTS topic match to attach. It uses the message's id order: the segment is the latest one in the session whose start message id is <= the target message id, and (if end_uuid is set) whose end message id is >= the target. When no range matches, it returns "". Kept deliberately simple; a missing topic table reads as "".

func TopicRowsExist 

func TopicRowsExist(con *sql.DB) bool

TopicRowsExist reports whether the topic_segment table holds any row — used to distinguish "query matched nothing" from "nothing is tagged yet". A missing table / read error reads as false. [agentproto.topicRowsExist]

func UpsertTopicSegment 

func UpsertTopicSegment(con *sql.DB, sessionID, startUUID, endUUID, topic, summary string, taggedAt float64) error

UpsertTopicSegment inserts or updates one topic segment, keyed by the stable (session_id, start_uuid). A per-key merge primitive, kept for callers that build up a segment set one row at a time (tests, ad-hoc tooling). It is NOT the tag-write path — a whole tagging is replaced as one unit via ReplaceSessionSegments, so a re-tag with shifted boundaries can't leave stale rows behind. origin_machine is left NULL — a locally-authored tag is "this machine" by construction, and a NULL origin is interpreted as this machine at export; only the cross-machine INGEST path (ReplaceSessionSegments) ever stamps a non-NULL, foreign origin. The external-content FTS triggers keep topic_fts in sync — an ON CONFLICT UPDATE fires the AFTER UPDATE trigger, which re-syncs the changed topic/summary.

func UpsertVerdict added in v0.6.0 

func UpsertVerdict(con *sql.DB, v Verdict) error

UpsertVerdict writes a session's verdict — the AUTHORING path (local floor write / local tag-write --routine). The local author's intent wins unconditionally; origin_machine stamps who wrote it. The cross-machine INGEST path is MergeVerdict (tagged_at-LWW), the author-vs-replicate split mirrored from the segment path.

func VecPrune 

func VecPrune(con *sql.DB, sid, contentHash string) error

VecPrune deletes the vector row keyed by (session_id, content_hash) — the stale-vector prune when the source text no longer exists. [semantic.VecIndex]

func VecRefreshMsgID 

func VecRefreshMsgID(con *sql.DB, sid, contentHash string, msgID int) error

VecRefreshMsgID re-points a stored vector at a churned message rowid without re-embedding (id churn on reindex is expected; the content hash is the stable key). [semantic.VecIndex]

func VecUpsert 

func VecUpsert(con *sql.DB, sid, contentHash string, msgID, dim int, vec []byte) error

VecUpsert inserts or replaces one vector row, keyed by (session_id, content_hash). vec is the packed little-endian float32 blob; dim is its element count. [semantic.VecIndex]

Types

type BrowseSession 

type BrowseSession struct {
	SessionID    string
	Project      string
	LastTS       float64
	MessageCount int
}

BrowseSession is one recent-session row returned by BrowseSessions: (id, project, last_ts, message_count). Preview text is a caller concern.

func BrowseScopedSessions added in v0.10.0 

func BrowseScopedSessions(con *sql.DB, since, before, sourceTool string, projects []string, limit int) ([]BrowseSession, error)

BrowseScopedSessions returns the most-recent TOP-LEVEL sessions (is_subagent=0) matching the optional date, source_tool, and project filters, newest first by last_ts. The rows are fully drained before returning.

func BrowseSessions 

func BrowseSessions(con *sql.DB, since, before string, limit int) ([]BrowseSession, error)

BrowseSessions returns a project's most-recent TOP-LEVEL sessions (is_subagent=0), newest first by last_ts. since/before ("" = no bound) are inclusive LOCAL-date bounds on last_ts (date(last_ts,'unixepoch','localtime')). The rows are fully drained before returning, so the single connection is free for follow-up queries (D3). [view.Browse]

type CorpusStats 

type CorpusStats struct {
	Sessions  int // top-level sessions (is_subagent=0)
	Subagents int // subagent threads (is_subagent=1)
	Messages  int
	User      int
	Assistant int
	First     string // earliest ts_iso[:10]
	Last      string // latest ts_iso[:10]
}

CorpusStats is the aggregate-counts result for one indexed project's db.

func GetCorpusStats 

func GetCorpusStats(dbp string) (CorpusStats, error)

GetCorpusStats returns aggregate counts for one indexed project's db (read-only). On a query error it returns a zero-value CorpusStats and nil error.

type Filter 

type Filter struct {
	IncludeSubagents bool   // false = top-level sessions only (s.is_subagent=0)
	Role             string // "" = any; else m.role=Role
	MinMessages      int    // 0 = no minimum; else s.message_count >= MinMessages
	SinceDate        string // "" = no bound; else substr(m.ts_iso,1,10) >= SinceDate (YYYY-MM-DD inclusive)
	BeforeDate       string // "" = no bound; else substr(m.ts_iso,1,10) <= BeforeDate (YYYY-MM-DD inclusive)

	// Scope, for the one store that holds every project. Which project a
	// session belongs to is a column here, so narrowing to a project is a WHERE
	// clause rather than a choice of which file to open. Both are empty by
	// default, which searches everything.
	//
	// Projects is an exact-match list, not a pattern: a caller with a regex
	// (--include-path) resolves it against DistinctProjects first and passes
	// the projects that matched. Keeping the pattern out of SQL means the
	// regex keeps Go's semantics instead of SQLite's.
	Projects   []string // empty = every project; else s.project IN (...)
	SourceTool string   // "" = every source; else s.source_tool = SourceTool
}

Filter is the shared WHERE composition for SearchHits / SearchAnchors (D5). The zero value applies only the top-level-sessions filter (IncludeSubagents false = s.is_subagent=0, matching the consumers' default).

type MessageRow 

type MessageRow struct {
	ID        int
	SessionID string
	Content   string
}

MessageRow is one corpus-wide (id, session_id, content) row.

func AllMessages 

func AllMessages(con *sql.DB) ([]MessageRow, error)

AllMessages returns every message's (id, session_id, content) — the vector indexer's full corpus scan. A NULL content reads as "". Unordered (table scan), matching the consumer. [semantic.VecIndex]

func MessagesForProjects added in v0.10.0 

func MessagesForProjects(con *sql.DB, projects []string) ([]MessageRow, error)

MessagesForProjects returns (id, session_id, content) for messages in the given project scopes. If projects is empty, it returns AllMessages. Unordered. [semantic.MeasureCoverage]

type Msg 

type Msg struct {
	ID      int
	Role    string
	Content string
}

Msg is the (id, role, content) triple read by the window/bookend queries.

func BookendMessages 

func BookendMessages(con *sql.DB, sid string, boundID int, hasBound, asc bool, limit int) ([]Msg, error)

BookendMessages returns up to `limit` user/assistant messages with non-empty content, ordered by id in the given direction. With hasBound, the window is bounded exclusive of boundID on the far side of the scan: ascending reads id<boundID (the run-up to a window), descending reads id>boundID (the tail after it) — matching the view's bookend queries. Without a bound it is the outline's session-start/-end bookend. [view.BuildAnchoredView, agentproto.bookendRows]

func MessagesAfter 

func MessagesAfter(con *sql.DB, sid string, anchorID, limit int) ([]Msg, error)

MessagesAfter returns up to `limit` messages strictly after anchorID (id>anchorID), ordered id ASC. [view.BuildAnchoredView]

func MessagesBefore 

func MessagesBefore(con *sql.DB, sid string, anchorID, limit int) ([]Msg, error)

MessagesBefore returns up to `limit` messages at or before anchorID (id<=anchorID — the anchor row is INCLUDED), ordered id DESC (nearest first; callers reverse for ascending display). [view.BuildAnchoredView]

type ProjectScope added in v0.9.0 

type ProjectScope struct {
	Project string
	CWD     string // "" when the session predates the scope columns
}

ProjectScope pairs a project label with the working directory it was recorded under. A pattern over paths (--include-path) needs the directory, while the filter that follows keys on the label, so both have to travel together.

func DistinctScopes added in v0.9.0 

func DistinctScopes(con *sql.DB) ([]ProjectScope, error)

DistinctScopes returns every (project, working directory) pair in the store, sorted by label. This is the one store's answer to "which projects exist and where do they live" — the question that used to require walking the transcript directories on disk. A project indexed from more than one directory appears once per directory, so a pattern matching any of them selects the label.

type SearchAnchor 

type SearchAnchor struct {
	ID            int
	SessionID     string
	UUID          string
	Role          string
	ISO           string
	Parent        string
	Content       string
	OnlyCopySince float64
	Snippet       string
	// Project is the label the session was indexed under. In a per-project
	// database every row carries the same value and the caller already knows
	// it; in the one store it is the only thing that says where a hit came
	// from, so it has to ride along with the row.
	Project string
}

SearchAnchor is one anchor-recall row: a SearchHit shape keyed by message id, plus the source uuid (the stable read-ref handle) and the session's only_copy_since watermark (>0 = source file deleted by CLI, RawClaw is only copy).

func SearchAnchors 

func SearchAnchors(con *sql.DB, match string, f Filter, s Sort, limit int) ([]SearchAnchor, error)

SearchAnchors runs the anchor-recall FTS5 query — the same filters and order as SearchHits, returning message ids + uuid + only_copy_since for the view layer to expand into bookend windows. [retrieve.MatchAnchors]

func SearchAnchorsSubstring added in v0.9.0 

func SearchAnchorsSubstring(con *sql.DB, match string, f Filter, s Sort, limit int) ([]SearchAnchor, error)

SearchAnchorsSubstring is SearchAnchors against the trigram index — the anchor-recall half of SearchHitsSubstring.

type SearchHit 

type SearchHit struct {
	SessionID  string
	Role       string
	ISO        string
	IsSubagent bool
	Parent     string
	Content    string
	Snippet    string
}

SearchHit is one flat keyword-recall row: the session/message columns plus the raw content (for the tool-stripped snippet rebuild + coverage count) and the FTS5-built snippet.

func SearchHits 

func SearchHits(con *sql.DB, match string, f Filter, s Sort, limit int) ([]SearchHit, error)

SearchHits runs the flat FTS5 keyword query and returns up to `limit` rows in the requested order. `match` is a finished FTS5 MATCH expression. The snippet format — snippet(messages_fts,0,'>>>','<<<','…',16) — is part of the output contract and stays byte-identical. [retrieve.searchScored]

func SearchHitsSubstring added in v0.9.0 

func SearchHitsSubstring(con *sql.DB, match string, f Filter, s Sort, limit int) ([]SearchHit, error)

SearchHitsSubstring is SearchHits against the trigram index instead of the word index: same filters, same order, same row shape. `match` is an FTS5 phrase, which the trigram tokenizer answers as a literal substring of the content — so this reaches the hits a word-boundary tokenizer cannot.

type SessionBacking added in v0.9.0 

type SessionBacking struct {
	SourceTool string
	SourcePath string
	CWD        string
	ParentID   string
	IsSubagent bool
}

SessionBacking is the live transcript identity recorded on a session row. It lets a caller re-read the exact backing container without rediscovering every transcript tree first.

func SessionBackingFor added in v0.9.0 

func SessionBackingFor(con *sql.DB, sid string) (backing SessionBacking, ok bool, err error)

SessionBackingFor returns the source metadata for sid. ok=false means the session row is absent; query errors are returned so callers can fall back to source discovery without mistaking a broken row for a live path.

type SessionMessage 

type SessionMessage struct {
	ID      int
	UUID    string
	Role    string
	Content string
}

SessionMessage is one full session-spine row: (id, uuid, role, content).

func LastMessages added in v0.9.0 

func LastMessages(con *sql.DB, sid string, limit int) ([]SessionMessage, error)

LastMessages returns a session's final `limit` messages as (role, content), newest first. Bounded on purpose: answering "what is this session doing now" only needs the tail, and a session can hold thousands of rows.

Ported from OpenClaw's tail-preview reader (src/gateway/session-utils.fs.ts, readLastMessagePreviewFromOpenTranscript), which seeks to size-16KB in the transcript file and keeps the last 20 lines. We have an index, so the bounded byte read becomes a bounded ORDER BY DESC — same contract, cheaper. Both roles are returned: the newest thing that happened is as often the agent's reply as the operator's instruction.

func SessionMessages 

func SessionMessages(con *sql.DB, sid string) ([]SessionMessage, error)

SessionMessages reads a session's messages in id order (id ascending) — the chronological spine the tag dump and segment-range mapping walk. [cli.loadSessionMessages]

type SessionRow added in v0.9.0 

type SessionRow struct {
	ID      string
	Project string
}

SessionRow is one session matched by id prefix, carrying the project label the row itself records. [agentproto.locateSession]

func SessionRowsByPrefix added in v0.9.0 

func SessionRowsByPrefix(con *sql.DB, prefix string, includeSubagents bool, projects []string, limit int) ([]SessionRow, error)

SessionRowsByPrefix answers "which session is this" against ONE database that holds every project. Because project is a column here, narrowing to a subset of projects is a WHERE clause rather than a choice of which file to open, and a session continued in a second directory is a single row rather than one row per project database — so the caller gets the merged session with nothing to reconcile afterwards.

projects narrows to those labels; an empty list means every project. limit bounds the read the same way SessionsByPrefix does: fetch just enough rows to DETECT a collision. includeSubagents=false adds is_subagent=0 (top-level only). [agentproto.locateSession]

type Sort 

type Sort int

Sort selects the ORDER BY for SearchHits / SearchAnchors (D5). 

const (
	// SortRelevance orders by FTS5 bm25 rank, then m.id (the default).
	SortRelevance Sort = iota
	// SortNewest orders by m.ts DESC, m.id DESC (a recency overlay — replaces
	// relevance entirely).
	SortNewest
	// SortOldest orders by m.ts ASC, m.id ASC.
	SortOldest
)

type SubagentRow added in v0.9.0 

type SubagentRow struct {
	ID           string
	MessageCount int
}

SubagentRow is one subagent child session of a parent session.

func SubagentsForSession added in v0.9.0 

func SubagentsForSession(con *sql.DB, parentSID string) ([]SubagentRow, error)

SubagentsForSession returns the child subagent sessions for a parent session ID.

type TopicHit 

type TopicHit struct {
	MsgID     int
	SessionID string
	Topic     string
	// Project is the label the segment's session was indexed under. In the one
	// store a topic result set spans projects, so the row is the only place that
	// says which project a label came from.
	Project string
}

TopicHit is one topic_fts match resolved to the segment's START message id (via a messages join on session_id+start_uuid). MsgID is the live rowid the fusion layer scores; SessionID + Topic carry the context to attach to it.

func MatchTopics 

func MatchTopics(con *sql.DB, query string, limit int, projects []string) ([]TopicHit, error)

MatchTopics runs an FTS query over topic_fts and, for each matched segment, resolves its START message to a live rowid (messages join on session_id+start_uuid). A segment whose start message is gone (churned/never indexed) is skipped — it has no anchor to surface. A missing topic table reads as no hits. Ordered by FTS rank, capped at limit.

Against the one store the rank is a GLOBAL one: bm25 folds in corpus statistics, so ordering is only meaningful when every candidate was scored against the same corpus. That is also why nothing here weights the topic column above the summary column. A label match already outranks a passing mention in a summary, because the label is the shorter field and bm25 normalizes by field length — a hand-tuned weight would be a second, worse copy of a judgement the corpus statistics already make.

projects narrows to a set of project labels (empty = every project), the same exact-match contract Filter.Projects has: a caller holding a path pattern resolves it in Go first. The sessions join is a LEFT join so a segment whose session row is missing still surfaces in an unnarrowed search — the project label is metadata for the caller, not a precondition for the hit.

type TopicSegment 

type TopicSegment struct {
	SessionID     string
	StartUUID     string
	EndUUID       string
	Topic         string
	Summary       string
	TaggedAt      float64
	OriginMachine string
}

TopicSegment is one tagged segment of a session, returned by TopicsForSession for the outline view. Keyed externally by (session_id, start_uuid). OriginMachine records which machine authored the tag (provenance.MachineID of the tagging machine) — the attribution the cross-machine ingest resolves on.

func TopicsForSession 

func TopicsForSession(con *sql.DB, sessionID string) ([]TopicSegment, error)

TopicsForSession returns the topic segments for one session, ordered by id (insertion order — roughly chronological as the tagger walks the session). Used by the outline view. A missing topic table reads as "no topics".

type VecKey added in v0.10.0 

type VecKey struct {
	SessionID   string
	ContentHash string
}

VecKey is the composite primary key (session_id, content_hash) for a vector row.

func VecKeys added in v0.10.0 

func VecKeys(con *sql.DB) ([]VecKey, error)

VecKeys returns every (session_id, content_hash) key in chunk_vec without loading the dense vector blobs. The rows are fully drained before returning. [semantic.MeasureCoverage]

type VecRow 

type VecRow struct {
	SessionID   string
	ContentHash string
	MsgID       int
	Vec         []byte
}

VecRow is one stored vector row: the composite key, the live message rowid, and the packed little-endian float32 blob. (The dim column is not surfaced — no consumer reads it; unpacking infers the dimension from the blob length.)

func VecAll 

func VecAll(con *sql.DB) ([]VecRow, error)

VecAll returns every chunk_vec row (unordered table scan) — the shared load for the indexer's have-map (which reads the key + msg_id) and the KNN scan (which reads msg_id + vec). The rows are fully drained before returning, so the single connection is free for follow-up queries (D3). [semantic.VecIndex, semantic.VecKNN]

type Verdict added in v0.6.0 

type Verdict struct {
	SessionID     string
	Verdict       string
	Source        string
	OriginMachine string
	TaggedAt      float64
}

Verdict is one session's verdict row.

func VerdictFor added in v0.6.0 

func VerdictFor(con *sql.DB, sessionID string) (Verdict, bool, error)

VerdictFor returns a session's verdict row, ok=false when it has none. A missing table reads as "no verdict" (non-fatal).

Source Files

View all Source files

Directories

Path Synopsis
Package storetest provides interface-built test fixtures for the store schema (D7): a production-schema db via store.Rebuild, plus row inserters.
 Package storetest provides interface-built test fixtures for the store schema (D7): a production-schema db via store.Rebuild, plus row inserters.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.