songya

package module
v0.0.0-...-f46ba71 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Nov 7, 2026 License: MIT Imports: 29 Imported by: 0

README

sōngyā Memory Engine

A memory engine that keeps connected records in Markdown files. Use it from an MCP client, its standalone browser editor, or as a Go library. Memories, entities, topics, and stories share one store, with open metadata and five structural relationship types.

Durability comes first: preserve original content, recover interrupted writes, and keep healthy records usable when something else is damaged. Lexical search, graph traversal, partial timelines and story composition run locally. Optional semantic search uses a configured embedding service.

The repository and Go module use github.com/aristath/songya; the MCP executable is songya-mcp. There is no Clara dependency or integration.

Standalone browser editor

go install github.com/aristath/songya/cmd/songya-web@latest
songya-web --store /absolute/path/to/memory-store

Open http://127.0.0.1:8664. Change the address with --listen 127.0.0.1:8665. The executable contains the HTML, CSS, Alpine.js and both editors. It needs no Node runtime, CDN, frontend service or Clara installation. Point it at the same store as the MCP to edit that store; engine writers coordinate across processes. The editor listens on loopback by default. There are no accounts or per-user access restrictions. Owner filters group memories and stories; entities and topics remain shared.

The front page is a full-width record table. Search content or filter by title, description, record type, metadata type, aliases and owner. Click a column header to sort in either direction. Record type identifies a memory, entity, topic or story; Type shows the selected user's configured classification. Type settings provides independent choices and clear definitions for each user and record kind. Defaults include 12 entity types; memories, topics and stories start without subtypes. Types for selects whose classifications to show, independently of the owner filter. The editor uses a dropdown, and chat receives the same definitions and allowed IDs. Shared identities can have different classifications for different users. Removing an in-use choice requires an explicit replacement or clearing its assignments.

Click a title to inspect the record in a read-only modal, or Edit to open the editor modal. Visual edits formatted Markdown; Source edits the complete Markdown/YAML document with syntax highlighting. Both modes share the same draft. Using Close, Escape or clicking outside the modal keeps unsaved work in the current tab. Metadata has text, number and boolean controls, with YAML values for lists and nested/custom structures. Creation/update timestamps are automatic and read-only. IDs come from filenames.

Connections searches for targets by name or text, accepts custom relationship types, and shows outgoing connections and incoming links. Opening another record keeps unsaved drafts in the current browser tab. Save explicitly publishes changes. Failed saves retain the draft; uncertain writes reuse the original request on retry, while rejected input can be corrected. Conflicts preserve your draft; Load saved version explicitly replaces it with the current stored record after confirmation. Discard draft/changes explicitly abandons local edits. The browser warns before leaving a tab with unsaved work. Drafts are not saved to disk until you press Save.

Delete permanently removes the live record immediately, without trash or a soft-delete state. The engine refuses deletion while other records link to it, including ordered story relations and retired-record redirects. Blockers are listed with links so you can open and edit them. Deletion never cascades or edits those referring records. Unreadable sources or unfinished writes can defer deletion; healthy reads and edits continue. Deleted records stay deleted through normal restart/recovery. Existing transaction receipts retain their original audit bytes; deletion does not expose a restore UI.

Raw HTML is shown as inert source content in the visual editor, never executed. Use Source for exact HTML or Markdown constructs outside the visual editor's CommonMark controls. Merely switching views does not rewrite the document.

Build from a checkout with go build -o songya-web ./cmd/songya-web. See UI development for asset builds and browser tests.

Start with MCP

Install Go at the version required by go.mod or newer. Go's automatic toolchain selection can download the required version. The store needs a local filesystem with hard links, advisory locks, and directory syncing.

git clone https://github.com/aristath/songya.git
cd songya
go install ./cmd/songya-mcp

Go puts the executable in GOBIN when configured, otherwise in bin under the first GOPATH directory (normally $HOME/go/bin). Check your paths with:

go env GOBIN GOPATH

Add the server to your MCP client's configuration, replacing both paths below with absolute paths on your machine. Clients have their own configuration file locations; this is the common mcpServers shape:

{
  "mcpServers": {
    "songya": {
      "command": "/absolute/path/to/songya-mcp",
      "args": ["--store", "/absolute/path/to/memory-store"]
    }
  }
}

The server creates missing store directories. Your client launches it over stdio; it opens no network listener. Running the executable directly waits for MCP protocol messages on stdin. Semantic search calls the embedding service below.

After connecting, use recall_memory({"text":"John"}) to retrieve connected accounts by name or subject. It includes surrounding story context and record references. Follow next as offset to continue larger accounts.

Ask your client to save a memory, for example:

Remember that I had coffee with John today. Check existing records first.

Use prepare_memory before saving, then save_memory with text, time and relations. The engine creates and connects the records automatically. Read the MCP guide for the tools, concrete requests, responses, retry handling, and optional imports.

Embeddings

Place config.json in the store, or pass --config /path/to/songya.json:

{
  "embeddings": {
    "model": "BAAI/bge-m3",
    "base_url": "http://127.0.0.1:18200/v1"
  }
}

These defaults match Clara's embedding setup. api_key is optional. Restart the MCP server after changing its configuration. The endpoint serves the model through an OpenAI-compatible /embeddings API; Songya does not download model weights.

search_memory({"text":"orchestral performer","mode":"semantic"}) searches entity names, aliases and descriptions, and memory titles, contents and semantic metadata. Entity and memory vectors refresh as their text changes. Use memory_maintenance({"action":"embed"}) to index all existing records ahead of time. Otherwise semantic searches embed missing content when needed.

Long text uses overlapping chunks of 2,000 characters; chunk_size adjusts this for another model. Vectors are rebuildable files under .index/embeddings. Changing the model or endpoint uses a separate cache. Reads and saves keep working when the service or cache is unavailable; search reports partial results or an explicit lexical fallback with diagnostics.

Use the Go library

From your application's Go module (for a new module, run go mod init example.com/songya-example first):

go get github.com/aristath/songya

Save this as main.go and run go run .:

package main

import (
	"fmt"
	"log"

	"github.com/aristath/songya"
)

func main() {
	store, err := songya.Open("./memory-store")
	if err != nil {
		log.Fatal(err)
	}
	saved, err := store.Create(songya.Memories,
		map[string]any{"user": "ari", "title": "Coffee with John"}, "I had coffee with John.")
	if err != nil {
		log.Fatal(err)
	}
	record, err := store.Read(saved.ID)
	if err != nil {
		log.Fatal(err)
	}
	fmt.Println(record.ID, record.Body)
}

Each run creates a new record. For revision-checked updates and retry-safe related changes, see the executable examples and engine reference.

Files and backups

Records live in memories/, entities/, topics/, and stories/. Each Markdown filename is its identity: a type prefix (m_, e_, t_, or s_) plus 12 random lowercase letters or digits. Metadata is YAML frontmatter; the body is Markdown. The engine assigns IDs and timestamps. No duplicate ID field is needed in YAML. Memories and stories require a user field. Entities and topics are shared. Select a user when searching, or use user:all to query everyone; this is grouping, with no access restrictions. See users.

Back up the whole store, including hidden files and directories. .batches/ retains transaction journals, original accounts, and before/proposed bytes; .imports/ retains imported originals and mappings. Hidden pending directories retain unfinished writes. Only .index/ is disposable.

For a consistent backup, stop writers and external editors while copying, or use an atomic filesystem snapshot. Restore into a directory on a supported local filesystem, then open that directory with the library or --store; recovery runs automatically. Keep .lock in place while a store is in use.

See durability and recovery for what pending writes mean and batch recovery for inspecting unfinished work.

Development

From the repository root:

go test -race -count=1 ./...
go vet ./...
go build -o /tmp/songya-mcp ./cmd/songya-mcp
/tmp/songya-mcp --help

The stdio tests build and launch the real server with the official MCP client. To instrument those child executables with the race detector too:

GOFLAGS=-race GORACE=atexit_sleep_ms=0 go test -count=1 ./mcpserver

Tests use temporary stores and import fixtures. Project priorities are in AGENTS.md.

Application recall

Store.RecallSemantic(ctx, queries, owners) batches individual source-message queries through one catalogue/vector pass. Owners are explicit; shared entities remain available. It ranks memories and entities semantically, promotes connected accounts using the three best entity matches, and includes at most one bounded containing-story passage. Results retain record IDs, owners, event times and semantic metadata. Missing embeddings produce partial diagnostics, never lexical scores. Query vectors reuse the same model/endpoint/content cache as records.

go run ./cmd/songya-query reads one JSON request from stdin: store, optional config, and command (configuration, list, read, stats or recall). Recall also takes owners and queries:[{text,limit}]. Configuration inspection excludes credentials. This application boundary exposes no write commands.

OpenExisting lets query clients inspect an existing store without creating record directories or publishing pending work. Missing sources remain explicit diagnostics. Normal Open retains automatic recovery for writers and the MCP.

Message retrieval is centralized in Songya: HTTP recall API, settings and client integration.

Documentation

Overview

Package songya stores connected Markdown records.

Index

Examples

Constants

View Source 
const IDLength = 2 + IDSuffixLength

IDLength includes the two-character prefix and the random suffix.

View Source 
const IDSuffixLength = 12

IDSuffixLength is the number of random characters after a record's prefix.

Variables

View Source 
var ErrConflict = errors.New("record changed since it was read")

ErrConflict means the committed bytes differ from the update's base revision.

View Source 
var RelationTypes = [...]string{"after", "caused_by", "part_of", "related_to", "possibly_same_as"}

Functions

func NewID 

func NewID(prefix string) (string, error)

NewID generates a candidate ID. The store must check its uniqueness before use.

func TypeInstructions 

func TypeInstructions(cfg TypeConfiguration) string

func TypeUser 

func TypeUser(ctx context.Context) string

func TypeUsers 

func TypeUsers(ctx context.Context, s *Store) ([]string, error)

func ValidID 

func ValidID(id string) bool

ValidID reports whether id uses the store's filename and reference format.

func WithOriginalText 

func WithOriginalText(ctx context.Context, text string) context.Context

WithOriginalText attaches the person's exact words, supplied by the client, not by the model. Generated record text remains in the before/after journal.

func WithTypeUser 

func WithTypeUser(ctx context.Context, user string) context.Context

Types

type Ambiguity 

type Ambiguity struct {
	Name       string      `json:"name"`
	Kind       Destination `json:"kind"`
	Candidates []string    `json:"candidates"`
	Truncated  bool        `json:"truncated,omitempty"`
}

type BatchRequest 

type BatchRequest struct {
	User    string   `json:"user,omitempty"`
	Key     string   `json:"key"`
	Account string   `json:"original_text,omitempty"`
	Changes []Change `json:"changes"`
	// contains filtered or unexported fields
}

type Catalog 

type Catalog struct {
	Records     []Record     `json:"records"`
	Diagnostics []Diagnostic `json:"diagnostics,omitempty"`
}

func (Catalog) Connections 

func (c Catalog) Connections(id string) []Edge

Connections returns all direct source links touching id, without paging or redirect resolution. Editors use the actual stored targets when changing links.

func (Catalog) ForUsers 

func (c Catalog) ForUsers(owners []string) Catalog

ForUsers selects explicit owners and shared identities. An empty selection includes only shared records; all is the explicit unrestricted selector.

func (Catalog) Select 

func (c Catalog) Select(q Query) []Record

Select applies the engine's lexical ranking/filters without another scan. Applications can page/project the returned records before crossing a boundary.

type Change 

type Change struct {
	Ref          string         `json:"ref,omitempty"`
	ID           string         `json:"id,omitempty"`
	Kind         Destination    `json:"kind,omitempty"`
	Revision     string         `json:"-"`
	Fields       map[string]any `json:"fields,omitempty"`
	Body         *string        `json:"body,omitempty"`
	Append       string         `json:"append,omitempty"`
	AddRelations []Relation     `json:"add_relations,omitempty"`
	// contains filtered or unexported fields
}

A nil body preserves the existing body on update, and means empty on creation.

type Configuration 

type Configuration struct {
	Embeddings EmbeddingConfig `json:"embeddings" yaml:"embeddings"`
	Recall     RecallPolicy    `json:"recall" yaml:"recall"`
}

func LoadConfiguration 

func LoadConfiguration(path string, fallback ...EmbeddingConfig) (Configuration, error)

type DeleteBlocked 

type DeleteBlocked struct {
	Records     []Record     `json:"records"`
	Diagnostics []Diagnostic `json:"diagnostics,omitempty"`
}

DeleteBlocked retains the complete list so a UI can navigate to each source.

func (*DeleteBlocked) Error 

func (e *DeleteBlocked) Error() string

type Destination 

type Destination string
const (
	Memories Destination = "memories"
	Entities Destination = "entities"
	Topics   Destination = "topics"
	Stories  Destination = "stories"
)

type Diagnostic 

type Diagnostic struct {
	Path    string `json:"path"`
	Message string `json:"message"`
}

Diagnostic identifies a recoverable problem without hiding healthy records.

type Document 

type Document struct {
	ID          string `yaml:"-"`
	Revision    string `yaml:"-"`
	Pending     bool   `yaml:"-"`
	Unconfirmed bool   `yaml:"-"`
	Frontmatter yaml.Node
	Body        string
	// contains filtered or unexported fields
}

Document keeps the complete YAML tree and Markdown body, including damaged fields. Diagnostics reports problems without making the record unavailable.

func NewDocument 

func NewDocument(metadata Metadata, body string) (*Document, error)

func Parse 

func Parse(data []byte) *Document

Parse always retains the content. Unreadable YAML is kept verbatim in an unparsed_frontmatter field; when boundaries are unknown, the full source is body.

func (*Document) Diagnostics 

func (d *Document) Diagnostics() []string

func (*Document) EditConnection 

func (d *Document) EditConnection(relation Relation, remove bool) error

EditConnection edits the single relations list. Other metadata and prose remain intact.

func (*Document) Marshal 

func (d *Document) Marshal() ([]byte, error)

Marshal preserves damaged metadata as data rather than rejecting the record.

func (*Document) Metadata 

func (d *Document) Metadata() Metadata

Metadata reads independent fields; one broken value cannot hide the others.

func (*Document) MoveConnection 

func (d *Document) MoveConnection(relation Relation, step int) error

MoveConnection changes narrative reading order without changing the connection.

func (*Document) OriginalSource 

func (d *Document) OriginalSource() []byte

OriginalSource returns the exact snapshot captured by Read, before parsing or edits. Its revision belongs to these same bytes, including malformed headers.

func (*Document) Record 

func (d *Document) Record(kind Destination) Record

Record returns the same tolerant metadata view used by search and timelines.

func (*Document) Set 

func (d *Document) Set(name string, value any) (err error)

Set edits one field, leaving other fields and their original representations intact.

func (*Document) SetType 

func (d *Document) SetType(user, typ string) error

SetType edits one user's assignment while preserving every other YAML value.

func (*Document) TypeFor 

func (d *Document) TypeFor(user string) string

func (*Document) Types 

func (d *Document) Types() map[string]string

type Edge 

type Edge struct {
	Source           string `json:"source"`
	SourceTitle      string `json:"source_title,omitempty"`
	TargetTitle      string `json:"target_title,omitempty"`
	Type             string `json:"type"`
	ReverseType      string `json:"reverse_type"`
	Target           string `json:"target"`
	Resolved         bool   `json:"resolved"`
	Unconfirmed      bool   `json:"unconfirmed,omitempty"`
	DetailsTruncated bool   `json:"details_truncated,omitempty"`
}

type EmbeddingConfig 

type EmbeddingConfig struct {
	Model     string `json:"model" yaml:"model"`
	BaseURL   string `json:"base_url" yaml:"base_url"`
	APIKey    string `json:"api_key,omitempty" yaml:"api_key"`
	ChunkSize int    `json:"chunk_size,omitempty" yaml:"chunk_size"`
}

EmbeddingConfig uses an OpenAI-compatible embedding endpoint. Defaults match Clara's BGE-M3 service; dimensions are learned from the returned vectors.

func LoadEmbeddingConfig 

func LoadEmbeddingConfig(path string) (EmbeddingConfig, error)

LoadEmbeddingConfig reads the same store configuration used by the MCP server. Readable fields are retained when a damaged field needs repair.

func (*EmbeddingConfig) UnmarshalYAML 

func (c *EmbeddingConfig) UnmarshalYAML(node *yaml.Node) error

Configuration fields decode independently; numeric strings are accepted.

type EmbeddingStatus 

type EmbeddingStatus struct {
	Model       string       `json:"model"`
	Records     int          `json:"records"`
	Indexed     int          `json:"indexed"`
	Chunks      int          `json:"chunks"`
	Dimensions  int          `json:"dimensions"`
	Diagnostics []Diagnostic `json:"diagnostics,omitempty"`
	Incomplete  bool         `json:"incomplete"`
}

type EpisodeTime 

type EpisodeTime struct {
	Date     *string `json:"date" yaml:"date"`
	Earliest *string `json:"earliest" yaml:"earliest"`
	Latest   *string `json:"latest" yaml:"latest"`
}

EpisodeTime describes the event, independently of automatic record timestamps.

type GraphQuery 

type GraphQuery struct {
	User      string `json:"user,omitempty"`
	ID        string `json:"id"`
	Direction string `json:"direction,omitempty"`
	Type      string `json:"type,omitempty"`
	Depth     int    `json:"depth,omitempty"`
	Offset    int    `json:"offset,omitempty"`
	Limit     int    `json:"limit,omitempty"`
}

type GraphResult 

type GraphResult struct {
	Resolution  *Resolution  `json:"resolution,omitempty"`
	Edges       []Edge       `json:"edges"`
	Next        int          `json:"next,omitempty"`
	Truncated   bool         `json:"truncated"`
	Diagnostics []Diagnostic `json:"diagnostics,omitempty"`
}

type ImportCandidate 

type ImportCandidate struct {
	Path     string `json:"path"`
	Revision string `json:"-"`
	Bytes    int    `json:"bytes"`
	Excerpt  string `json:"excerpt"`
}

type ImportMapping 

type ImportMapping struct {
	Source           string         `json:"source"`
	Revision         string         `json:"-"`
	Archive          string         `json:"archive"`
	BatchKey         string         `json:"batch_key"`
	Status           string         `json:"status"`
	Parts            []ImportedPart `json:"parts"`
	CoverageComplete bool           `json:"coverage_complete"`
	Uncovered        []Span         `json:"uncovered,omitempty"`
	Diagnostics      []Diagnostic   `json:"diagnostics,omitempty"`
}

type ImportPart 

type ImportPart struct {
	Start  int            `json:"start"`
	End    int            `json:"end"`
	Kind   Destination    `json:"kind,omitempty"`
	Fields map[string]any `json:"fields,omitempty"`
}

type ImportPreview 

type ImportPreview struct {
	Files       []ImportCandidate `json:"files"`
	Next        int               `json:"next,omitempty"`
	Truncated   bool              `json:"truncated"`
	Diagnostics []Diagnostic      `json:"diagnostics,omitempty"`
}

type ImportRequest 

type ImportRequest struct {
	User     string       `json:"user,omitempty"`
	Path     string       `json:"path"`
	Revision string       `json:"-"`
	Parts    []ImportPart `json:"parts,omitempty"`
}

type ImportedPart 

type ImportedPart struct {
	ID    string `json:"id"`
	Start int    `json:"start"`
	End   int    `json:"end"`
}

type Match 

type Match struct {
	Type             string      `json:"type,omitempty"`
	User             string      `json:"user,omitempty"`
	ID               string      `json:"id,omitempty"`
	Source           string      `json:"source"`
	Kind             Destination `json:"kind"`
	Revision         string      `json:"-"`
	Pending          bool        `json:"pending,omitempty"`
	Unconfirmed      bool        `json:"unconfirmed,omitempty"`
	Title            string      `json:"title"`
	Aliases          []string    `json:"aliases,omitempty"`
	Excerpt          string      `json:"excerpt"`
	Redirects        []string    `json:"redirects,omitempty"`
	Score            float64     `json:"score"`
	Reasons          []string    `json:"reasons,omitempty"`
	Diagnostics      []string    `json:"diagnostics,omitempty"`
	DetailsTruncated bool        `json:"details_truncated,omitempty"`
}

type MemoryInput 

type MemoryInput struct {
	Key       string          `json:"-"`
	ID        string          `json:"id,omitempty"`
	User      string          `json:"user,omitempty"`
	Type      *string         `json:"type,omitempty"`
	Title     *string         `json:"title,omitempty"`
	Text      *string         `json:"text,omitempty"`
	Append    string          `json:"append,omitempty"`
	Time      any             `json:"time,omitempty"`
	Metadata  map[string]any  `json:"metadata,omitempty"`
	Relations []RelationInput `json:"relations,omitempty"`
}

Key belongs to the caller's request metadata, never the model's arguments.

type MergeRequest 

type MergeRequest struct {
	Key     string         `json:"key"`
	Sources []Version      `json:"sources"`
	Kind    Destination    `json:"kind,omitempty"`
	Fields  map[string]any `json:"fields,omitempty"`
	Body    *string        `json:"body,omitempty"`
}

type Metadata 

type Metadata struct {
	User      string     `yaml:"user,omitempty"`
	Title     string     `yaml:"title"`
	CreatedAt time.Time  `yaml:"created_at"`
	UpdatedAt time.Time  `yaml:"updated_at"`
	Relations []Relation `yaml:"relations,omitempty"`
}

type NamedInput 

type NamedInput struct {
	Key         string          `json:"-"`
	Create      bool            `json:"create,omitempty"`
	ID          string          `json:"id,omitempty"`
	Name        string          `json:"name,omitempty"`
	Type        *string         `json:"type,omitempty"`
	User        string          `json:"user,omitempty"`
	Aliases     []string        `json:"aliases,omitempty"`
	Description *string         `json:"description,omitempty"`
	Append      string          `json:"append,omitempty"`
	Metadata    map[string]any  `json:"metadata,omitempty"`
	Relations   []RelationInput `json:"relations,omitempty"`
}

type Part 

type Part struct {
	Kind   Destination    `json:"kind,omitempty"`
	Fields map[string]any `json:"fields,omitempty"`
	Body   string         `json:"body"`
}

type Preparation 

type Preparation struct {
	Key                string       `json:"key"`
	Query              string       `json:"query"`
	User               string       `json:"user,omitempty"`
	Matches            []Match      `json:"matches"`
	Ambiguities        []Ambiguity  `json:"ambiguities"`
	PossibleDuplicates [][]string   `json:"possible_duplicates"`
	Next               int          `json:"next,omitempty"`
	Truncated          bool         `json:"truncated"`
	Diagnostics        []Diagnostic `json:"diagnostics,omitempty"`
}

type Query 

type Query struct {
	Mode           string      `json:"mode,omitempty"`
	User           string      `json:"user,omitempty"`
	Text           string      `json:"text,omitempty"`
	Kind           Destination `json:"kind,omitempty"`
	RelatedTo      string      `json:"related_to,omitempty"`
	RelationType   string      `json:"relation_type,omitempty"`
	IncludeRetired bool        `json:"include_retired,omitempty"`
	Offset         int         `json:"offset,omitempty"`
	Limit          int         `json:"limit,omitempty"`
}

type RecallHop 

type RecallHop struct {
	From string `json:"from"`
	To   string `json:"to"`
	Type string `json:"type"`
}

type RecallPage 

type RecallPage struct {
	Text        string       `json:"text,omitempty"`
	Base64      string       `json:"base64,omitempty"`
	Offset      int          `json:"offset"`
	Next        int          `json:"next,omitempty"`
	Truncated   bool         `json:"truncated"`
	Diagnostics []Diagnostic `json:"diagnostics,omitempty"`
}

type RecallPolicy 

type RecallPolicy struct {
	DirectLimit     int           `json:"direct_limit" yaml:"direct_limit"`
	MinScore        float64       `json:"min_score" yaml:"min_score"`
	SearchKinds     []Destination `json:"search_kinds" yaml:"search_kinds"`
	ReturnKinds     []Destination `json:"return_kinds" yaml:"return_kinds"`
	ConnectionDepth int           `json:"connection_depth" yaml:"connection_depth"`
	ConnectedLimit  int           `json:"connected_limit" yaml:"connected_limit"`
	RelationTypes   []string      `json:"relation_types" yaml:"relation_types"`
}

RecallPolicy belongs to the store, never to a requesting application.

func DefaultRecallPolicy 

func DefaultRecallPolicy() RecallPolicy

func (*RecallPolicy) UnmarshalYAML 

func (p *RecallPolicy) UnmarshalYAML(n *yaml.Node) error

Decode fields independently so a damaged option does not erase healthy ones.

type RecallQuery 

type RecallQuery struct {
	Text   string `json:"text"`
	User   string `json:"user,omitempty"`
	Offset int    `json:"offset,omitempty"`
	Limit  int    `json:"limit,omitempty"`
}

RecallQuery selects connected accounts. Offset and Limit count narrative bytes.

type RecallRequest 

type RecallRequest struct {
	Message  string   `json:"message,omitempty"`
	Messages []string `json:"messages,omitempty"`
	Owners   []string `json:"owners"`
}

RecallRequest accepts either one message or a batch sharing an ownership scope. Policies are deliberately absent: the store owns them.

func (RecallRequest) Validate 

func (r RecallRequest) Validate() error

type RecallResponse 

type RecallResponse struct {
	Results []SemanticRecallResult `json:"results"`
	Policy  RecallPolicy           `json:"policy"`
}

type Receipt 

type Receipt struct {
	Key         string            `json:"key"`
	Status      string            `json:"status"`
	Refs        map[string]string `json:"refs,omitempty"`
	Records     []SavedRecord     `json:"records,omitempty"`
	Diagnostics []Diagnostic      `json:"diagnostics,omitempty"`
}

type Recollection 

type Recollection struct {
	ID               string      `json:"id"`
	Kind             Destination `json:"kind"`
	User             string      `json:"user,omitempty"`
	Title            string      `json:"title"`
	Time             EpisodeTime `json:"time"`
	Content          string      `json:"content"`
	Reason           string      `json:"reason"`
	Score            *float64    `json:"score,omitempty"`
	OriginID         string      `json:"origin_id,omitempty"`
	Depth            int         `json:"depth"`
	Path             []RecallHop `json:"path,omitempty"`
	ContentTruncated bool        `json:"content_truncated,omitempty"`
}

type Record 

type Record struct {
	User        string            `json:"user,omitempty"`
	ID          string            `json:"id"`
	Source      string            `json:"source"`
	Kind        Destination       `json:"kind"`
	Revision    string            `json:"-"`
	Pending     bool              `json:"pending,omitempty"`
	Unconfirmed bool              `json:"unconfirmed,omitempty"`
	Title       string            `json:"title"`
	Name        string            `json:"name,omitempty"`
	Aliases     []string          `json:"aliases,omitempty"`
	Description string            `json:"description,omitempty"`
	Type        string            `json:"type,omitempty"`
	Types       map[string]string `json:"types,omitempty"`
	Relations   []Relation        `json:"relations,omitempty"`
	Redirects   []string          `json:"redirects,omitempty"`
	Time        EpisodeTime       `json:"time"`
	Body        string            `json:"body"`
	Context     string            `json:"context,omitempty"`
	Diagnostics []string          `json:"diagnostics,omitempty"`
}

Record is the derived search/graph view. Source Markdown is authoritative.

type Relation 

type Relation struct {
	Type   string `yaml:"type" json:"type"`
	Target string `yaml:"target" json:"target"`
}

type RelationInput 

type RelationInput struct {
	Type   string `json:"type" yaml:"type"`
	Target any    `json:"target" yaml:"target"`
}

Targets are record IDs, entity names, or a named entity/topic to create or reuse. Persisted relations always contain the engine-assigned target ID.

type Resolution 

type Resolution struct {
	ID          string       `json:"id"`
	Targets     []string     `json:"targets"`
	Ambiguous   bool         `json:"ambiguous"`
	Unconfirmed bool         `json:"unconfirmed,omitempty"`
	Truncated   bool         `json:"truncated,omitempty"`
	Diagnostics []Diagnostic `json:"diagnostics,omitempty"`
}

type SavedRecord 

type SavedRecord struct {
	ID            string   `json:"id"`
	Revision      string   `json:"-"`
	Outcome       string   `json:"outcome"`
	ChangedFields []string `json:"changed_fields"`
}

type SearchResult 

type SearchResult struct {
	Mode        string       `json:"mode"`
	Incomplete  bool         `json:"incomplete,omitempty"`
	Matches     []Match      `json:"matches"`
	Next        int          `json:"next,omitempty"`
	Truncated   bool         `json:"truncated"`
	Diagnostics []Diagnostic `json:"diagnostics,omitempty"`
}

type SemanticRecallResult 

type SemanticRecallResult struct {
	Mode        string         `json:"mode"`
	Status      string         `json:"status"`
	Items       []Recollection `json:"items"`
	Incomplete  bool           `json:"incomplete"`
	Truncated   bool           `json:"truncated"`
	Diagnostics []Diagnostic   `json:"diagnostics,omitempty"`
}

type Span 

type Span struct {
	Start int `json:"start"`
	End   int `json:"end"`
}

type SplitRequest 

type SplitRequest struct {
	Key    string  `json:"key"`
	Source Version `json:"source"`
	Parts  []Part  `json:"parts"`
}

type Store 

type Store struct {
	// contains filtered or unexported fields
}

Store holds no open files between calls. Diagnostics is safe across goroutines.

Example 
package main

import (
	"fmt"
	"os"

	"github.com/aristath/songya"
)

func main() {
	path, err := os.MkdirTemp("", "songya-example-")
	if err != nil {
		panic(err)
	}
	defer os.RemoveAll(path)
	store, err := songya.Open(path)
	if err != nil {
		panic(err)
	}
	created, err := store.Create(songya.Memories, map[string]any{
		"user":   "ari",
		"title":  "Example",
		"custom": map[string]any{"labels": []string{"one", "two"}},
	}, "Markdown body.")
	if err != nil {
		panic(err)
	}
	read, err := store.Read(created.ID)
	if err != nil {
		panic(err)
	}
	fmt.Println(read.Body)
	updated, err := store.Update(read.ID, read.Revision, map[string]any{"title": "Edited"}, "More detail.")
	if err != nil {
		panic(err)
	}
	fmt.Println(updated.Body)
}

Output:
Markdown body.
More detail.

func Open 

func Open(path string, configuration ...EmbeddingConfig) (*Store, error)

Open opens the root and lock. Problems in individual directories are reported through Diagnostics rather than disabling healthy parts of the store.

func OpenExisting 

func OpenExisting(path string, configuration ...EmbeddingConfig) (*Store, error)

OpenExisting opens without creating record directories or publishing recovery work. Query clients can report missing sources instead of silently replacing them with empty directories. Normal writers keep Open's automatic recovery.

func (*Store) AbandonBatch 

func (s *Store) AbandonBatch(ctx context.Context, key string) (Receipt, error)

AbandonBatch explicitly withdraws an uncommitted batch. Its journal, original account, and proposals stay intact. Only bytes published by this exact batch are rolled back; later external edits or deletions are never replaced.

func (*Store) ApplyBatch 

func (s *Store) ApplyBatch(ctx context.Context, request BatchRequest) (Receipt, error)
Example 
package main

import (
	"context"
	"fmt"
	"os"

	"github.com/aristath/songya"
)

func main() {
	path, err := os.MkdirTemp("", "songya-capture-")
	if err != nil {
		panic(err)
	}
	defer os.RemoveAll(path)
	store, err := songya.Open(path)
	if err != nil {
		panic(err)
	}
	ctx := context.Background()
	prepared, err := store.Prepare(ctx, "I met John for coffee.", "ari", 5)
	if err != nil {
		panic(err)
	}
	body := "I met John for coffee."
	receipt, err := store.ApplyBatch(ctx, songya.BatchRequest{User: "ari", Key: prepared.Key, Account: body, Changes: []songya.Change{
		{Ref: "john", Kind: songya.Entities, Fields: map[string]any{"name": "John"}},
		{Ref: "episode", Fields: map[string]any{"relations": []songya.Relation{{Type: "with", Target: "$john"}}}, Body: &body},
	}})
	if err != nil {
		panic(err)
	}
	doc, err := store.Read(receipt.Refs["episode"])
	if err != nil {
		panic(err)
	}
	fmt.Println(receipt.Status)
	fmt.Println(doc.Body)
	fmt.Println(doc.Metadata().Relations[0].Target == receipt.Refs["john"])
}

Output:
committed
I met John for coffee.
true

func (*Store) BatchData 

func (s *Store) BatchData(ctx context.Context, key, id, view string) ([]byte, error)

BatchData exposes retained work without changing it. Both original and proposed record bytes remain available after commit or abandonment. Receipt data can be byte-paged by a client even when a batch contains thousands of records.

func (*Store) BatchStatus 

func (s *Store) BatchStatus(ctx context.Context, key string) (Receipt, error)

func (*Store) Compose 

func (s *Store) Compose(ctx context.Context, id string, offset, limit int) (StoryPage, error)

Compose expands the story’s ordered relations. Prose is never another link store.

func (*Store) Configuration 

func (s *Store) Configuration() (Configuration, error)

Configuration is reloaded per operation.

func (*Store) ConfigurationSnapshot 

func (s *Store) ConfigurationSnapshot() (Configuration, []byte, error)

ConfigurationSnapshot couples the effective values to the exact source bytes used for optimistic settings saves, including incomplete readable settings.

func (*Store) Create 

func (s *Store) Create(destination Destination, fields map[string]any, body string) (*Document, error)

Create assigns the filename ID and UTC timestamps. Metadata problems are diagnostics, not reasons to discard the body. Pending=true means the complete record is durably staged and readable, while final publication will be retried.

func (*Store) Delete 

func (s *Store) Delete(ctx context.Context, id, revision string) error

Delete permanently removes one unreferenced, unchanged record. Checks and removal hold the same cross-process lock used by all engine writers.

func (*Store) Diagnostics 

func (s *Store) Diagnostics() []string

func (*Store) Directory 

func (s *Store) Directory() string

Directory is the canonical root used by applications storing companion state.

func (*Store) Graph 

func (s *Store) Graph(ctx context.Context, q GraphQuery) (GraphResult, error)

func (*Store) Import 

func (s *Store) Import(ctx context.Context, sourceRoot string, request ImportRequest) (ImportMapping, error)

Import keeps a checksum-named verbatim original and uses one idempotent batch per source path. Repartitioning an already imported source requires explicit organization of its mapped records, rather than duplicate active imports.

func (*Store) ImportSource 

func (s *Store) ImportSource(sourceRoot, source string) ([]byte, string, error)

func (*Store) IndexEmbeddings 

func (s *Store) IndexEmbeddings(ctx context.Context) (EmbeddingStatus, error)

IndexEmbeddings refreshes missing or damaged derived vectors. Writes to memories never depend on the endpoint; changed text gets new cache keys.

func (*Store) Merge 

func (s *Store) Merge(ctx context.Context, request MergeRequest) (Receipt, error)

Merge preserves every source body under its retired ID. The default canonical body concatenates those originals; a reviewed replacement may be supplied.

func (*Store) Prepare 

func (s *Store) Prepare(ctx context.Context, account, user string, limit int) (Preparation, error)

Prepare performs deterministic local retrieval. The caller interprets the shortlist and submits one complete ApplyBatch request; no inference is hidden.

func (*Store) PreviewImports 

func (s *Store) PreviewImports(ctx context.Context, sourceRoot string, offset, limit int) (ImportPreview, error)

PreviewImports never interprets episode boundaries. The client reviews exact source byte ranges; omitted parts import the entire file verbatim.

func (*Store) Read 

func (s *Store) Read(id string) (*Document, error)

func (*Store) ReadArchive 

func (s *Store) ReadArchive(revision string) ([]byte, error)

ReadArchive returns the checksum-verified original retained by an import.

func (*Store) ReadSource 

func (s *Store) ReadSource(source string) ([]byte, error)

ReadSource exposes only Markdown sources in the four record directories. This also keeps records with damaged filenames inspectable without inventing IDs.

func (*Store) Rebuild 

func (s *Store) Rebuild(ctx context.Context) (Catalog, error)

Rebuild ignores cached parsing and derives a fresh catalogue from source data.

func (*Store) Recall 

func (s *Store) Recall(ctx context.Context, q RecallQuery) (RecallPage, error)

Recall returns canonical prose with record labels and story context. It does not generate summaries, choose a primary story, or modify stored accounts.

func (*Store) Recover 

func (s *Store) Recover() error

Recover resumes completed pending writes when the write lock is available. Open reports deferred recovery without blocking access to readable records.

func (*Store) Resolve 

func (s *Store) Resolve(ctx context.Context, id string) (Resolution, error)

func (*Store) Retrieve 

func (s *Store) Retrieve(ctx context.Context, request RecallRequest) (RecallResponse, error)

Retrieve is the authoritative policy, ranking and traversal implementation. A batch scans and embeds each record once. Missing vectors never gain lexical scores.

func (*Store) RetrieveWithConfiguration 

func (s *Store) RetrieveWithConfiguration(ctx context.Context, request RecallRequest, config Configuration, configIssue error) (RecallResponse, error)

RetrieveWithConfiguration captures a turn's effective configuration for durable recovery. HTTP callers cannot override it through request fields.

func (*Store) SaveMemory 

func (s *Store) SaveMemory(ctx context.Context, input MemoryInput) (Receipt, error)

SaveMemory translates the caller's concepts into one durable related write. Entity lookup and creation run under that write's process-wide store lock.

func (*Store) SaveNamed 

func (s *Store) SaveNamed(ctx context.Context, kind Destination, input NamedInput) (Receipt, error)

func (*Store) SaveSource 

func (s *Store) SaveSource(ctx context.Context, key, id string, kind Destination, revision, source string) (Receipt, error)

SaveSource edits a complete document through the same durable receipts as MCP writes. A stable key makes retries safe; updates require the bytes' revision.

func (*Store) SaveStory 

func (s *Store) SaveStory(ctx context.Context, input StoryInput) (Receipt, error)

func (*Store) SaveTypeConfiguration 

func (s *Store) SaveTypeConfiguration(ctx context.Context, cfg TypeConfiguration, version string, replacements map[Destination]map[string]string, key string) (Receipt, error)

func (*Store) Scan 

func (s *Store) Scan(ctx context.Context) (Catalog, error)

Scan refreshes byte revisions on every call. Cached parsing never masks direct edits, and a damaged cache is rebuilt from the readable source files.

func (*Store) Search 

func (s *Store) Search(ctx context.Context, q Query) (SearchResult, error)

func (*Store) Split 

func (s *Store) Split(ctx context.Context, request SplitRequest) (Receipt, error)

Split makes the one-to-many redirect explicit. The original remains readable; neither incoming links nor stories are assigned to an arbitrary replacement.

func (*Store) Timeline 

func (s *Store) Timeline(ctx context.Context, q Query) (Timeline, error)

Timeline returns a partial chronology, never a guessed total ordering. Global analysis visits explicit edges; date-only edges are derived for this page, avoiding a quadratic graph of every dated pair in the entire store.

func (*Store) TypeConfiguration 

func (s *Store) TypeConfiguration(ctx context.Context, user string) (TypeConfiguration, error)

func (*Store) Update 

func (s *Store) Update(id, revision string, fields map[string]any, body string) (*Document, error)

Update applies metadata changes and replaces the body of a committed record. Pass the revision from Read or a committed Create result. Identity and creation time are preserved. An error with Pending=true retains this attempt in staging; Recover can retry it without replacing a conflicting committed version.

func (*Store) UseConfiguration 

func (s *Store) UseConfiguration(path string)

UseConfiguration selects a configuration file at construction, before sharing the Store with goroutines. All later operations read its current contents.

type StoryInput 

type StoryInput struct {
	Key       string          `json:"-"`
	ID        string          `json:"id,omitempty"`
	User      string          `json:"user,omitempty"`
	Type      *string         `json:"type,omitempty"`
	Title     string          `json:"title,omitempty"`
	Text      *string         `json:"text,omitempty"`
	Append    string          `json:"append,omitempty"`
	Metadata  map[string]any  `json:"metadata,omitempty"`
	Relations []RelationInput `json:"relations,omitempty"`
}

type StoryPage 

type StoryPage struct {
	User          string       `json:"user,omitempty"`
	UserTruncated bool         `json:"user_truncated,omitempty"`
	ID            string       `json:"id"`
	Revision      string       `json:"-"`
	Text          string       `json:"text,omitempty"`
	Base64        string       `json:"base64,omitempty"`
	Offset        int          `json:"offset"`
	Next          int          `json:"next,omitempty"`
	Truncated     bool         `json:"truncated"`
	Unconfirmed   bool         `json:"unconfirmed,omitempty"`
	Diagnostics   []Diagnostic `json:"diagnostics,omitempty"`
}

type TemporalConstraint 

type TemporalConstraint struct {
	Before string `json:"before"`
	After  string `json:"after"`
	Usable bool   `json:"usable"`
	Reason string `json:"reason,omitempty"`
}

type TemporalRecord 

type TemporalRecord struct {
	User             string      `json:"user,omitempty"`
	ID               string      `json:"id"`
	Title            string      `json:"title"`
	Time             EpisodeTime `json:"time"`
	Relations        []Relation  `json:"relations,omitempty"`
	Unordered        bool        `json:"unordered"`
	Unconfirmed      bool        `json:"unconfirmed,omitempty"`
	DetailsTruncated bool        `json:"details_truncated,omitempty"`
}

type Timeline 

type Timeline struct {
	Groups      map[string][]string  `json:"groups,omitempty"`
	Records     []TemporalRecord     `json:"records"`
	Constraints []TemporalConstraint `json:"constraints"`
	Next        int                  `json:"next,omitempty"`
	Truncated   bool                 `json:"truncated"`
	Diagnostics []Diagnostic         `json:"diagnostics,omitempty"`
}

type TypeConfiguration 

type TypeConfiguration struct {
	User        string                           `json:"user" yaml:"user"`
	Kinds       map[Destination][]TypeDefinition `json:"kinds" yaml:"kinds"`
	Version     string                           `json:"version,omitempty" yaml:"-"`
	Diagnostics []string                         `json:"diagnostics,omitempty" yaml:"-"`
	Usage       map[Destination]map[string]int   `json:"usage,omitempty" yaml:"-"`
}

func DefaultTypeConfiguration 

func DefaultTypeConfiguration(user string) TypeConfiguration

type TypeDefinition 

type TypeDefinition struct {
	ID          string `json:"id" yaml:"id"`
	Name        string `json:"name" yaml:"name"`
	Description string `json:"description" yaml:"description"`
}

type Version 

type Version struct {
	ID       string `json:"id"`
	Revision string `json:"-"`
}

Source Files

View all Source files

Directories

Path Synopsis
cmd
songya-mcp command
songya-query command
songya-query is the JSON query boundary for application recall and browsing.
 songya-query is the JSON query boundary for application recall and browsing.
songya-web command
Package mcpserver exposes the standalone memory engine over MCP stdio.
 Package mcpserver exposes the standalone memory engine over MCP stdio.
Package web serves Songya's standalone editor; all persistence stays in the engine.
 Package web serves Songya's standalone editor; all persistence stays in the engine.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.