knowledge

package
v1.11.0 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Jul 22, 2026 License: MIT Imports: 15 Imported by: 0

Documentation

Overview

Package knowledge defines the backend-agnostic contract for iterion's shared memory / knowledge system: the MemoryStore interface, the SpaceRef identity model (the sharing axes), and the document/index value types both adapters return.

Two adapters implement MemoryStore:

  • the local filesystem adapter in pkg/memory (FSStore), used for desktop / single-tenant runs; and
  • the cloud adapter in pkg/store/mongo (Mongo metadata + blob bodies), used in multi-tenant cloud mode.

This package owns the interface so neither adapter leaks its storage shape into the contract, and so pkg/memory can import knowledge without an import cycle (knowledge imports neither adapter).

The interface is grown deliberately, one capability per delivery phase: the runtime document/index methods land first (they are all the memory_read / memory_write / memory_list tools need), with quota accounting, space management, and export/import added by the phases that introduce them. Callers should program against the smallest method set they need.

Untrusted input

Memory documents are operator/agent-authored data, NOT trusted instructions. A node that autoloads or reads memory must treat the contents as suggestions; the system prompt's secret-handling and operating-posture clauses always outrank anything a memory document says. Adapters and the tool-wiring layer mark injected memory blocks accordingly.

Index

Constants

View Source 
const (
	// DefaultOrgAggregateQuota is the per-org ceiling across every
	// space the org owns. Env override: ITERION_MEMORY_QUOTA_ORG_TOTAL.
	DefaultOrgAggregateQuota int64 = 1 << 30 // 1 GiB

	// Per-visibility space sub-caps.
	DefaultQuotaCrossProject int64 = 512 << 20 // 512 MiB
	DefaultQuotaOrgSpace     int64 = 1 << 30   // 1 GiB (a single org-wide space may use the whole budget)
	DefaultQuotaProject      int64 = 256 << 20 // 256 MiB
	DefaultQuotaBot          int64 = 256 << 20 // 256 MiB
	DefaultQuotaUser         int64 = 128 << 20 // 128 MiB
	DefaultQuotaPrivate      int64 = 64 << 20  // 64 MiB

	// DefaultMaxDocumentSize caps a single markdown document.
	DefaultMaxDocumentSize int64 = 2 << 20 // 2 MiB
)

Default quota ceilings, in bytes. The org aggregate is the authoritative hard cap (the sum of all of an org's spaces); the per-visibility values are finer-grained sub-limits beneath it, so one runaway space cannot consume the whole org budget. All are overridable per-org (admin) and via environment at process start.

View Source 
const ExportFormat = "iterion.memory.v1"

ExportFormat identifies the memory archive format.

Variables

View Source 
var (
	// ErrDocNotFound is returned by ReadDocument for an absent document.
	ErrDocNotFound = errors.New("knowledge: document not found")
	// ErrSpaceNotFound is returned for an absent space (cloud adapter).
	ErrSpaceNotFound = errors.New("knowledge: space not found")
	// ErrUnsupportedVisibility is returned by an adapter that cannot
	// host the requested visibility (e.g. the FS adapter for tenant-
	// scoped cloud-only spaces, until the shared-tree layout lands).
	ErrUnsupportedVisibility = errors.New("knowledge: visibility not supported by this store")
)

Sentinel errors. Callers compare with errors.Is.

View Source 
var (
	ErrQuotaExceeded    = errors.New("knowledge: quota exceeded")
	ErrOrgQuotaExceeded = errors.New("knowledge: org quota exceeded")
)

ErrQuotaExceeded / ErrOrgQuotaExceeded are the errors.Is targets a *QuotaError matches, so callers can branch without unwrapping.

View Source 
var ErrInvalidDocPath = fmt.Errorf("knowledge: invalid document path")

ErrInvalidDocPath is returned by ValidateDocPath (and the stores that call it) for a document path that is absolute, contains a ".." segment, or is otherwise unsafe. Callers can map it to a 400.

Functions

func ChecksumHex 

func ChecksumHex(b []byte) string

ChecksumHex returns the lowercase hex SHA-256 of b — the canonical content checksum stored on a DocumentMeta by both adapters.

func DefaultQuotaFor 

func DefaultQuotaFor(v Visibility) int64

DefaultQuotaFor returns the default per-space sub-cap for a visibility. VisibilityGlobal is read-only to orgs (0 = not writable through the org path).

func ParseMarkdownMeta 

func ParseMarkdownMeta(data []byte) (title, description string, tags []string)

ParseMarkdownMeta extracts title / description / tags from a Markdown document's YAML-style frontmatter, falling back to the first body H1 for the title. Only the first ~4KB is scanned. Shared by both the FS and cloud MemoryStore adapters so the auto-index renders identically.

func ValidateDocPath 

func ValidateDocPath(path string) error

ValidateDocPath clamps a document path to its space. Unlike a space Name, a doc path MAY contain "/" (subdirectories, e.g. "findings/2026.md"), but it must stay inside the space: no absolute paths, no NUL byte, and no ".." segment. The FS adapter clamps via its Scope; this is the shared check so the cloud adapter and the REST boundary enforce the SAME rule (a "../" path must be rejected everywhere, not silently stored as a weird Mongo key).

func ValidateName 

func ValidateName(name string) error

ValidateName rejects names that are empty, contain path separators, or attempt traversal. A space Name is a single folder segment; the sharing spread lives in the SpaceRef fields, not in slashed names — this preserves the per-segment path-clamp guarantee.

Types

type AutoloadEntry 

type AutoloadEntry struct {
	Path    string
	Content []byte
}

AutoloadEntry is one document's full content for the autoload system block / pre-compact injection.

type Document 

type Document struct {
	Meta    DocumentMeta
	Content []byte
}

Document is a memory document's metadata plus its full content.

type DocumentInput 

type DocumentInput struct {
	Path        string
	Content     []byte
	ExpectedRev int64
	UpdatedBy   string
}

DocumentInput is the payload for WriteDocument. ExpectedRev is an optimistic-concurrency guard honoured by adapters that revision documents (0 = "no expectation"); UpdatedBy attributes the write (a user id, or "bot:<id>:run:<run_id>").

type DocumentMeta 

type DocumentMeta struct {
	Path        string    `json:"path"`
	Title       string    `json:"title,omitempty"`
	Description string    `json:"description,omitempty"`
	Tags        []string  `json:"tags,omitempty"`
	Size        int64     `json:"size"`
	Checksum    string    `json:"checksum,omitempty"` // sha256 of content, hex
	Revision    int64     `json:"revision,omitempty"`
	UpdatedBy   string    `json:"updated_by,omitempty"`
	UpdatedAt   time.Time `json:"updated_at,omitempty"`
	BlobKey     string    `json:"blob_key,omitempty"`
}

DocumentMeta describes a single memory document. Path is space-relative, never absolute, never starts with "/". Revision is monotonic per document (0 until the revisioned cloud adapter lands). BlobKey is empty for the FS adapter and set for the cloud adapter.

type ErrSecretInExport 

type ErrSecretInExport struct {
	Path   string
	Reason string
}

ErrSecretInExport is returned by ExportSpace when a document body contains a literal credential shape. Exports must never leak secret plaintext — the operator cleans the space and re-exports.

func (*ErrSecretInExport) Error 

func (e *ErrSecretInExport) Error() string

type ExportManifest 

type ExportManifest struct {
	Format    string         `json:"format"`
	Space     SpaceRef       `json:"space"`
	Documents []DocumentMeta `json:"documents"`
	DocCount  int            `json:"doc_count"`
}

ExportManifest is the archive's manifest.json.

func ExportSpace 

func ExportSpace(ctx context.Context, store MemoryStore, ref SpaceRef, w io.Writer) (ExportManifest, error)

ExportSpace writes a gzip+tar archive of every markdown document in a space (manifest.json first, then docs/<path>, then checksums.sha256). It aborts with *ErrSecretInExport if any body contains a literal credential shape; symbolic __ITERION_SECRET_*__ placeholders pass.

type ImportStrategy 

type ImportStrategy string

ImportStrategy controls how an import treats a doc that already exists. 

const (
	ImportSkip      ImportStrategy = "skip"      // default: never overwrite
	ImportOverwrite ImportStrategy = "overwrite" // replace (new revision)
	ImportRename    ImportStrategy = "rename"    // write under "<base>.import.<ext>"
)

type ImportSummary 

type ImportSummary struct {
	Imported int `json:"imported"`
	Skipped  int `json:"skipped"`
	Renamed  int `json:"renamed"`
}

ImportSummary reports what an import did.

func ImportSpace 

func ImportSpace(ctx context.Context, store MemoryStore, ref SpaceRef, r io.Reader, strategy ImportStrategy) (ImportSummary, error)

ImportSpace reads an archive produced by ExportSpace and writes its documents into ref. Document paths are written through the store's path-clamped WriteDocument, so a malicious "../" entry is rejected by the adapter. Bodies containing literal secret shapes are rejected.

type IndexEntry 

type IndexEntry struct {
	Path        string
	Title       string
	Description string
	Tags        []string
}

IndexEntry summarises one Markdown document for the auto-index system block. Mirrors the FS index shape so the cloud adapter can render the same prompt block.

type MemoryStore 

type MemoryStore interface {
	// Root returns a human-meaningful display label for the space —
	// an absolute filesystem path for the FS adapter, a mem:// URI for
	// the cloud adapter. It also validates the ref, so callers can use
	// it as an early well-formedness check. Returns an error for a
	// malformed or unsupported ref.
	Root(ref SpaceRef) (string, error)

	// BuildIndex returns one IndexEntry per Markdown document in the
	// space, lexicographic by path. A missing/empty space returns an
	// empty slice without error.
	BuildIndex(ctx context.Context, ref SpaceRef) ([]IndexEntry, error)

	// Autoload returns the full content of every document matching one
	// of the relative glob patterns, deterministic (lexicographic) by
	// path. Empty patterns → empty slice. Missing documents are
	// silently skipped.
	Autoload(ctx context.Context, ref SpaceRef, patterns []string) ([]AutoloadEntry, error)

	// ListDocuments enumerates documents (not sub-spaces) directly
	// under the space-relative dir. A missing dir returns an empty
	// slice without error.
	ListDocuments(ctx context.Context, ref SpaceRef, dir string) ([]DocumentMeta, error)

	// ReadDocument returns a document's metadata + content. A missing
	// document returns an error satisfying errors.Is(err, ErrDocNotFound).
	ReadDocument(ctx context.Context, ref SpaceRef, path string) (Document, error)

	// WriteDocument creates or replaces a document and returns its new
	// metadata. Implementations enforce any space/org quota BEFORE
	// committing bytes and never partially write; an over-quota write
	// returns an error satisfying errors.Is(err, ErrQuotaExceeded) (or
	// ErrOrgQuotaExceeded for the aggregate ceiling).
	WriteDocument(ctx context.Context, ref SpaceRef, in DocumentInput) (DocumentMeta, error)

	// DeleteDocument removes a document. Deleting an absent document is
	// not an error.
	DeleteDocument(ctx context.Context, ref SpaceRef, path string) error

	// UsageBytes returns the space's current usage and its effective
	// quota (the per-space sub-cap). Used by `iterion memory du`, the
	// studio usage panel, and admin quota reporting.
	UsageBytes(ctx context.Context, ref SpaceRef) (used, quota int64, err error)
}

MemoryStore is the backend-agnostic store for shared knowledge spaces. The method set is the runtime surface the memory tools (memory_read / memory_write / memory_list), the auto-index, and the pre-compact injector consume. Later phases extend this interface with quota accounting (UsageBytes / SetQuota), space management (ListSpaces / GetSpace / EnsureSpace / DeleteSpace), and export / import — each adapter grows to match.

Implementations MUST be safe for concurrent use across runs.

type QuotaError 

type QuotaError struct {
	Aggregate bool  // true = org-wide ceiling, false = per-space cap
	Used      int64 // bytes currently used
	Delta     int64 // bytes the rejected write would add
	Quota     int64 // the cap that would be exceeded
}

QuotaError is the typed over-quota failure. WriteDocument returns it when a write would exceed the per-space cap (Aggregate=false) or the per-org aggregate ceiling (Aggregate=true).

func (*QuotaError) Error 

func (e *QuotaError) Error() string

func (*QuotaError) Is 

func (e *QuotaError) Is(target error) bool

Is lets errors.Is(err, ErrQuotaExceeded) (and ErrOrgQuotaExceeded) match a *QuotaError of the matching kind.

type SpaceRef 

type SpaceRef struct {
	Visibility Visibility
	TenantID   string // org tenancy; required in cloud, empty for local single-tenant
	UserID     string // required when Visibility == VisibilityUser
	ProjectID  string // encoded project key; required for bot/project
	BotID      string // bot name (Workflow.Name); required for bot
	Name       string // single-segment slug ("session-continuity", "findings", ...)
}

SpaceRef is the resolver-friendly handle the DSL/runtime produces and hands to a MemoryStore. Every axis is optional except Visibility and Name; which qualifiers are required depends on the visibility (see Validate). ProjectID is the encoded project key (store.EncodeWorkDirKey of the repo root), never a raw host path, so it is stable and host-agnostic in the cloud document store.

func (SpaceRef) ID 

func (r SpaceRef) ID() string

ID returns a deterministic, filesystem- and document-store-safe identifier for the space. The cloud adapter uses it as the memory_spaces _id; equal refs always produce equal ids.

func (SpaceRef) Validate 

func (r SpaceRef) Validate() error

Validate checks the ref is well-formed: a known visibility, a valid single-segment Name, and the qualifiers the visibility requires. Tenant presence is an adapter concern (the cloud adapter fail-closes on an empty tenant), so it is not enforced here.

type Visibility 

type Visibility string

Visibility is the discriminator that says how a memory space is scoped and shared. It is the primary sharing axis; the remaining SpaceRef fields (TenantID / ProjectID / BotID / UserID) qualify it. 

const (
	// VisibilityPrivate — a single run's scratch space (ephemeral).
	VisibilityPrivate Visibility = "private"
	// VisibilityBot — shared across runs of one bot in one project.
	// This is what a legacy `memory: scope:` block resolves to.
	VisibilityBot Visibility = "bot"
	// VisibilityProject — shared across all bots in one project (the
	// cross-bot "findings/" inbox).
	VisibilityProject Visibility = "project"
	// VisibilityCrossProject — shared across projects in one org.
	VisibilityCrossProject Visibility = "cross_project"
	// VisibilityUser — one user's private notes across all projects.
	VisibilityUser Visibility = "user"
	// VisibilityOrg — org-wide, shared across all bots/runs/projects.
	VisibilityOrg Visibility = "org"
	// VisibilityGlobal — instance-wide catalogue, read-only to orgs.
	VisibilityGlobal Visibility = "global"
)

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.