docindex

Documentation

Overview

Package docindex is satelle's directory monitor for authored markdown.

The architecture splits the system of record: stories/tasks/ledger are dynamic sqlite primitives, while authored artifacts (documents, workflows, principles, skills) are MARKDOWN ON DISK — the files are the source of truth. This package syncs those files into a sqlite index so the CLI and web can query them without the markdown becoming a hand-managed store.

Sync is the core: walk the configured per-kind dirs, upsert changed files (detected by size+mtime), and prune rows whose file disappeared. Watch wraps Sync in a poll loop — a dependency-free monitor (satellites indexes by scanning, not fsnotify), so the static no-cgo binary stays dependency-light. SQL is libSQL-compatible.

Index

• Variables
• func AuthoredExt(kind string) string
• func Frontmatter(body string) (lines []string, ok bool)
• func FrontmatterBody(body string) (lines []string, rest string, ok bool)
• func Indexable(path string) bool
• func MaterializeOKF(dir, heading string, items []OKFItem, now time.Time) error
• func MetaTables(body, key string) ([]map[string]string, bool)
• func Migrate(db *sql.DB) error
• func OKFConformance(name, body string) error
• type Doc
• type DocRef
• type OKFItem
• type Store
  • func New(db *sql.DB) *Store
  • func (s *Store) Count(ctx context.Context, kind string) (int, error)
  • func (s *Store) Fingerprint(ctx context.Context) (string, error)
  • func (s *Store) Get(ctx context.Context, kind, name string) (Doc, error)
  • func (s *Store) List(ctx context.Context, kind string) ([]Doc, error)
  • func (s *Store) SetDefaults(defs []Doc)
  • func (s *Store) Sync(ctx context.Context, dirs map[string]string, now time.Time) (SyncResult, error)
  • func (s *Store) Watch(ctx context.Context, dirs map[string]string, interval time.Duration, ...) error
• type SyncResult

Variables

var ErrNotFound = errors.New("docindex: not found")

ErrNotFound is returned when a Get misses.

Functions

func AuthoredExt

func AuthoredExt(kind string) string

AuthoredExt is the extension a NEW authored doc of the given kind is written with. Prose kinds are markdown; the workflows kind is TOML, because the only doc it may hold is a route-source half and a route source is records rather than prose (sty_81bb0dde).

Here rather than at each writer for the same reason Indexable is: the create path and the walk that reads it back must agree on the extension, and they disagreed the moment one of them was updated alone — `workflow create` wrote step.md, which the route resolver then refused as an unconverted repo.

func Frontmatter

func Frontmatter(body string) (lines []string, ok bool)

Frontmatter returns a document's frontmatter as `key: value` lines, whichever form the file is authored in (sty_81bb0dde).

Two forms exist because two kinds of file do:

• a `---` YAML block, for a markdown DOCUMENT (skills, principles, tasks);
• a `[meta]` table, for a TOML file (the route source), which is records with terse commentary and uses no markdown feature.

FORMAT IS SNIFFED, not threaded. A body beginning with `---` is the markdown form; anything else is decoded as TOML and its `[meta]` table read. That is unambiguous because a file is wholly one format — there is no document that is half YAML-frontmatter and half TOML. Sniffing is what lets every existing caller keep its signature: `structure.Doc` receives a body and no format, and does not need one.

It returns LINES rather than a map on purpose. Every consumer — structure's fmScalar/fmHas, wfhook's scalar and hooks-block reader, the embedded conformance helpers — already works against `key: value` lines. A map would force all of them to be rewritten for no gain.

ok is false when there is no frontmatter at all: no `---` block, or TOML with no `[meta]` table. A body that is neither valid TOML nor markdown-with- frontmatter is also (nil, false) — reporting the decode error is the job of whoever parses the file for real, not of a frontmatter reader.

func FrontmatterBody

func FrontmatterBody(body string) (lines []string, rest string, ok bool)

FrontmatterBody is Frontmatter plus the remainder after the frontmatter — the document's prose. For the TOML form there is no separable remainder, so the whole body is returned; a caller that renders `rest` as prose is looking at a markdown document by construction.

func Indexable

func Indexable(path string) bool

Indexable reports whether a path is an authored substrate file the index ingests. Markdown is the DOCUMENT form; TOML is the form for a file that is records rather than prose — today the route source (sty_81bb0dde).

This is one predicate rather than an `.md` literal at each walk, because the three walks that used to carry that literal independently are exactly how a format ends up half-supported.

func MaterializeOKF

func MaterializeOKF(dir, heading string, items []OKFItem, now time.Time) error

MaterializeOKF renders items into dir as a read-only OKF reference folder: one <Name>.md per item (via renderOKFItem), plus the reserved index.md (link list) and log.md (date-grouped changelog). It is the SINGLE materialization path for generated OKF surfaces (the story backlog, the summary sub-bundle). It is idempotent — each file is written only when its content changes — and it prunes stale generated files: a *generated* concept file (carrying the marker) whose name is not in items is deleted, while authored files and reserved files are left untouched. heading titles the index (e.g. "Backlog").

func MetaTables

func MetaTables(body, key string) ([]map[string]string, bool)

MetaTables returns the array-of-tables declared under `[[meta.<key>]]`, each entry flattened to its scalar fields (sty_81bb0dde).

It is the TOML counterpart of the markdown frontmatter's indented block list:

hooks:                        [[meta.hooks]]
  - operation: create_review  operation = "create_review"
    skill: some-review        skill     = "some-review"

Generic on purpose — this package knows about frontmatter, not about hooks. The caller names the key and owns what the fields mean.

ok is false when the body is not TOML or declares no such array, which a caller reads as "not declared this way" and not as an error.

func Migrate

func Migrate(db *sql.DB) error

Migrate creates the authored_docs table on db. Idempotent.

func OKFConformance

func OKFConformance(name, body string) error

OKFConformance checks a single documents file for OKF v0.1 conformance: a concept document must carry YAML frontmatter with a non-empty `type`. Reserved files (index.md, log.md) are exempt. Returns nil when conformant.

Types

type Doc

type Doc struct {
	Kind string `json:"kind"`
	Name string `json:"name"` // filename without its extension
	Path string `json:"path"` // absolute path on disk, or embedded:<kind>/<name><ext>
	// Ext is the source extension (".md" or ".toml"). Set on embedded defaults so
	// SetDefaults can synthesise an honest provenance path; on-disk docs carry it
	// in Path already (sty_81bb0dde).
	Ext       string    `json:"ext,omitempty"`
	Headline  string    `json:"headline,omitempty"`
	Body      string    `json:"body"`
	Hash      string    `json:"hash"` // sha256 of body, hex
	Size      int64     `json:"size"`
	ModTime   time.Time `json:"mod_time"`
	IndexedAt time.Time `json:"indexed_at"`
	Embedded  bool      `json:"embedded,omitempty"` // a binary-shipped canonical default, not an on-disk file
}

Doc is one indexed authored file.

type DocRef

type DocRef struct {
	Kind string `json:"kind"`
	Name string `json:"name"`
}

DocRef identifies an authored doc by kind and name.

type OKFItem

type OKFItem struct {
	Name        string
	Type        string
	Title       string
	Description string
	Body        string
	Tags        []string
	Timestamp   time.Time
}

OKFItem is one record to materialize into an OKF folder as a read-only concept document. Name is the filename stem; Type is the required OKF type; Body is the markdown body (frontmatter is synthesized around it).

type Store

type Store struct {
	// contains filtered or unexported fields
}

Store indexes authored markdown into the authored_docs table. It also carries the binary's embedded canonical defaults, consulted ONLY as a by-name fallback in Get (sty_94da9ac9): List and Count enumerate just the on-disk .satelle docs, so a default is never shown as a project doc — it resolves by name (the gating baseline, on-demand principles) but is otherwise materialised onto disk by init. The disk index itself (Sync) stays purely file-driven.

func New

func New(db *sql.DB) *Store

New returns a Store bound to db.

func (*Store) Count

func (s *Store) Count(ctx context.Context, kind string) (int, error)

Count returns the effective doc count for a kind (empty kind = all kinds), matching List's disk+virtual overlay (sty_29e5a9a5).

func (*Store) Fingerprint

func (s *Store) Fingerprint(ctx context.Context) (string, error)

Fingerprint returns a cheap change-signal for the index — count plus the latest indexed_at — so a poller can detect mutations without loading bodies.

func (*Store) Get

func (s *Store) Get(ctx context.Context, kind, name string) (Doc, error)

Get returns one indexed doc by (kind, name), or ErrNotFound.

func (*Store) List

func (s *Store) List(ctx context.Context, kind string) ([]Doc, error)

List returns the effective docs for a kind, name-sorted: on-disk rows plus embedded defaults whose (kind,name) is not present on disk (sty_29e5a9a5 / epic:substrate-planes — virtual sparse defaults). Disk always wins. Empty kind returns every kind. Sync stays file-driven; the overlay is READ-TIME only.

func (*Store) SetDefaults

func (s *Store) SetDefaults(defs []Doc)

SetDefaults installs the embedded canonical defaults overlaid under the disk index. Each input needs only Kind, Name, and Body; the rest (Headline, Hash, synthetic Path, Embedded flag) is filled here. Replaces any prior defaults.

func (*Store) Sync

func (s *Store) Sync(ctx context.Context, dirs map[string]string, now time.Time) (SyncResult, error)

Sync brings the index in line with the markdown on disk for the given kind→dir map. For each kind it walks the dir (recursively), upserts every .md file whose size+mtime differs from the index, and prunes rows for files that disappeared. A missing dir is not an error — its rows are pruned (the kind simply has no authored content yet).

func (*Store) Watch

func (s *Store) Watch(ctx context.Context, dirs map[string]string, interval time.Duration, onSync func(SyncResult, error)) error

Watch runs Sync immediately, then on every interval tick until ctx is cancelled. onSync, if non-nil, is called with each pass's result (and any error) so callers can log progress. It returns ctx.Err() when cancelled. This is the "directory monitor": a poll loop, dependency-free.

type SyncResult

type SyncResult struct {
	Indexed int      `json:"indexed"`           // files inserted or updated
	Pruned  int      `json:"pruned"`            // index rows whose file no longer exists
	Scanned int      `json:"scanned"`           // .md files seen on disk
	Changed []DocRef `json:"changed,omitempty"` // the (kind, name) upserted this pass
}

SyncResult reports what a Sync pass changed.