vocab

Documentation

Overview

Sidecar index reader (tasks/167): serves one scheme from the artifacts BuildSidecar wrote, holding only the small structures resident -- the URI and identifier RRILs, the RRTI search router, and the RRSR offset index -- while search postings and Term payloads range-fetch from the store on demand (a bounded cache absorbs the editor's hot set). Reads are lock-free except the cache. Unlike the map path, sidecar reads can fail (the store is remote): failures log and report a miss, never an invented term.

Sidecar index builder (tasks/167): serializes one scheme's terms into range-servable roaringrange artifacts so the server never materializes a big vocabulary as Go maps. Layout under <prefix>sidecar/:

<scheme>.rrsr.bin/.idx  full Term JSON per doc (RRSR record store)
<scheme>.uri.rril       term URI -> doc, retired terms included
<scheme>.id1/2/3.rril   canon identifier tiers (own/exactMatch/closeMatch),
                        live terms only -- MatchIdentifier's precedence
<scheme>.search.rrt     RRTI over normalized labels; posting IDs doc<<1|alt
<scheme>.manifest.json  source snapshot path+ETag; presence arms the scheme

Doc ids are the scheme's term URIs in sorted order, so RRIL postings for one key surface the smallest URI first and output is deterministic.

Package vocab loads controlled vocabularies from SKOS authority grains and serves the in-memory term index behind term validation, the picker's autocomplete, and neighborhood browsing. A vocabulary's quads live in its authority:<vocab> named graph (ARCHITECTURE §5), so the loader routes terms to schemes by graph name -- one authorities tree can carry homosaurus, lcsh, and local terms side by side. This replaces qllpoc's embedded homosaurus-min.json with a vocabulary-agnostic store-backed load.

Index

• Constants
• Variables
• func NormalizeFolk(raw string) (string, error)
• func PickLabel(labels map[string]string, prefer ...string) string
• type Index
  • func Load(ctx context.Context, st blob.Store, prefix string, schemes []string) (*Index, error)
  • func (ix *Index) Ancestors(scheme, id string) []*Term
  • func (ix *Index) LabelResolver() func(iri string) (string, map[string]string, bool)
  • func (ix *Index) Lookup(scheme, id string) (*Term, bool)
  • func (ix *Index) MatchIdentifier(uri string) (*Term, bool)
  • func (ix *Index) MatchLabel(scheme, label string) []LabelMatch
  • func (ix *Index) Path(scheme, id string) []TermRef
  • func (ix *Index) Reload(ctx context.Context, st blob.Store, prefix string, schemes []string) error
  • func (ix *Index) Resolve(id string) (*Term, bool)
  • func (ix *Index) Schemes() []string
  • func (ix *Index) Search(scheme, q string, limit int) []*Term
  • func (ix *Index) Terms(scheme string) []*Term
• type LabelMatch
• type SidecarManifest
  • func BuildSidecar(ctx context.Context, st blob.Store, prefix, scheme, source string) (*SidecarManifest, error)
• type Term
  • func (t *Term) Label(lang string) string
• type TermRef

Constants

const FolkScheme = "folk"

FolkScheme is the reserved scheme key for folksonomy tags -- community terms that are not (yet) in any controlled vocabulary. Folk terms carry their normalized text as the TermRef ID.

Variables

var ErrBadFolkTerm = errors.New("vocab: unusable folksonomy term")

ErrBadFolkTerm reports a raw tag the normalizer refuses.

Functions

func NormalizeFolk

func NormalizeFolk(raw string) (string, error)

NormalizeFolk canonicalizes a raw community tag: Unicode NFKC, lowercase, whitespace collapsed, length-bounded, with control characters, URLs, and markup rejected outright. The result is the tag's identity (dedup key and TermRef ID); raw patron text itself never reaches the graph -- a normalized novel term still has to pass moderation before it becomes suggestible.

func PickLabel

func PickLabel(labels map[string]string, prefer ...string) string

PickLabel returns the best display label from a language-tagged label map (the convention every label-bearing shape shares, tasks/116): each preferred tag in order, then English, then untagged, then the lexicographically first remaining tag -- deterministic where map order is not. Empty-string labels never win; "" means no usable label, and the caller supplies its own fallback (term URI, raw ID).

Types

type Index

type Index struct {
	// contains filtered or unexported fields
}

Index is the loaded term index. Reads are lock-free over an immutable snapshot; Reload builds a fresh snapshot and swaps it atomically, so every holder of the *Index (terms handler, suggestion gate, publisher) sees authority edits without rewiring (tasks/046).

func Load

func Load(ctx context.Context, st blob.Store, prefix string, schemes []string) (*Index, error)

Load reads every authority grain under prefix from the store and indexes the terms of the requested schemes (nil/empty schemes = all authority graphs found).

func (*Index) Ancestors

func (ix *Index) Ancestors(scheme, id string) []*Term

Ancestors returns the term's full transitive skos:broader closure (the term itself excluded), breadth-first so nearer ancestors come first, deterministic (broader lists are sorted at load), cycle-safe, and depth-capped. Unlike Path -- one shortest chain for breadcrumbs -- this is every ancestor: what a consumer materializing hierarchy metadata (ancestor term descriptions on enrichment/publish, tasks/178) needs for polyhierarchical terms. Broader URIs that do not resolve in the scheme are skipped. A root or unknown term yields nil.

func (*Index) LabelResolver

func (ix *Index) LabelResolver() func(iri string) (string, map[string]string, bool)

LabelResolver adapts the index to the editor's label-companion contract (editor.LabelResolver, tasks/145): term IRI -> (scheme, lang->prefLabel). Safe on a nil index -- it returns nil, which disables companions.

func (*Index) Lookup

func (ix *Index) Lookup(scheme, id string) (*Term, bool)

Lookup returns the term by scheme and URI -- the validation gate: only terms that resolve here are accepted into suggestions or subject edits.

func (*Index) MatchIdentifier

func (ix *Index) MatchIdentifier(uri string) (*Term, bool)

MatchIdentifier returns the live term a canonicalized identifier URI resolves to: the term's own URI first, then its skos:exactMatch and closeMatch siblings -- the identifier-based reconciliation gate for external headings that carry $0 (tasks/088). Within a tier, map and sidecar candidates resolve to the smallest scheme then URI.

func (*Index) MatchLabel

func (ix *Index) MatchLabel(scheme, label string) []LabelMatch

MatchLabel returns the scheme's terms whose pref or alt label normalizes exactly to label -- the auto-linking gate (tasks/046): only whole-heading matches produce suggestions, never prefix guesses.

func (*Index) Path

func (ix *Index) Path(scheme, id string) []TermRef

Path returns the term's ancestor chain as TermRefs ordered root → … → direct parent (the term itself is excluded), following skos:broader. A polyhierarchical term takes the shortest chain to a root, ties broken by URI order; cycles and broader URIs missing from the scheme terminate the walk. A root term (or unknown term) yields nil.

func (*Index) Reload

func (ix *Index) Reload(ctx context.Context, st blob.Store, prefix string, schemes []string) error

Reload rebuilds the index from the store and atomically swaps it in -- the post-authority-edit refresh path. A failed reload leaves the previous snapshot serving.

func (*Index) Resolve

func (ix *Index) Resolve(id string) (*Term, bool)

Resolve returns the term for a URI regardless of scheme (schemes checked in sorted order for determinism) -- the editor's chip renderer resolves stored subject references without knowing where they came from (tasks/071). A Homosaurus IRI whose release segment differs from the installed release's resolves through its version-stable homoit id (tasks/188); Lookup, the write-side validation gate, stays exact so edits store the installed release's canonical IRI.

func (*Index) Schemes

func (ix *Index) Schemes() []string

Schemes lists the loaded vocabulary keys, sorted.

func (*Index) Search

func (ix *Index) Search(scheme, q string, limit int) []*Term

Search returns up to limit terms whose pref or alt label (any language) starts with q, deduped, ordered by label.

func (*Index) Terms

func (ix *Index) Terms(scheme string) []*Term

Terms returns every term of a scheme ordered by label -- the authorities management listing. Retired terms are included (marked by MergedInto).

type LabelMatch

type LabelMatch struct {
	Term *Term
	Alt  bool
}

LabelMatch is one exact-label hit: the term plus whether the match came through an alt (used-for) label rather than a preferred label.

type SidecarManifest

type SidecarManifest struct {
	Version    int    `json:"version"`
	Scheme     string `json:"scheme"`
	Source     string `json:"source"`
	SourceETag string `json:"sourceETag"`
	// SourceSchemes lists every authority scheme the source file carries --
	// the loader may skip parsing the file only when all of them are
	// sidecar-armed, so a shared source never silently drops a scheme.
	SourceSchemes []string `json:"sourceSchemes"`
	Terms         int      `json:"terms"`
	Live          int      `json:"live"`
}

SidecarManifest arms a scheme for sidecar serving: it names the source snapshot (and its ETag at build time) the artifacts were built from. A mismatched or missing source, or loose quads for the scheme elsewhere in the authorities tree, bypasses the sidecar for that snapshot build -- the map path remains the correctness backstop.

func BuildSidecar

func BuildSidecar(ctx context.Context, st blob.Store, prefix, scheme, source string) (*SidecarManifest, error)

BuildSidecar builds and stores the sidecar artifacts for scheme from the installed snapshot at source (usually <prefix>vocab/<scheme>.nq). It parses the snapshot with the same routing the map loader uses, so the two paths index identical terms.

type Term

type Term struct {
	Scheme     string              `json:"scheme"`
	ID         string              `json:"id"`                   // the authority URI
	Labels     map[string]string   `json:"labels"`               // lang -> prefLabel ("" key = untagged)
	AltLabels  map[string][]string `json:"altLabels,omitempty"`  // lang -> used-for labels
	Definition map[string]string   `json:"definition,omitempty"` // lang -> scope note
	Broader    []string            `json:"broader,omitempty"`
	Narrower   []string            `json:"narrower,omitempty"`
	Related    []string            `json:"related,omitempty"`
	ExactMatch []string            `json:"exactMatch,omitempty"`
	CloseMatch []string            `json:"closeMatch,omitempty"`
	// MergedInto marks a retired term: it was merged into the referenced
	// URI (lcat:mergedInto, tasks/046). Retired terms resolve via Lookup
	// (so old references still label) but leave the search index.
	MergedInto string `json:"mergedInto,omitempty"`
}

Term is one controlled-vocabulary concept.

func (*Term) Label

func (t *Term) Label(lang string) string

Label returns the term's best label for lang: exact match, then English, then untagged, then any; the term URI when no label exists.

type TermRef

type TermRef struct {
	Scheme string `json:"scheme"`
	ID     string `json:"id"`
	Label  string `json:"label,omitempty"`
}

TermRef points at a suggestible term: a controlled-vocabulary concept (Scheme = a loaded vocabulary key, ID = authority URI) or a folksonomy tag (Scheme = FolkScheme, ID = normalized tag text). Label is display-only and never trusted for identity.