vocab

Documentation

Overview

Sidecar index reader: serves one scheme from the artifacts BuildSidecar wrote, holding only the small structures resident -- the URI and identifier RRILs, the RRTI search router, and the RRSR offset index -- while search postings and Term payloads range-fetch from the store on demand (a bounded cache absorbs the editor's hot set). Reads are lock-free except the cache. Unlike the map path, sidecar reads can fail (the store is remote): failures log and report a miss, never an invented term.

Sidecar index builder: serializes one scheme's terms into range-servable roaringrange artifacts so the server never materializes a big vocabulary as Go maps. The on-disk layout, the manifest shape and the remove/orphan lifecycle live in storage/vocabsidecar (root); this file is the builder that writes into that layout.

Doc ids are the scheme's term URIs in sorted order, so RRIL postings for one key surface the smallest URI first and output is deterministic.

Package vocab loads controlled vocabularies from SKOS authority grains and serves the in-memory term index behind term validation, the picker's autocomplete, and neighborhood browsing. A vocabulary's quads live in its authority:<vocab> named graph (ARCHITECTURE §5), so the loader routes terms to schemes by graph name -- one authorities tree can carry homosaurus, lcsh, and local terms side by side. This replaces qllpoc's embedded homosaurus-min.json with a vocabulary-agnostic store-backed load.

Index

• Constants
• Variables
• func BuildSidecar(ctx context.Context, st blob.Store, prefix, scheme, source string) (*vocabsidecar.SidecarManifest, error)
• func NormalizeFolk(raw string) (string, error)
• func PickLabel(labels map[string]string, prefer ...string) string
• type Equivalent
• type Index
  • func Load(ctx context.Context, st blob.Store, prefix string, schemes []string) (*Index, error)
  • func (ix *Index) Ancestors(scheme, id string) []*Term
  • func (ix *Index) Equivalents(uri string) ([]Equivalent, bool)
  • func (ix *Index) LabelResolver() func(iri string) (string, map[string]string, bool)
  • func (ix *Index) Lookup(scheme, id string) (*Term, bool)
  • func (ix *Index) MatchIdentifier(uri string) (*Term, bool)
  • func (ix *Index) MatchLabel(scheme, label string) []LabelMatch
  • func (ix *Index) Path(scheme, id string) []TermRef
  • func (ix *Index) Reload(ctx context.Context, st blob.Store, prefix string, schemes []string) error
  • func (ix *Index) Resolve(id string) (*Term, bool)
  • func (ix *Index) SchemeStats(scheme string) (sidecar bool, residentTerms int)
  • func (ix *Index) Schemes() []string
  • func (ix *Index) Search(scheme, q string, limit int) []*Term
  • func (ix *Index) Terms(scheme string) []*Term
• type LabelMatch
• type Term
  • func (t *Term) Label(lang string) string
• type TermRef

Constants

const FolkScheme = "folk"

FolkScheme is the reserved scheme key for folksonomy tags -- community terms that are not (yet) in any controlled vocabulary. Folk terms carry their normalized text as the TermRef ID.

Variables

var ErrBadFolkTerm = errors.New("vocab: unusable folksonomy term")

ErrBadFolkTerm reports a raw tag the normalizer refuses.

Functions

func BuildSidecar

func BuildSidecar(ctx context.Context, st blob.Store, prefix, scheme, source string) (*vocabsidecar.SidecarManifest, error)

BuildSidecar builds and stores the sidecar artifacts for scheme from the installed snapshot at source (usually <prefix>vocab/<scheme>.nq). It parses the snapshot with the same routing the map loader uses, so the two paths index identical terms.

func NormalizeFolk

func NormalizeFolk(raw string) (string, error)

NormalizeFolk canonicalizes a raw community tag: Unicode NFKC, lowercase, whitespace collapsed, length-bounded, with control characters, URLs, and markup rejected outright. The result is the tag's identity (dedup key and TermRef ID); raw patron text itself never reaches the graph -- a normalized novel term still has to pass moderation before it becomes suggestible.

func PickLabel

func PickLabel(labels map[string]string, prefer ...string) string

PickLabel returns the best display label from a language-tagged label map (the convention every label-bearing shape shares): each preferred tag in order, then English, then untagged, then the lexicographically first remaining tag -- deterministic where map order is not. Empty-string labels never win; "" means no usable label, and the caller supplies its own fallback (term URI, raw ID).

Types

type Equivalent

type Equivalent struct {
	ID     string            `json:"id"`
	Scheme string            `json:"scheme,omitempty"`
	Labels map[string]string `json:"labels,omitempty"`
	// Strength: "exact" | "close" (direct links) | "pivot-exact" |
	// "pivot-close" (transitive; the weaker hop names it).
	Strength string `json:"strength"`
	// Via is the pivot URI for transitive equivalents.
	Via string `json:"via,omitempty"`
	// Known is false when the URI is only a link target -- not a term in any
	// loaded vocabulary. Still reported: an LCSH link is meaningful even when
	// LCSH is not snapshot locally.
	Known bool `json:"known"`
}

Equivalent is one cross-scheme equivalent of a term, found through skos:exactMatch/closeMatch links in either direction, or through a one-hop pivot (two terms linking the same intermediate URI -- the FAST -> LCSH <- Homosaurus shape). Strength names the weakest hop, because a suggestion is only as good as its shakiest link; the editor must say which.

type Index

type Index struct {
	// contains filtered or unexported fields
}

Index is the loaded term index. Reads are lock-free over an immutable snapshot; Reload builds a fresh snapshot and swaps it atomically, so every holder of the *Index (terms handler, suggestion gate, publisher) sees authority edits without rewiring.

func Load

func Load(ctx context.Context, st blob.Store, prefix string, schemes []string) (*Index, error)

Load reads every authority grain under prefix from the store and indexes the terms of the requested schemes (nil/empty schemes = all authority graphs found).

func (*Index) Ancestors

func (ix *Index) Ancestors(scheme, id string) []*Term

Ancestors returns the term's full transitive skos:broader closure (the term itself excluded), breadth-first so nearer ancestors come first, deterministic (broader lists are sorted at load), cycle-safe, and depth-capped. Unlike Path -- one shortest chain for breadcrumbs -- this is every ancestor: what a consumer materializing hierarchy metadata (ancestor term descriptions on enrichment/publish) needs for polyhierarchical terms. Broader URIs that do not resolve in the scheme are skipped. A root or unknown term yields nil.

func (*Index) Equivalents

func (ix *Index) Equivalents(uri string) ([]Equivalent, bool)

Equivalents returns a term's cross-scheme equivalents in strength order: direct outbound links, direct inbound links (terms whose matches point at it -- the load-bearing direction for community vocabularies that link TO LCSH), and one-hop pivots in both shapes (shared link target; a linked term's other links). The source term itself is excluded; duplicates keep their strongest strength. False is returned when the URI is not a term in any loaded vocabulary.

func (*Index) LabelResolver

func (ix *Index) LabelResolver() func(iri string) (string, map[string]string, bool)

LabelResolver adapts the index to the editor's label-companion contract (editor.LabelResolver): term IRI -> (scheme, lang->prefLabel). Safe on a nil index -- it returns nil, which disables companions.

func (*Index) Lookup

func (ix *Index) Lookup(scheme, id string) (*Term, bool)

Lookup returns the term by scheme and URI -- the validation gate: only terms that resolve here are accepted into suggestions or subject edits.

func (*Index) MatchIdentifier

func (ix *Index) MatchIdentifier(uri string) (*Term, bool)

MatchIdentifier returns the live term a canonicalized identifier URI resolves to: the term's own URI first, then its skos:exactMatch and closeMatch siblings -- the identifier-based reconciliation gate for external headings that carry $0. Within a tier, map and sidecar candidates resolve to the smallest scheme then URI.

func (*Index) MatchLabel

func (ix *Index) MatchLabel(scheme, label string) []LabelMatch

MatchLabel returns the scheme's terms whose pref or alt label normalizes exactly to label -- the auto-linking gate: only whole-heading matches produce suggestions, never prefix guesses.

func (*Index) Path

func (ix *Index) Path(scheme, id string) []TermRef

Path returns the term's ancestor chain as TermRefs ordered root → … → direct parent (the term itself is excluded), following skos:broader. A polyhierarchical term takes the shortest chain to a root, ties broken by URI order; cycles and broader URIs missing from the scheme terminate the walk. A root term (or unknown term) yields nil.

func (*Index) Reload

func (ix *Index) Reload(ctx context.Context, st blob.Store, prefix string, schemes []string) error

Reload rebuilds the index from the store and atomically swaps it in -- the post-authority-edit refresh path. A failed reload leaves the previous snapshot serving.

func (*Index) Resolve

func (ix *Index) Resolve(id string) (*Term, bool)

Resolve returns the term for a URI regardless of scheme (schemes checked in sorted order for determinism) -- the editor's chip renderer resolves stored subject references without knowing where they came from. A Homosaurus IRI whose release segment differs from the installed release's resolves through its version-stable homoit id; Lookup, the write-side validation gate, stays exact so edits store the installed release's canonical IRI.

func (*Index) SchemeStats

func (ix *Index) SchemeStats(scheme string) (sidecar bool, residentTerms int)

SchemeStats reports how a scheme is served: sidecar is true when it is artifact-backed (its terms stay on disk), and residentTerms is the count held in resident maps -- the whole scheme for a map-backed one, or just the live-pick overlay for a sidecar-backed one. Together they turn an unexplained process RSS into a one-line answer per scheme. Nil-safe: a nil index reports (false, 0).

func (*Index) Schemes

func (ix *Index) Schemes() []string

Schemes lists the loaded vocabulary keys, sorted.

func (*Index) Search

func (ix *Index) Search(scheme, q string, limit int) []*Term

Search returns up to limit terms whose pref or alt label (any language) starts with q, deduped, ordered by label.

func (*Index) Terms

func (ix *Index) Terms(scheme string) []*Term

Terms returns every term of a scheme ordered by label -- the authorities management listing. Retired terms are included (marked by MergedInto).

type LabelMatch

type LabelMatch struct {
	Term *Term
	Alt  bool
}

LabelMatch is one exact-label hit: the term plus whether the match came through an alt (used-for) label rather than a preferred label.

type Term

type Term struct {
	Scheme     string              `json:"scheme"`
	ID         string              `json:"id"`                   // the authority URI
	Labels     map[string]string   `json:"labels"`               // lang -> prefLabel ("" key = untagged)
	AltLabels  map[string][]string `json:"altLabels,omitempty"`  // lang -> used-for labels
	Definition map[string]string   `json:"definition,omitempty"` // lang -> scope note
	Broader    []string            `json:"broader,omitempty"`
	Narrower   []string            `json:"narrower,omitempty"`
	Related    []string            `json:"related,omitempty"`
	ExactMatch []string            `json:"exactMatch,omitempty"`
	CloseMatch []string            `json:"closeMatch,omitempty"`
	// MergedInto marks a retired term: it was merged into the referenced
	// URI (lcat:mergedInto). Retired terms resolve via Lookup
	// (so old references still label) but leave the search index.
	MergedInto string `json:"mergedInto,omitempty"`
}

Term is one controlled-vocabulary concept.

func (*Term) Label

func (t *Term) Label(lang string) string

Label returns the term's best label for lang: exact match, then English, then untagged, then any; the term URI when no label exists.

type TermRef

type TermRef struct {
	Scheme string `json:"scheme"`
	ID     string `json:"id"`
	Label  string `json:"label,omitempty"`
}

TermRef points at a suggestible term: a controlled-vocabulary concept (Scheme = a loaded vocabulary key, ID = authority URI) or a folksonomy tag (Scheme = FolkScheme, ID = normalized tag text). Label is display-only and never trusted for identity.