bibframe

package
v0.48.0 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Jul 9, 2026 License: MIT Imports: 19 Imported by: 0

Documentation

Overview

Package bibframe crosswalks codex records into canonical BIBFRAME N-Quads grains. It wraps libcodex's record<->BIBFRAME conversion (github.com/freeeve/libcodex/bibframe) and adds libcat's provenance graphs plus RDFC-1.0 canonical emission, so each grain re-serializes byte-for-byte when unchanged -- the clean-diff invariant in ARCHITECTURE §3.

Index

Constants

View Source 
const (

	// PredBarcode and PredItemNote are lcat: extensions -- BIBFRAME models
	// them as identifier/note subgraphs, heavier than a holdings row needs.
	PredBarcode  = LcatNS + "barcode"
	PredItemNote = LcatNS + "itemNote"
)

The minimal bf:Item holdings model (tasks/051): call number, shelving location, barcode, and a note -- never circulation state, which is the ILS's (ARCHITECTURE §5 keeps live state out of the graph). Items are editorial statements on skolem nodes under their Instance, so they survive re-ingest and edit like everything else.

View Source 
const (
	// PredTombstoned marks a retired Work: object is the successor Work IRI
	// (redirect target) or the literal "true" (gone, no successor).
	PredTombstoned = LcatNS + "tombstoned"
	// PredSuppressed hides a Work from projection: object literal "true".
	PredSuppressed = LcatNS + "suppressed"
	// PredWithdrawn flags a Work whose sole bib feed no longer lists it
	// (tasks/078 reconciliation): object literal is the ISO date of the pass
	// that noticed. A flag, not a deletion -- identity and editorial
	// statements survive a title returning to the collection.
	PredWithdrawn = LcatNS + "withdrawnFromFeed"
	// PredSuppressedBy records what set the suppression ("feed-reconcile"
	// for the auto-suppress policy), so clearing a withdrawal un-suppresses
	// only suppressions that pass created.
	PredSuppressedBy = LcatNS + "suppressedBy"
	// PredFeedKept records a curator's "keep despite withdrawal" decision:
	// object literal "true". Reconciliation never re-flags a kept Work.
	PredFeedKept = LcatNS + "feedWithdrawalKept"
)

Work visibility (tasks/051): the delete stance is never row-deletion -- a tombstoned Work disappears from projection and leaves a redirect entry (to a successor Work when named, else an empty target the host serves as gone), while a suppressed Work merely hides from projection with no redirect and is fully restorable. Both are editorial statements, so they survive re-ingest like every other curation decision.

View Source 
const ExtraPred = LcatNS + "extra/"

ExtraPred is the reserved predicate namespace for adopter "extras": per-Work fields that are not BIBFRAME (e.g. cover, rating, dateRead) but a provider wants carried through to catalog.json's `extra` object (tasks/026). A key K is emitted as the predicate ExtraPred+K on the Work node, in the feed provenance graph; the projector harvests the same namespace back into Work.Extra, and the Hugo module forwards it to page params (tasks/022).

View Source 
const LcatNS = "https://github.com/freeeve/libcat/ns#"

LcatNS is libcat's own namespace for statements BIBFRAME 2.0 does not cover (ARCHITECTURE §5, "extend, don't fight the model"). Its clustering-correction predicates live in the editorial graph, so they are preserved across re-ingest and the computed clustering key cannot override a human decision (§4).

View Source 
const LocalAuthorityNS = "https://github.com/freeeve/libcat/authority/"

LocalAuthorityNS is the IRI namespace for locally minted authority terms (tasks/046). Shared vocabularies (Homosaurus, LCSH) bring their own IRIs; a heading a deployment coins itself gets one under this namespace so bf:subject references stay absolute IRIs like any other controlled term.

View Source 
const PredMARCVerbatim = LcatNS + "marcVerbatim"

PredMARCVerbatim carries a crosswalk-lossy MARC field verbatim on its Instance node (tasks/049): on MARC ingest the known-loss tags (fidelity.go) are serialized field-exact into the feed graph instead of silently dropping; the MARC view shows them, edits to them land editorially under the same predicate (with the tasks/042 override marker shadowing the feed copies), and MARC export re-attaches them so the original forms round-trip.

View Source 
const PredMergedInto = LcatNS + "mergedInto"

PredMergedInto records an editorial merge: the retired Work (subject) was merged into the surviving Work (object). It is the under-merge fix -- two records that should be one Work but clustered apart. The retired Work's grain is dropped on the next ingest (its Instances resolve onto the survivor); the projector emits a redirect so the retired id's URL survives.

View Source 
const PredOverrides = LcatNS + "overrides"

PredOverrides marks an editorial property ownership claim: the statement <subject> lcat:overrides <predicateIRI> in the editorial graph declares that, for this subject and predicate, every feed:* statement is shadowed -- consumers (projector, doc mapper, MARC materializer) show only the editorial values. This gives replace (marker + editorial values), partial removal (marker + the kept subset re-asserted editorially), full removal (marker alone), and one-click revert (delete the marker and the editorial values; the feed resurfaces) -- all without reification, and surviving re-ingest because the editorial graph is preserved verbatim (ARCHITECTURE §5, tasks/042).

View Source 
const PredSplitFrom = LcatNS + "splitFrom"

PredSplitFrom records the provenance of an editorial split: the new Work (subject) was split off from the original over-merged Work (object). It pairs with per-Instance PredWorkAssignment pins that force the split to reproduce on re-ingest even though the computed key still clusters the Instances together.

View Source 
const PredTag = LcatNS + "tag"

PredTag carries an uncontrolled folksonomy tag as a plain literal on a Work. Feed tags arrive as labeled blank bf:Topic nodes, but editorial-class statements must be blank-free, so approved community tags use this predicate instead; the projector folds both shapes into the same Tags dimension (tasks/012).

View Source 
const PredTagAlias = LcatNS + "tagAlias"

PredTagAlias records that a controlled term subsumes an uncontrolled tag: <termURI> lcat:tagAlias "tag" in an authority-class graph. Written when a folksonomy/feed tag is promoted to a controlled term (tasks/044); the projector then suppresses the tag on Works that carry the aliasing subject (the tag "became" the subject), and pickers auto-suggest the controlled term when the tag is typed.

View Source 
const PredWorkAssignment = LcatNS + "workAssignment"

PredWorkAssignment pins an Instance (subject) to a specific Work (object), overriding the computed clustering key -- the mechanism that makes an editorial split reproducible. It lives in the editorial graph, so the pin is preserved across ingest.

View Source 
const SuppressedByReconcile = "feed-reconcile"

SuppressedByReconcile is the PredSuppressedBy actor the auto-suppress reconciliation policy writes.

Variables

View Source 
var CoreFields = []string{
	"001", "006", "007", "008", "020", "040", "100", "245", "250", "260",
	"300", "306", "336", "337", "338", "347", "490", "500", "511", "520",
	"521", "533", "538", "650", "655", "700", "776", "856",
}

CoreFields are the MARC tags guaranteed to survive MARC -> BIBFRAME -> MARC on every vendored sample: the identifiers, primary/added agents, title, publication, extent, carrier, summary, subjects, genre, and access link an adopter judges fidelity by. A regression breaks the build.

View Source 
var ErrBlankNode = errors.New("bibframe: editorial patches must not use blank nodes")

ErrBlankNode reports a patch quad using a blank node.

View Source 
var KnownLoss = map[string]string{
	"037": "source of acquisition decodes as an 024-shaped identifier (vendor convention)",
	"084": "other classification number decodes as 072",
}

KnownLoss maps each tag that does NOT survive the round-trip to why -- now only the vendor-convention fields, which decode to their modeled equivalents rather than their original tags: deliberate non-goals, not losses of data. The reconstruction arc: libcodex v0.9.0 moved 008/336/500 to CoreFields (tasks/053), v0.11.0 moved 306/347/490/511/521/533/538/776 (tasks/055), and v0.12.0 finished 006/007 (tasks/056). These tags are what the lcat:marcVerbatim sidecar stores at MARC ingest so exports and the MARC view can reproduce the original forms (tasks/049). (libcodex v0.18.0 moved 040 to CoreFields: bf:AdminMetadata models the agency chain and an internal marcKey note carries the field exactly, so cataloging source is a modeled field again -- record-level provenance beside, not instead of, the named graphs; tasks/192/194.)

Functions

func AddAuthorityMergeMarker 

func AddAuthorityMergeMarker(grainNQ []byte, loserURI, winnerURI, vocab string) ([]byte, error)

AddAuthorityMergeMarker retires an authority term: <loser> lcat:mergedInto <winner> lands in the loser grain's authority:<vocab> graph, so the vocab index sees the retirement on reload and the decision survives description edits. Idempotent.

func AddMergeMarker 

func AddMergeMarker(grainNQ []byte, from, to string) ([]byte, error)

AddMergeMarker adds the editorial lcat:mergedInto statement (from -> to) to a Work grain's N-Quads and returns the re-canonicalized grain, so the decision is recorded durably in the survivor's grain and preserved across re-ingest. It is idempotent: a grain that already records this merge is returned unchanged (re-canonicalized). Thin wrapper over ApplyEditorialPatch.

func AddSplitMarkers 

func AddSplitMarkers(grainNQ []byte, newWork, fromWork string, instanceIDs []string) ([]byte, error)

AddSplitMarkers records an editorial split in a grain's N-Quads: a lcat:splitFrom provenance statement (newWork split off fromWork) plus one lcat:workAssignment pin per Instance, all in the editorial graph, and returns the re-canonicalized grain. Adding a marker that already exists is a no-op, so it is idempotent. Recorded in the source Work's grain, the pins are recovered on the next ingest and force the pinned Instances onto newWork. Thin wrapper over ApplyEditorialPatch.

func AppendAuthoritySubject 

func AppendAuthoritySubject(grainNQ []byte, workID string, subj AuthoritySubject, vocab string) ([]byte, error)

AppendAuthoritySubject records a controlled subject on a Work: the bf:subject link lands in the editorial: graph (a human/reviewed decision), while the term's own description (prefLabel per language, broader links) lands in the vocabulary's authority:<vocab> graph so the projector's label index can resolve it (tasks/012). Idempotent.

func AppendAuthorityTerms added in v0.34.0 

func AppendAuthorityTerms(grainNQ []byte, vocab string, terms []AuthoritySubject) ([]byte, error)

AppendAuthorityTerms writes standalone term descriptions (prefLabel per language, broader links) into the authority:<vocab> graph without linking them to any Work -- skos:broader ancestor-chain metadata, so hierarchy nodes no Work carries directly still resolve labels in the projection (tasks/178). Terms with no metadata contribute nothing. Idempotent.

func ApplyEditorialPatch 

func ApplyEditorialPatch(grainNQ []byte, p Patch) ([]byte, error)

ApplyEditorialPatch applies p to the grain's editorial: graph and returns the re-canonicalized grain -- the general form of the merge/split marker writers, and the write path record editing and suggestion publishing share. Additions are idempotent (an already-present statement is not duplicated); removals match exact statements in the editorial graph only.

func ApplyPatch 

func ApplyPatch(grainNQ []byte, graph rdf.Term, p Patch) ([]byte, error)

ApplyPatch applies p to one named graph of the grain and returns the re-canonicalized grain.

func ApplyShadow 

func ApplyShadow(g *rdf.Graph, overrides Overrides) *rdf.Graph

ApplyShadow returns g without the triples the overrides shadow -- the filter consumers run over feed-class graphs before reading them, so editorially-owned properties show only their editorial values.

func AuthorityGrainPath 

func AuthorityGrainPath(id string) string

AuthorityGrainPath maps an authority id to its grain path in the repo layout: data/authorities/<xx>/<id>.nq, sharded like Work grains so no directory holds 100k+ entries (ARCHITECTURE §3).

func AuthorityGraph 

func AuthorityGraph(vocab string) rdf.Term

AuthorityGraph returns the named-graph term for a controlled vocabulary's statements: the authority:<vocab> provenance class from ARCHITECTURE §5 (term labels, definitions, hierarchy). Like editorial:, authority graphs are preserved verbatim across feed re-ingest.

func BuildAuthorityGrain 

func BuildAuthorityGrain(oldGrain []byte, t AuthorityTerm, vocab string) ([]byte, error)

BuildAuthorityGrain (re)serializes a local authority term's grain: the term's statements replace the authority:<vocab> graph wholesale (the term owns its grain), preserving any statements other graphs might carry. oldGrain may be nil/empty for a fresh term.

func BuildWorkGrain 

func BuildWorkGrain(wg WorkGroup, provider string) ([]byte, error)

BuildWorkGrain serializes one WorkGroup to its canonical grain bytes -- the per-Work unit BuildWorks writes, exposed for store-backed ingest (tasks/050), where grains land through blob CAS instead of a Sink.

func ClearTombstone 

func ClearTombstone(grainNQ []byte, workID string) ([]byte, error)

ClearTombstone restores a retired Work.

func ClearWithdrawn 

func ClearWithdrawn(grainNQ []byte, workID string) ([]byte, error)

ClearWithdrawn removes a Work's withdrawal flag.

func DecodeGrainMARC 

func DecodeGrainMARC(grain []byte) ([]*codex.Record, error)

DecodeGrainMARC materializes a grain's MARC records the framework-aware way (tasks/049): editorial lcat:overrides shadow the feed statements they claim (so an edited field decodes to its editorial value, not both), and each record's verbatim sidecar fields are re-attached in tag order. The mapping of records to instance nodes mirrors libcodex Decode's one-record- per-Work+Instance enumeration; if the counts ever disagree, verbatim attachment is skipped rather than misattached.

func DecodeGrainMARCSource added in v0.47.0 

func DecodeGrainMARCSource(grain []byte, org string) ([]*codex.Record, error)

DecodeGrainMARCSource is DecodeGrainMARC plus the tasks/192 cataloging- source derivation: given the deployment's MARC organization code, each decoded record's 040 derives from graph facts AT DECODE TIME -- the field is never stored, so it cannot drift from the named-graph provenance that remains the statement-level source of truth. A record that arrived with a 040 gains the deployment as one trailing $d (modifying agency) when the grain carries editorial statements; a record with no 040 -- the born- digital feeds -- synthesizes 040 $a<org>$c<org> as its own cataloging source. An empty org code decodes unchanged.

func DecodeVerbatimField 

func DecodeVerbatimField(s string) (codex.Field, error)

DecodeVerbatimField parses the sidecar literal form back to a field.

func EditorialGraph 

func EditorialGraph() rdf.Term

EditorialGraph is the named graph for human- and authority-owned statements (ARCHITECTURE §5): merge/split decisions, curated subjects, and any hand- authored triples. Unlike feed:<provider> statements -- regenerated on every ingest -- editorial: statements are preserved verbatim across re-ingest.

func EncodeVerbatimField 

func EncodeVerbatimField(f codex.Field) string

EncodeVerbatimField serializes one MARC field to the sidecar's stable literal form.

func EnrichmentGraph 

func EnrichmentGraph(name string) rdf.Term

EnrichmentGraph returns the named-graph term for a machine-enrichment source's statements (tasks/039). Enrichment graphs are regenerable -- an enricher drop-and-replaces its own graph on each run -- but, not being feed:<provider>, they are preserved across feed re-ingest like editorial statements.

func FeedGraph 

func FeedGraph(provider string) rdf.Term

FeedGraph returns the named-graph term for a provider feed's statements: the feed:<provider> provenance graph from ARCHITECTURE §5. Feed statements are regenerated on ingest and never hand-edited.

func Grain 

func Grain(rec *codex.Record, graph rdf.Term) ([]byte, error)

Grain crosswalks one record to BIBFRAME and returns its canonical N-Quads grain: every statement tagged with the given provenance graph and RDFC-1.0 canonicalized (blank-node labels + statement order), so an unchanged record re-serializes to identical bytes.

func GrainFromGraph 

func GrainFromGraph(g *rdf.Graph, graph rdf.Term) ([]byte, error)

GrainFromGraph canonicalizes one BIBFRAME graph into its N-Quads grain, every statement tagged with the given provenance graph and RDFC-1.0 canonicalized so an unchanged input re-serializes to identical bytes.

func GrainPath 

func GrainPath(id string) string

GrainPath returns the sink-relative path for a work id: data/works/<xx>/<id>.nq, sharded by a hash prefix so no directory holds an unbounded number of files (ARCHITECTURE §3). Paths use forward slashes; the Sink maps them onto its backend.

func InstanceIRI 

func InstanceIRI(id string) string

InstanceIRI returns the node IRI libcodex mints for an Instance id.

func LocalAuthorityIRI 

func LocalAuthorityIRI(id string) string

LocalAuthorityIRI returns the absolute IRI for a locally minted authority id (identity.Mint with the "a" prefix).

func LossyTag 

func LossyTag(tag string) (string, bool)

LossyTag reports whether a tag is on the known-loss table, with the documented reason -- the editor's non-blocking warning and the sidecar's capture predicate share this one gate.

func MARCRecordNodes 

func MARCRecordNodes(grain []byte) ([]string, error)

MARCRecordNodes exposes the record -> instance-node mapping DecodeGrainMARC uses, so callers (the MARC view) can address a specific record's grain nodes. Node i corresponds to record i of DecodeGrainMARC; a Work-only record maps to its Work node.

func ReadMARC 

func ReadMARC(r io.Reader) ([]*codex.Record, error)

ReadMARC reads every record from an ISO 2709 (.mrc) stream.

func ReplaceGraph 

func ReplaceGraph(grainNQ []byte, graph rdf.Term, quads []rdf.Quad) ([]byte, error)

ReplaceGraph drops every statement in the given named graph and replaces them with quads (whose own G fields are ignored), returning the re-canonicalized grain. This is the idempotent re-enrichment primitive: an enricher owns its enrichment:<name> graph outright.

func ReplaceSubjectReference 

func ReplaceSubjectReference(grainNQ []byte, workID, loserURI string, winner AuthoritySubject, winnerVocab string) ([]byte, error)

ReplaceSubjectReference rewrites one Work's controlled-subject reference after an authority merge (tasks/046): the editorial bf:subject link to the loser is retracted along with every authority-graph statement about the loser the grain carries for the projector's label index, and the winner is appended through the same path suggestion publishing uses. Feed-graph statements are never touched (editorial rewrites do not reach into feed:<provider> graphs); a feed-asserted subject needs the tasks/042 override semantics instead.

func RetiredWorks 

func RetiredWorks(merges []identity.Merge) []string

RetiredWorks returns the sorted, distinct retired Work ids among merges (the From side of every merge). Their grains are removed on ingest once their Instances have moved to the survivor.

func ScanMerges 

func ScanMerges(nq []byte) ([]identity.Merge, error)

ScanMerges recovers the editorial merge decisions in one grain's N-Quads: every lcat:mergedInto statement in the editorial graph, as From->To Work-id pairs. A grain with none yields nil.

func ScanMergesDataset 

func ScanMergesDataset(ds *rdf.Dataset) []identity.Merge

ScanMergesDataset is ScanMerges for callers that already hold the parsed dataset (the work index scans everything off one parse).

func ScanPins 

func ScanPins(nq []byte) ([]identity.Pin, error)

ScanPins recovers the editorial split pins in one grain's N-Quads: every lcat:workAssignment statement in the editorial graph, as Instance->Work id pairs. A grain with none yields nil.

func SerializeGrains 

func SerializeGrains(dir string, sink storage.Sink) (int, error)

SerializeGrains regenerates the bulk catalog.nq from the committed per-Work grains under dir, without re-ingesting from a provider cache. It walks every grain (*.nq, skipping catalog.nq itself), and re-emits each grain's statements through one shared encoder -- so blank-node labels stay unique across the corpus (ARCHITECTURE §3) rather than colliding as an independently-canonicalized per-grain concatenation would -- in Work-id order. The grains stay the source of truth; catalog.nq is a derived merge of them.

The output represents the same RDF as an ingest-produced catalog.nq (it projects identically) but is not byte-identical to it: ingest serializes each Work's freshly built graph, whereas this re-serializes the on-disk canonical grains, so blank-node labels differ. SerializeGrains is itself deterministic -- the same grains yield the same bytes -- so a re-serialize is a clean (empty) diff. It returns the number of grains merged.

func SetFeedKept 

func SetFeedKept(grainNQ []byte, workID string, kept bool) ([]byte, error)

SetFeedKept records (or clears) a curator's keep-despite-withdrawal decision.

func SetItems 

func SetItems(grainNQ []byte, instanceID string, items []Item) ([]byte, error)

SetItems replaces an Instance's holdings wholesale: every editorial item statement under the Instance's item namespace is dropped and the given items re-asserted on freshly numbered skolem nodes. Returns the re-canonicalized grain.

func SetSuppressed 

func SetSuppressed(grainNQ []byte, workID string, suppressed bool) ([]byte, error)

SetSuppressed hides or unhides a Work from projection. Unsuppressing also drops any PredSuppressedBy provenance.

func SetSuppressedBy 

func SetSuppressedBy(grainNQ []byte, workID, actor string) ([]byte, error)

SetSuppressedBy records what set a Work's suppression.

func SetTombstone 

func SetTombstone(grainNQ []byte, workID, redirectTo string) ([]byte, error)

SetTombstone retires a Work: any prior tombstone statement is replaced by one pointing at redirectTo (a Work id) or, with redirectTo empty, the no-successor literal. Returns the re-canonicalized grain.

func SetWithdrawn 

func SetWithdrawn(grainNQ []byte, workID, date string) ([]byte, error)

SetWithdrawn flags a Work as gone from its sole bib feed on the given ISO date; ClearWithdrawn removes the flag.

func SubjectQuad 

func SubjectQuad(workID, termURI string) rdf.Quad

SubjectQuad builds the editorial statement linking a Work to a controlled subject's authority URI.

func TagAliasQuad 

func TagAliasQuad(termURI, tag string) rdf.Quad

TagAliasQuad builds the alias statement.

func TagQuad 

func TagQuad(workID, tag string) rdf.Quad

TagQuad builds the editorial statement for one approved folksonomy tag.

func VerbatimFields 

func VerbatimFields(rec *codex.Record) []string

VerbatimFields serializes a record's known-loss fields (fidelity.go) -- what a MARC ingest provider hands the grain writer for the sidecar.

func WorkID 

func WorkID(rec *codex.Record) (string, error)

WorkID returns a stable, filesystem-safe id for a record's grain, taken from the control number (MARC 001) and falling back to a hash of the record when absent. A record with no 001 whose encoding fails is an error: hashing the nil fallback would give every such record the same id, silently overwriting grains (tasks/115). Phase 0 only: ARCHITECTURE §4's identity model replaces this with a minted, provider-independent id in identity/ (Phase 1), which also changes the grain's subject IRIs and filename.

func WorkIRI 

func WorkIRI(id string) string

WorkIRI returns the node IRI libcodex mints for a Work id (the "#<id>Work" fragment the grains use), so editorial overlay statements reference the same node the feed does.

Types

type AuthoritySubject 

type AuthoritySubject struct {
	URI     string
	Labels  map[string]string // language tag -> label (e.g. "en", "es")
	Broader []string          // parent authority URIs (skos:broader)
}

AuthoritySubject is one controlled-vocabulary subject a provider asserts for a Work: a stable authority URI, its human labels by language tag, and its skos:broader parent URIs. It is the graph-emission shape behind ingest.SubjectEnricher (tasks/026).

type AuthorityTerm 

type AuthorityTerm struct {
	URI        string              `json:"uri"`
	PrefLabel  map[string]string   `json:"prefLabel"`
	AltLabel   map[string][]string `json:"altLabel,omitempty"`
	Definition map[string]string   `json:"definition,omitempty"`
	Broader    []string            `json:"broader,omitempty"`
	Narrower   []string            `json:"narrower,omitempty"`
	Related    []string            `json:"related,omitempty"`
	ExactMatch []string            `json:"exactMatch,omitempty"`
	// MergedInto records the term's retirement: it was merged into the
	// referenced term (lcat:mergedInto). Preserved across description edits.
	MergedInto string `json:"mergedInto,omitempty"`
}

AuthorityTerm is the editable description of one authority concept -- the SKOS fields the authority-topic profile exposes (tasks/046). Maps are keyed by language tag ("" = untagged).

func ParseAuthorityGrain 

func ParseAuthorityGrain(grainNQ []byte, uri, vocab string) (AuthorityTerm, error)

ParseAuthorityGrain recovers the term description for uri from an authority grain's authority:<vocab> graph -- the inverse of BuildAuthorityGrain.

func (AuthorityTerm) Quads 

func (t AuthorityTerm) Quads() ([]rdf.Quad, error)

Quads serializes the term description to its authority-graph statements. The graph field of the returned quads is unset; the caller chooses the authority:<vocab> graph.

type BuildStats 

type BuildStats struct {
	Records int // records read from the source
	Grains  int // per-Work grain files written
}

BuildStats reports what a corpus build produced.

func BuildCorpus 

func BuildCorpus(sink storage.Sink, records []*codex.Record, provider string) (BuildStats, error)

BuildCorpus writes one canonical N-Quads grain per record into sink (at GrainPath) in the provider's feed graph, plus a bulk catalog.nq. Because it writes through the Sink, the same build runs against a local directory, cloud object storage, or a git tree unchanged.

catalog.nq is not a byte-concatenation of the grain files: each grain canonicalizes its blanks to _:c14nN independently, so it is re-serialized through one shared encoder to keep blank labels unique across the corpus. All records are held in memory for the sorted bulk write; at large scale (ARCHITECTURE §3) that becomes an out-of-core / fan-out concern.

func BuildMARC 

func BuildMARC(sink storage.Sink, marc io.Reader, provider string) (BuildStats, error)

BuildMARC reads an ISO 2709 MARC stream -- e.g. an OverDrive Marketplace MARC Express export -- and builds the corpus into sink.

func BuildWorks 

func BuildWorks(sink storage.Sink, works []WorkGroup, provider string) (BuildStats, error)

BuildWorks writes one canonical N-Quads grain per Work into sink (at GrainPath(WorkID)) in the provider's feed graph, plus a bulk catalog.nq. Each grain carries the shared Work and its Instances via libcodex's WorkInstances, so a clustered Work (multiple editions/formats) is one per-Work file with minted, provider-independent ids at both tiers. A WorkGroup's preserved Editorial statements are merged back in, so a feed re-ingest is clobber-safe (§5). It reports the number of Works (grains) and Instances written.

type GroupInstance 

type GroupInstance struct {
	InstanceID string
	Instance   codexbf.Instance
	// Verbatim carries the record's crosswalk-lossy MARC fields serialized
	// field-exact (EncodeVerbatimField), emitted into the feed graph under
	// PredMARCVerbatim so nothing is silently dropped (tasks/049). Empty for
	// non-MARC providers, leaving the grain unchanged.
	Verbatim []string
}

GroupInstance is one Instance of a WorkGroup: its minted id and Instance-level BIBFRAME.

type Item 

type Item struct {
	ID         string `json:"id"`
	CallNumber string `json:"callNumber,omitempty"`
	Location   string `json:"location,omitempty"`
	Barcode    string `json:"barcode,omitempty"`
	Note       string `json:"note,omitempty"`
}

Item is one holding of an Instance.

func ItemsOf 

func ItemsOf(grainNQ []byte, instanceID string) ([]Item, error)

ItemsOf reads an Instance's holdings from a grain, sorted by node id.

type Overrides 

type Overrides map[string]map[string]bool

Overrides is the shadow set scanned from a dataset: subject IRI -> overridden predicate IRIs.

func ScanOverrides 

func ScanOverrides(ds *rdf.Dataset) Overrides

ScanOverrides collects the editorial lcat:overrides markers from a parsed dataset.

func (Overrides) Shadows 

func (o Overrides) Shadows(subject, predicate string) bool

Shadows reports whether (subject, predicate) is editorially owned.

type Patch 

type Patch struct {
	Add    []rdf.Quad
	Remove []rdf.Quad
}

Patch is a set of statement additions and removals applied to one provenance graph of a grain. Each quad's own G field is ignored: the target graph is chosen by the applying function. Subjects and objects must be IRIs or literals -- blank nodes are rejected because editorial-class statements must not introduce blank labels that could collide with the feed graph's during joint canonicalization (see grainWithEditorial).

func OverridePatch 

func OverridePatch(subjectIRI string, predicates ...string) Patch

OverridePatch builds the editorial patch claiming ownership of the given predicates on one subject node (apply with ApplyEditorialPatch; pair with the replacement values in the same patch).

func RevertPatch 

func RevertPatch(subjectIRI string, predicates ...string) Patch

RevertPatch builds the editorial patch releasing ownership (the removal half of revert; the caller also removes its editorial replacement values).

type Prior 

type Prior struct {
	Grains    []identity.GrainIdentity
	Editorial map[string][]byte // Work id -> raw N-Quads of its non-feed statements
	Merges    []identity.Merge  // editorial lcat:mergedInto decisions to seed (tasks/001)
	Pins      []identity.Pin    // editorial lcat:workAssignment split pins to seed (tasks/001)
}

Prior is the committed state a re-ingest recovers from the grains under a build directory: the identity to seed the resolver (so ids do not churn) and the editorial statements to preserve per Work (so a feed re-ingest is clobber-safe). It is the read side of the derive-from-grains model (ARCHITECTURE §4/§5, Decision A): the grains are the durable identity map and the editorial store.

func LoadPrior 

func LoadPrior(dir, provider string) (Prior, error)

LoadPrior reads every per-Work grain (*.nq, skipping the bulk catalog.nq) under dir, returning the recovered identity and the preserved editorial statements keyed by Work id. A missing directory (a first build) yields empty state and no error.

func LoadPriorStore 

func LoadPriorStore(ctx context.Context, st blob.Store, prefix, provider string) (Prior, map[string]string, error)

LoadPriorStore is LoadPrior over a blob.Store: it reads every per-Work grain (*.nq, skipping the bulk catalog.nq) under prefix, returning the recovered Prior plus each grain's ETag keyed by path. The ETags feed conditional writes: a re-ingest that read a grain at etag E writes it back with IfMatch E, so an editorial publish landing mid-ingest surfaces as ErrPreconditionFailed instead of being clobbered (the writer re-reads, re-extracts non-feed graphs, unions, and retries). An empty tree (a first build) yields empty state and no error.

type WorkGroup 

type WorkGroup struct {
	WorkID    string
	Work      codexbf.Work
	Instances []GroupInstance
	// Editorial is the raw N-Quads of the Work's human/authority-owned
	// statements, preserved verbatim from the prior grain so a feed re-ingest
	// never clobbers them (ARCHITECTURE §5). Empty when there are none.
	Editorial []byte
	// Extras are the Work's non-BIBFRAME adopter display fields (e.g. cover,
	// rating, dateRead) that a provider carries through to catalog.json's `extra`
	// object (tasks/026). They are emitted into the feed provenance graph under
	// ExtraPred, so their origin is tracked like every other feed statement. Empty
	// when the provider supplies none, leaving the grain byte-for-byte unchanged.
	Extras map[string]string
	// Subjects are controlled-vocabulary subjects (authority URI + localized labels +
	// skos:broader) a provider derived for the Work, e.g. by promoting genre tags
	// through an authority table (tasks/026). They are emitted into the feed graph as
	// a bf:subject link to the URI plus the authority's prefLabel/broader statements,
	// so the projector resolves them as controlled subjects (tasks/012/015). Empty
	// when the provider supplies none.
	Subjects []AuthoritySubject
	// Terms are standalone vocabulary term descriptions -- typically the
	// skos:broader ancestor chains of Subjects (tasks/180). They are emitted
	// into the feed graph as the term's prefLabel/broader statements with no
	// bf:subject link, so ancestor concepts stay labeled in the projection's
	// term sideband (tasks/178). Empty when the provider supplies none.
	Terms []AuthoritySubject
}

WorkGroup is one clustered Work ready to serialize: its minted id, its shared Work-level BIBFRAME, and the Instances (each with its own minted id) that realize it. It is the direct-BIBFRAME, two-tier-identity unit a native provider produces after resolution (ARCHITECTURE §4).

type WorkVisibility 

type WorkVisibility struct {
	Tombstoned bool   `json:"tombstoned"`
	RedirectTo string `json:"redirectTo,omitempty"` // successor Work id, when named
	Suppressed bool   `json:"suppressed"`
	// Withdrawn is the date the feed reconciliation flagged this Work as
	// gone from its sole bib feed ("" = not withdrawn).
	Withdrawn string `json:"withdrawn,omitempty"`
	// SuppressedBy names what set the suppression ("" = a curator).
	SuppressedBy string `json:"suppressedBy,omitempty"`
	// Kept records a curator's decision to keep the Work despite withdrawal.
	Kept bool `json:"kept,omitempty"`
}

WorkVisibility is one Work's projection stance.

func Visibility 

func Visibility(grainNQ []byte, workID string) (WorkVisibility, error)

Visibility reads a Work's stance from its grain.

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.