wikidata

Documentation

Overview

Package wikidata resolves a Work's creators to Wikidata entities and caches their EXPLICITLY-STATED demographic claims as enrichment statements -- the creator-demographics half of the diversity-audit feature.

The contract, in order of importance:

• No name inference, ever. Resolution goes ISBN -> edition (P212/P957) -> work (P629) -> author (P50): every hop is a cataloged identifier or an explicit Wikidata statement. A creator's name is never matched against anything, and a work without a resolvable identifier yields nothing.
• Explicit claims only, with provenance. Only the values Wikidata states outright for P21 (sex or gender), P27 (country of citizenship), P91 (sexual orientation), and P172 (ethnic group) are recorded, each as the claim's own entity URI plus label, alongside the QID it came from, what identifier matched it, and the retrieval date. Birth/death dates are deliberately not fetched.
• Aggregate use. These statements exist so a collection-level audit can report distributions with coverage; they are enrichment-graph data, not display fields, and the projector does not surface them on work pages.

Coverage will be partial and skewed (Wikidata's own coverage is); the audit reading this data is responsible for reporting match rate and unknowns first. Statements land in the enrichment:wikidata graph, dropped and replaced on each run, so a re-run refreshes and a removed claim upstream disappears here too.

Index

• Constants
• type Doer
• type Enricher
  • func New(opts ...Option) *Enricher
  • func (e *Enricher) Enrich(ctx context.Context, works []ingest.WorkSummary) ([]ingest.Enrichment, error)
  • func (e *Enricher) Name() string
  • func (e *Enricher) RunStats() ingest.EnrichStats
  • func (e *Enricher) Skipped() int
• type Option
  • func WithClient(d Doer) Option
  • func WithDelay(d time.Duration) Option
  • func WithEndpoint(u string) Option
  • func WithLogger(l *slog.Logger) Option
  • func WithRetryBase(d time.Duration) Option

Constants

const DefaultEndpoint = "https://query.wikidata.org/sparql"

DefaultEndpoint is the public Wikidata Query Service SPARQL endpoint.

const Name = "wikidata"

Name is the enrichment source name; statements land in enrichment:wikidata.

Types

type Doer

type Doer interface {
	Do(req *http.Request) (*http.Response, error)
}

Doer is the HTTP seam, injectable for tests.

type Enricher

type Enricher struct {
	// contains filtered or unexported fields
}

Enricher resolves creators via the Wikidata Query Service.

func New

func New(opts ...Option) *Enricher

New returns the Wikidata creator-demographics enricher.

func (*Enricher) Enrich

func (e *Enricher) Enrich(ctx context.Context, works []ingest.WorkSummary) ([]ingest.Enrichment, error)

Enrich resolves each Work's creators by cataloged identifiers, in two passes: author authority IDs first (VIAF/LCNAF/ISNI/ORCID -> the direct wdt:P214/P244/P213/P496 lookups -- indexed exact matches, the high-recall path), then ISBN -> edition -> author for works the first pass left unmatched. Works with no resolvable identifier, or whose identifiers Wikidata does not know, are absent from the result -- RunEnrich leaves them untouched, and the audit reports them as unmatched rather than guessing. Never a name, in either pass.

func (*Enricher) Name

func (e *Enricher) Name() string

Name implements ingest.Enricher.

func (*Enricher) RunStats

func (e *Enricher) RunStats() ingest.EnrichStats

RunStats implements ingest.StatsReporter: the run's counters. Safe to call while Enrich runs -- counters advance per batch, which is what makes a polled progress display possible.

func (*Enricher) Skipped

func (e *Enricher) Skipped() int

Skipped reports how many batches the last Enrich call abandoned after retries; their works were left untouched for a re-run to backfill.

type Option

type Option func(*Enricher)

Option configures the enricher.

func WithClient

func WithClient(d Doer) Option

WithClient injects the HTTP client (tests; a caller with its own limits).

func WithDelay

func WithDelay(d time.Duration) Option

WithDelay overrides the politeness pause between SPARQL batches.

func WithEndpoint

func WithEndpoint(u string) Option

WithEndpoint points at a different SPARQL endpoint (a mirror, a test stub).

func WithLogger

func WithLogger(l *slog.Logger) Option

WithLogger wires progress logging: per-batch INFO lines, retry and skipped-batch WARNs -- a multi-minute synchronous run must be observable from the server log. Nil (the default) logs nothing.

func WithRetryBase

func WithRetryBase(d time.Duration) Option

WithRetryBase overrides the first retry pause (tests use 0).