diversity

Documentation

Overview

Package diversity maps a work's controlled subjects to diversity-audit categories (libcat tasks/365). It is the "content demographics" half of the diversity-audit feature: what works are *about*, derived from their subject headings. Creator demographics are a separate, opt-in axis (tasks/368).

The crosswalk is data-driven from an embedded TOML seed, which an operator may override with their own file. Matching a subject to a category is a statement about the work's topic, never about the identity of its creators.

Index

• type Auditor
  • func NewAuditor(cw *Crosswalk) *Auditor
  • func (a *Auditor) Add(subjects []SubjectRef)
  • func (a *Auditor) Report() Report
• type Category
• type CategoryTally
• type Crosswalk
  • func Default() *Crosswalk
  • func Load(overridePaths ...string) (*Crosswalk, error)
  • func (c *Crosswalk) Categories() []Category
  • func (c *Crosswalk) Categorize(uri, label string) []string
  • func (c *Crosswalk) CategorizeSubjects(subjects []struct{ ... }) []string
  • func (c *Crosswalk) Label(id string) string
• type Report
• type SubjectRef

Types

type Auditor

type Auditor struct {
	// contains filtered or unexported fields
}

Auditor streams works into a coverage-first content-diversity tally. Build it once from a crosswalk, Add each work's subjects, then read Report. It is not safe for concurrent Add.

func NewAuditor

func NewAuditor(cw *Crosswalk) *Auditor

NewAuditor returns an Auditor over the given crosswalk.

func (*Auditor) Add

func (a *Auditor) Add(subjects []SubjectRef)

Add folds one work's subjects into the tally. A work counts toward CoveredWorks when it carries at least one subject with a URI or a non-empty label, and toward a category once when any of its subjects maps there. A work with no usable subjects contributes only to TotalWorks -- it dilutes coverage, which is the point of reporting coverage.

func (*Auditor) Report

func (a *Auditor) Report() Report

Report snapshots the tally as a coverage-first Report, with categories in the crosswalk's stable reporting order. Shares are 0 when their denominator is 0.

type Category

type Category struct {
	ID       string   `toml:"id"`
	Label    string   `toml:"label"`
	Keywords []string `toml:"keywords"`
	URIs     []string `toml:"uris"`
}

Category is one diversity-audit category and the controlled subjects that map to it: an exact authority-URI match or a whole-word/phrase match on a subject's heading label. The taxonomy is an editorial choice; this is just its data shape.

type CategoryTally

type CategoryTally struct {
	ID           string  `json:"id"`
	Label        string  `json:"label"`
	Works        int     `json:"works"`
	ShareCovered float64 `json:"shareCovered"`
	ShareTotal   float64 `json:"shareTotal"`
}

CategoryTally is one category's representation. Works counts each work once, however many of its subjects matched. ShareCovered is Works/CoveredWorks (the share among works that could be categorized at all); ShareTotal is Works/TotalWorks (the share of the whole corpus).

type Crosswalk

type Crosswalk struct {
	// contains filtered or unexported fields
}

Crosswalk categorizes subjects. It is built once (from the seed plus any operator overrides) and is read-only thereafter, so it is safe for concurrent use.

func Default

func Default() *Crosswalk

Default returns the crosswalk built from the embedded seed alone. A malformed seed is a programming error (the embedded file is guarded by a golden test), so it panics rather than returning a silently empty categorizer.

func Load

func Load(overridePaths ...string) (*Crosswalk, error)

Load returns the crosswalk built from the embedded seed with each override file merged over it in order. Merge is by category id: an override category unions its keywords and uris onto the matching seed category and, when it sets a non-empty label, replaces the label; an override category with a new id is appended. A missing or malformed override file is an error -- an operator who points libcat at a crosswalk expects it to be honored, not silently ignored.

func (*Crosswalk) Categories

func (c *Crosswalk) Categories() []Category

Categories returns the categories in stable reporting order (seed order, then any operator-added categories). Keyword/URI detail is omitted; this is for labeling aggregated counts.

func (*Crosswalk) Categorize

func (c *Crosswalk) Categorize(uri, label string) []string

Categorize returns the ids of every category a single subject maps to, in stable reporting order. A subject matches a category by exact authority URI (when uri is non-empty) or by a whole-word/phrase match of any of the category's keywords against the heading label. Passing "" for either argument skips that dimension.

func (*Crosswalk) CategorizeSubjects

func (c *Crosswalk) CategorizeSubjects(subjects []struct{ URI, Label string }) []string

CategorizeSubjects returns the ids of every category any of the given subjects maps to, deduplicated and in stable reporting order -- the work-level roll-up a caller (tasks/366) uses to count a work once per matched category. Each subject is a (uri, label) pair; either may be empty.

func (*Crosswalk) Label

func (c *Crosswalk) Label(id string) string

Label returns a category's display label, or the id itself if unknown.

type Report

type Report struct {
	// TotalWorks is every work the audit saw.
	TotalWorks int `json:"totalWorks"`
	// CoveredWorks is the works carrying at least one subject (URI or label) --
	// the only works that can be categorized, and the honest denominator.
	CoveredWorks int `json:"coveredWorks"`
	// Coverage is CoveredWorks/TotalWorks in [0,1]. A low value means the audit
	// speaks for only part of the collection; read every category share with it.
	Coverage float64 `json:"coverage"`
	// Categories are the diversity categories in the crosswalk's reporting order.
	Categories []CategoryTally `json:"categories"`
}

Report is a coverage-first content-diversity audit of a corpus. It reports what the works are *about*, never who created them. Every category share is stated against an explicit denominator so undercounting is visible rather than hidden: CoveredWorks (works that carry any subject at all) is the honest base for representation, while ShareTotal shows the dilution across the whole corpus, including works no one has subjected yet.

type SubjectRef

type SubjectRef struct {
	URI    string
	Labels []string
}

SubjectRef is one of a work's subjects as the audit sees it: an authority URI (may be empty for a bare-string ILS heading) and its heading labels (may be empty for a URI-only reference). Either dimension feeds the crosswalk.