csvmap

Documentation

Overview

Package csvmap is the generic CSV ingest provider: it reads a spreadsheet-shaped export -- the lowest-common-denominator dump every ILS and collection tool can produce -- into ingest records driven by a declarative TOML mapping of columns to fields, so a deployment sideloads a CSV with a profile, not code. Works sharing ISBNs (or a stable id column) with another feed merge in the shared clustering pipeline.

Index

• Constants
• func New(cfg ingest.Config) (ingest.Provider, error)
• type Mapping
  • func LoadMapping(path string) (*Mapping, error)
• type Provider
  • func (p *Provider) Name() string
  • func (p *Provider) Records(ctx context.Context) ([]ingest.Record, error)
  • func (p *Provider) Role() ingest.Role

Constants

const ProviderName = "csv"

ProviderName is the registry key and default provenance feed (feed:csv).

Functions

func New

func New(cfg ingest.Config) (ingest.Provider, error)

New builds the provider from an ingest.Config: Source is the .csv path and Params["mapping"] the mapping TOML path; Feed overrides the provenance feed name.

Types

type Mapping

type Mapping struct {
	// IDScheme namespaces the durable provider id ("<scheme>:<rowid>") minted
	// when an id column is mapped. Defaults to the feed name; keep it stable
	// across exports or every work re-mints.
	IDScheme string `toml:"id-scheme"`
	// Class is the BIBFRAME work class (default "Text").
	Class string `toml:"class"`
	// DefaultLanguage is the ISO 639-2/B code used when a row carries no
	// (mappable) language (default "eng").
	DefaultLanguage string `toml:"default-language"`
	// Delimiter is the field separator, e.g. "\t" for TSV (default ",").
	Delimiter string `toml:"delimiter"`
	// MultiSeparator splits multi-valued cells -- creators, isbns, subjects
	// (default ";").
	MultiSeparator string `toml:"multi-separator"`
	// Columns maps record fields to header names. Fields: id, title,
	// subtitle, summary, creator, isbn, subject, language. title is required;
	// subjects are uncontrolled labels (feed tags), not authority URIs.
	Columns map[string]string `toml:"columns"`
	// Extras maps work extra keys to header names -- adopter display fields
	// carried through to catalog.json's extra object (e.g. cover, rating).
	Extras map[string]string `toml:"extras"`
	// Languages maps the export's language codes to ISO 639-2/B. An unmapped
	// three-letter code passes through; anything else falls back to
	// DefaultLanguage.
	Languages map[string]string `toml:"languages"`
}

Mapping declares how a CSV export's columns map onto ingest records -- the whole provider config, written as a TOML file so sideloading a spreadsheet means editing a mapping, not writing Go.

func LoadMapping

func LoadMapping(path string) (*Mapping, error)

LoadMapping reads and validates a mapping TOML file.

type Provider

type Provider struct {
	// contains filtered or unexported fields
}

Provider reads a mapped CSV file into ingest records, one per row.

func (*Provider) Name

func (p *Provider) Name() string

Name is the provenance feed the run writes (feed:<name>).

func (*Provider) Records

func (p *Provider) Records(ctx context.Context) ([]ingest.Record, error)

Records reads every row, in file order (the file is the adopter's own export, so its order is already deterministic). Rows without a title are dropped with a warning tally, mirroring the nquads provider.

func (*Provider) Role

func (p *Provider) Role() ingest.Role

Role marks this an ingest-role provider.