dataapi

package module
v0.0.0-...-ee184b8 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Apr 20, 2022 License: MIT Imports: 14 Imported by: 0

README

RRCHNM Data API

This repository provides an API to access data stored in a PostgreSQL database. It is a component of American Religious Ecologies, America's Public Bible, Death by Numbers and other projects at the Roy Rosenzweig Center for History and New Media.

Endpoints

The following endpoints are offered by the API.

AHCB counties and states

Spatial polygons from the Atlas of Historic County Boundaries are available by date. The results will always be filtered by the date provided. Use the id, state-terr-id, or state-code to filter geographically.

GET /ahcb/states/:date/
GET /ahcb/counties/:date/
GET /ahcb/counties/:date/id/:id/
GET /ahcb/counties/:date/state-terr-id/:state_terr_id/
GET /ahcb/counties/:date/state-code/:state_code/

Parameters:

  • date: The date of the historic boundaries, specified as an ISO-8601 string (e.g., 1848-07-05). If the date requested is before or after the minimum or after the maximum dates for that type of geometry, the minimum or maximum will be silently returned.
  • id: A comma-separated list of AHCB IDs for counties (e.g., vas_fairfax).
  • state_terr_id: A comma-separated list of AHCB IDs for states and territories (e.g., va_state).
  • state_code: A comma-separated list two-letter codes for states, roughly corresponding to postal codes (.e.g, va).

Response:

A GeoJSON feature collection in EPSG 4326 with one feature per state or county. The properties of each feature will include information such as the square mileage of the polygons.

Catholic dioceses

Historical spatial point locations for Catholic dioceses in the United States, Canada, and Mexico.

GET /catholic-dioceses/

Parameters:

  • None

Response:

A JSON array of objects, each object representing a Catholic diocese. The date that the diocese was created (date_erected) is an ISO-8601 string, and the date that the diocese became an archdiocese (date_metropolitan) is either an ISO-8601 string or an empty string if the diocese did not become an archdiocese.

Catholic dioceses per decade

Counts of the number of dioceses established per decade

GET /catholic-dioceses/per-decade/

Parameters:

  • None

Response:

A JSON array of objects, each object representing the count of dioceses per decade.

North America

Country polygons from Natural Earth for North America.

GET /ne/northamerica/

Parameters:

  • None

Response:

A GeoJSON feature collection in EPSG 4326 with one feature per country.

Presbyterians

Presbyterian membership data per year for 1826-1926.

GET /presbyterians/

Parameters:

  • None

Response:

A JSON array of objects, each object representing a year of membership data.

Populated places

Lists of populated places from the USGS Geographic Names Information System in a state or county.

GET /pop-places/state/:state/county/
GET /pop-places/state/:state/place/
GET /pop-places/county/:county/place/
GET /pop-places/place/:place_id/

Parameters:

  • state: The state as a two-character abbreviation (e.g., CA).
  • county: An ACHB county ID (e.g., vas_fairfax).
  • place_id: The ID for a place (e.g., 611119).

Response:

A JSON array of objects where the objrects are all the counties in a state, or all the places in a state or county. The objects have both the unique identifiers and human-readable names.

For example, you should be able to query /pop-places/state/va/county to get all the counties in Virginia, and then query /pop-places/county/vas_fairfax/place to get all the places in Fairfax county. If you had the ID for a place like Centreville, you could look up its details by querying /pop-places/place/1491083/.

Endpoints
GET /

Parameters:

  • None

Response:

A JSON array of objects. Each object is an endpoint for the API, with a sample URL for that endpoint.

Census of Religious Bodies Denomination Families
GET /relcensus/denomination-families

Parameters:

  • None

Response:

A JSON object containing keys for different ways of classifying denominations. (Only the family_relec is implemented.) Each sub-object contains an array of objects describing the denomination families.

Census of Religious Bodies Denominations
GET /relcensus/denominations?family_relec=:family

Parameters:

  • family_relec: An optional query parameter to return just denominations that are part of a particular family.

Response:

A JSON array containing objects describing the denominations. If the family_relec query parameter is absent, then all of the denominations are returned.

Census of Religious Bodies Denomination Data for Cities
GET /relcensus/city-membership?year=:year&denomination=:denomination

Parameters:

  • year: An mandatory query parameter for the year of the census.
  • denomination: A mandatory query parameter with the specific denomination requested.

Response:

A JSON array containing objects describing the membership data for a denomination in a specific city in a specific year.

Census of Religious Bodies Total Data for Cities
GET /relcensus/city-total-membership?year=:year

Parameters:

  • year: An mandatory query parameter for the year of the census.

Response:

A JSON array containing objects describing the aggregate membership data for all denominations in a specific city in a specific year.

Bills of Mortality Parish Data

A unique list of London parishes.

GET /bom/parishes/

Parameters:

  • None

Response:

A JSON array of objects, each object representing a parish.

Bills of Mortality Data
GET /bom/bills?year=:year

Parameters:

  • year: An optional query parameter for the year or year range of the bills.
  • count_type: An optional query parameter with with a specific count type requested.
  • parishes: An optional query parameter with a specific parish or set of parishes requested.

Response:

A JSON array containing objects describing the bills data for a given year, count (burial vs. plague), or specific parishes.

Compiling or running a container

Using a version of Go that supports Go modules, you should be able to run go build in the project root to install dependencies.

There is a Makefile in cmd/dataapi/ that can be used for compiling and for running the service locally.

If you just need to run the Data API locally, it may be most convenient to just run a Docker container served from the GitHub Container Registry. There are versions that are tagged with each of the GitHub branches that have been pushed, so that you can try the development version. You still need to set the environment variables, as below. It may be most convenient to run the Docker container with the Makefile in the root of this repository.

Configuration

Set the following environment variables to configure the server:

  • DATAAPI_DBHOST (default: localhost)
  • DATAAPI_DBPORT (default: 5432)
  • DATAAPI_DBNAME (default: none)
  • DATAAPI_DBUSER (default: none)
  • DATAAPI_DBPASS (default: none)
  • DATAAPI_SSL (default: require; see pq docs)
  • DATAAPI_INTERFACE (default: 0.0.0.0)
  • DATAAPI_PORT (default: 8090)
  • DATAAPI_LOGGING (default: on)

If logging is on, then access logs will be written to stdout in the Apache Common Log format. Errors and status messages will always be written to stderr.

Obviously this service requires that you be able to access the database.

Testing

You can run the tests with go test -v inside the directory that contains the package for the command: cmd/dataapi.

Documentation

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

This section is empty.

Types

type APBIndexItem 

type APBIndexItem struct {
	Reference string `json:"reference"`
	Text      string `json:"text"`
	Count     int    `json:"count"`
}

APBIndexItem is an entry in one of the different indexes to verses

type APBIndexItemText 

type APBIndexItemText struct {
	Reference string `json:"reference"`
	Text      string `json:"text"`
}

APBIndexItemText is an entry in one of the different indexes to verses, with the reference and the text of the verse.

type APBIndexItemWithYear 

type APBIndexItemWithYear struct {
	Reference string `json:"reference"`
	Text      string `json:"text"`
	Count     int    `json:"count"`
	Peak      int    `json:"peak"`
}

APBIndexItemWithYear is an index item with the peak year

type BibleBook 

type BibleBook struct {
	Book  string `json:"book"`
	Part  string `json:"part"`
	Order int    `json:"order"`
}

BibleBook describes a book of the Bible and which part of the Bible it is in.

type BibleSimilarityEdge 

type BibleSimilarityEdge struct {
	A string `json:"source"`
	B string `json:"target"`
	N int    `json:"n"`
}

BibleSimilarityEdge describes an edge between two parts of the Bible

type CatholicDiocese 

type CatholicDiocese struct {
	City             string    `json:"city"`
	State            string    `json:"state"`
	Country          string    `json:"country"`
	Rite             string    `json:"rite"`
	YearErected      int64     `json:"year_erected"`
	YearMetropolitan NullInt64 `json:"year_metropolitan"`
	YearDestroyed    NullInt64 `json:"year_destroyed"`
	Lon              float32   `json:"lon"`
	Lat              float32   `json:"lat"`
}

CatholicDiocese describes a diocese of the Roman Catholic Church.

type CatholicDiocesesPerDecade 

type CatholicDiocesesPerDecade struct {
	Decade int64 `json:"decade"`
	Count  int64 `json:"n"`
}

CatholicDiocesesPerDecade shows how many dioceses were established in North America per year.

type ChristeningsByYear 

type ChristeningsByYear struct {
	ChristeningsDesc string    `json:"christenings_desc"`
	TotalCount       NullInt64 `json:"count"`
	WeekNo           int       `json:"week_no"`
	WeekID           string    `json:"week_id"`
	Year             int       `json:"year"`
}

ChristeningsByYear describes a christen's description and various metadata for a given christening.

type CityMembership 

type CityMembership struct {
	Year           int       `json:"year"`
	Group          string    `json:"group"`
	City           string    `json:"city"`
	State          string    `json:"state"`
	Denominations  int       `json:"denominations"`
	Churches       int       `json:"churches"`
	Members        int       `json:"members"`
	Population1926 NullInt64 `json:"population_1926"`
	Lon            float64   `json:"lon"`
	Lat            float64   `json:"lat"`
}

CityMembership gives the membership (and population) statistics for some aggregation of denominations in a given year.

type Config 

type Config struct {
	// contains filtered or unexported fields
}

The Config type stores configuration which is read from environment variables.

type Denomination 

type Denomination struct {
	Name           string     `json:"name"`
	ShortName      string     `json:"short_name"`
	DenominationID NullString `json:"denomination_id"`
	FamilyCensus   NullString `json:"family_census"`
	FamilyRelec    string     `json:"family_relec"`
}

Denomination describes a denomination's names and various systems of classification.

type DenominationFamily 

type DenominationFamily struct {
	Name string `json:"name"`
}

DenominationFamily describes a group of denominations. There can be different ways of categorizing denominations.

type Endpoint 

type Endpoint struct {
	Name     string       `json:"name"`
	URL      string       `json:"path"`
	Examples []ExampleURL `json:"examples,omitempty"`
}

Endpoint describes an endpoint available in this API and provides a sample path.

type ExampleURL 

type ExampleURL struct {
	URL     string `json:"url"`
	Purpose string `json:"purpose"`
}

ExampleURL provides an example URL to a different way of querying the API for any given endpoint.

type GeneralBillsByYear 

type GeneralBillsByYear struct {
	ParishName string    `json:"name"`
	CountType  string    `json:"count_type"`
	TotalCount NullInt64 `json:"count"`
	Year       int       `json:"year"`
	WeekNo     int       `json:"week_no"`
	WeekID     string    `json:"week_id"`
}

GeneralBillsByYear describes a parish's names and various metadata for a given parish.

type NullInt64 

type NullInt64 struct {
	sql.NullInt64
}

NullInt64 embeds the sql.NullInt64 type, so that it can be extended to change the JSON marshaling.

func (NullInt64) MarshalJSON 

func (v NullInt64) MarshalJSON() ([]byte, error)

MarshalJSON marshals a null integer as `{"int": null}` rather than using an object inside the field. See https://stackoverflow.com/questions/33072172/how-can-i-work-with-sql-null-values-and-json-in-a-good-way

func (*NullInt64) UnmarshalJSON 

func (v *NullInt64) UnmarshalJSON(data []byte) error

UnmarshalJSON unmarshals a null integer represented in JSON as `{"int": null}` into our embedded type that allows nulls. See https://stackoverflow.com/questions/33072172/how-can-i-work-with-sql-null-values-and-json-in-a-good-way

type NullString 

type NullString struct {
	sql.NullString
}

NullString embeds the sql.NullString type, so that it can be extended to change the JSON marshaling.

func (NullString) MarshalJSON 

func (v NullString) MarshalJSON() ([]byte, error)

MarshalJSON marshals a null string as `{"string": null}` rather than using an object inside the field. See https://stackoverflow.com/questions/33072172/how-can-i-work-with-sql-null-values-and-json-in-a-good-way

func (*NullString) UnmarshalJSON 

func (v *NullString) UnmarshalJSON(data []byte) error

UnmarshalJSON unmarshals a null string represented in JSON as `{"string": null}` into our embedded type that allows nulls. See https://stackoverflow.com/questions/33072172/how-can-i-work-with-sql-null-values-and-json-in-a-good-way

type Parish 

type Parish struct {
	ParishID      int    `json:"id"`
	Name          string `json:"name"`
	CanonicalName string `json:"canonical_name"`
}

Parish describes a denomination's names and various systems of classification.

type ParishByYear 

type ParishByYear struct {
	ParishName string    `json:"name"`
	CountType  string    `json:"count_type"`
	TotalCount NullInt64 `json:"count"`
	StartDay   string    `json:"start_day"`
	StartMonth string    `json:"start_month"`
	EndDay     string    `json:"end_day"`
	EndMonth   string    `json:"end_month"`
	Year       int       `json:"year"`
	WeekNo     int       `json:"week_no"`
	WeekID     string    `json:"week_id"`
}

ParishByYear describes a parish's names and various metadata for a given parish.

type Place 

type Place struct {
	PlaceID int    `json:"place_id"`
	Place   string `json:"place"`
	MapName string `json:"map_name"`
}

Place represents a place with ID and name

type PlaceCounty 

type PlaceCounty struct {
	CountyAHCB string `json:"county_ahcb"`
	County     string `json:"name"`
}

PlaceCounty represents a county with ID and name

type PlaceDetails 

type PlaceDetails struct {
	PlaceID    int    `json:"place_id"`
	Place      string `json:"place"`
	MapName    string `json:"map_name"`
	County     string `json:"county"`
	CountyAHCB string `json:"county_ahcb"`
	State      string `json:"state"`
}

PlaceDetails represents details about a place

type PresbyteriansByYear 

type PresbyteriansByYear struct {
	Year     int `json:"year"`
	Members  int `json:"members"`
	Churches int `json:"churches"`
}

PresbyteriansByYear holds aggregate data on Presbyterian membership and churches.

type Server 

type Server struct {
	Server     *http.Server
	Database   *sql.DB
	Router     *mux.Router
	Config     Config
	Statements map[string]*sql.Stmt
}

The Server type shares access to the database.

func NewServer 

func NewServer() *Server

NewServer creates a new Server and connects to the database or fails trying.

func (*Server) AHCBCountiesByIDHandler 

func (s *Server) AHCBCountiesByIDHandler() http.HandlerFunc

AHCBCountiesByIDHandler returns a GeoJSON FeatureCollection containing counties from AHCB. The handler will get the county boundaries for a particular date and by county ID (or IDs if given a comma-separated string of values).

func (*Server) AHCBCountiesByStateCodeHandler 

func (s *Server) AHCBCountiesByStateCodeHandler() http.HandlerFunc

AHCBCountiesByStateCodeHandler returns a GeoJSON FeatureCollection containing counties from AHCB. The handler will get the county boundaries for a particular date and by state code (or state codes if given a comma-separated string of values).

func (*Server) AHCBCountiesByStateTerrIDHandler 

func (s *Server) AHCBCountiesByStateTerrIDHandler() http.HandlerFunc

AHCBCountiesByStateTerrIDHandler returns a GeoJSON FeatureCollection containing counties from AHCB. The handler will get the county boundaries for a particular date and by state/territory ID (or IDs if given a comma-separated string of values).

func (*Server) AHCBCountiesHandler 

func (s *Server) AHCBCountiesHandler() http.HandlerFunc

AHCBCountiesHandler returns a GeoJSON FeatureCollection containing counties from AHCB. The handler will get the county boundaries for a particular date.

func (*Server) AHCBStatesHandler 

func (s *Server) AHCBStatesHandler() http.HandlerFunc

AHCBStatesHandler returns a GeoJSON FeatureCollection containing states from AHCB. The handler will get the county boundaries for a particular date.

func (*Server) APBIndexAllHandler 

func (s *Server) APBIndexAllHandler() http.HandlerFunc

APBIndexAllHandler returns basically all available verses in their biblical order.

func (*Server) APBIndexBiblicalOrderHandler 

func (s *Server) APBIndexBiblicalOrderHandler() http.HandlerFunc

APBIndexBiblicalOrderHandler returns verses in their biblical order.

func (*Server) APBIndexChronologicalHandler 

func (s *Server) APBIndexChronologicalHandler() http.HandlerFunc

APBIndexChronologicalHandler returns verses in chronological order by their peak.

func (*Server) APBIndexFeaturedHandler 

func (s *Server) APBIndexFeaturedHandler() http.HandlerFunc

APBIndexFeaturedHandler returns featured verses.

func (*Server) APBIndexTopHandler 

func (s *Server) APBIndexTopHandler() http.HandlerFunc

APBIndexTopHandler returns top verses.

func (*Server) BibleBooksHandler 

func (s *Server) BibleBooksHandler() http.HandlerFunc

BibleBooksHandler returns the books of the Bible (in the KJV).

func (*Server) BibleSimilarityHandler 

func (s *Server) BibleSimilarityHandler() http.HandlerFunc

BibleSimilarityHandler returns the information about the network of similarities within the Bible.

func (*Server) BibleTrendHandler 

func (s *Server) BibleTrendHandler() http.HandlerFunc

BibleTrendHandler returns the rates of quotation per year for a verse

func (*Server) BillsHandler 

func (s *Server) BillsHandler() http.HandlerFunc

BillsHandler returns ...

func (*Server) CatholicDiocesesHandler 

func (s *Server) CatholicDiocesesHandler() http.HandlerFunc

CatholicDiocesesHandler returns a JSON array of Catholic dioceses. Though the spatial data is stored in the database as a geometry, it is returned as simple lon/lat coordinates because that is easiest to process in the visualizations.

func (*Server) CatholicDiocesesPerDecadeHandler 

func (s *Server) CatholicDiocesesPerDecadeHandler() http.HandlerFunc

CatholicDiocesesPerDecadeHandler returns a JSON array of the number of dioceses established in North America per year.

func (*Server) ChristeningsHandler 

func (s *Server) ChristeningsHandler() http.HandlerFunc

ChristeningsHandler returns ...

func (*Server) CityMembershipHandler 

func (s *Server) CityMembershipHandler() http.HandlerFunc

CityMembershipHandler returns the statistics for all the cities for a single denomination in a single year. It must be filtered by year and denomination.

func (*Server) CountiesInState 

func (s *Server) CountiesInState() http.HandlerFunc

CountiesInState returns a list of all the counties in a state, with IDs from AHCB.

func (*Server) DenominationFamiliesHandler 

func (s *Server) DenominationFamiliesHandler() http.HandlerFunc

DenominationFamiliesHandler returns

func (*Server) DenominationsHandler 

func (s *Server) DenominationsHandler() http.HandlerFunc

DenominationsHandler returns the denominations that are available. Optionally, it can be filtered to get just the denominations in a particular family.

func (*Server) EndpointsHandler 

func (s *Server) EndpointsHandler() http.HandlerFunc

EndpointsHandler describes the endpoints that are available in this API, with sample URLs to show how the API works.

func (*Server) GeneralBillsHandler 

func (s *Server) GeneralBillsHandler() http.HandlerFunc

GeneralBillsHandler returns ...

func (*Server) Middleware 

func (s *Server) Middleware()

Middleware registers the middleware functions that should be used.

func (*Server) NENorthAmericaHandler 

func (s *Server) NENorthAmericaHandler() http.HandlerFunc

NENorthAmericaHandler returns a GeoJSON FeatureCollection containing country polygons for North America from Natural Earth.

func (*Server) NotFoundHandler 

func (s *Server) NotFoundHandler() http.HandlerFunc

NotFoundHandler returns 404 errors

func (*Server) ParishesHandler 

func (s *Server) ParishesHandler() http.HandlerFunc

ParishesHandler returns

func (*Server) Place 

func (s *Server) Place() http.HandlerFunc

Place returns the details about a populated place

func (*Server) PlacesInCounty 

func (s *Server) PlacesInCounty() http.HandlerFunc

PlacesInCounty returns a list of all the populated places in a county.

func (*Server) PresbyteriansHandler 

func (s *Server) PresbyteriansHandler() http.HandlerFunc

PresbyteriansHandler returns the aggregate data on Presbyterian memberhsip and churches.

func (*Server) Routes 

func (s *Server) Routes()

Routes registers the handlers for the URLs that should be served.

func (*Server) Run 

func (s *Server) Run()

Run starts the API server.

func (*Server) Shutdown 

func (s *Server) Shutdown()

Shutdown closes the connection to the database and shutsdown the server.

func (*Server) VerseHandler 

func (s *Server) VerseHandler() http.HandlerFunc

VerseHandler returns information about a verse, and other verses which are related to it, if any.

func (*Server) VerseQuotationsHandler 

func (s *Server) VerseQuotationsHandler() http.HandlerFunc

VerseQuotationsHandler returns the instances of quotations for a verse

func (*Server) VerseTrendHandler 

func (s *Server) VerseTrendHandler() http.HandlerFunc

VerseTrendHandler returns the rates of quotation per year for a verse

type Verse 

type Verse struct {
	Reference string   `json:"reference"`
	Text      string   `json:"text"`
	Related   []string `json:"related"`
}

Verse describes the reference and text of a single Bible verse

type VerseQuotation 

type VerseQuotation struct {
	Reference   string  `json:"reference"`
	DocID       string  `json:"docID"`
	Date        string  `json:"date"`
	Probability float32 `json:"probability"`
	Title       string  `json:"title"`
	State       string  `json:"state"`
}

VerseQuotation is a single instance of a quotation

type VerseTrend 

type VerseTrend struct {
	Year                int     `json:"year"`
	N                   int     `json:"n"`
	QuotationRateSmooth float64 `json:"smoothed"`
}

VerseTrend is the rate of quotations in a single year for a single verse in a given corpus. The quotation rate is expressed in quotations per million words; the smoothed rate has the same units, and is a centered three-year rolling average.

type VerseTrendResponse 

type VerseTrendResponse struct {
	Reference string       `json:"reference"`
	Corpus    string       `json:"corpus"`
	Trend     []VerseTrend `json:"trend"`
}

VerseTrendResponse wraps the data with the queries that were made

Source Files

View all Source files

Directories

Path Synopsis
cmd
dataapi

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.