wikipedia

Documentation

Index

• func FetchGetGeneralTerm(term string) (results []Page, related []Page, lang string, actualTitle string)
• func IsDateBeforeUTCToday(requestedDate time.Time) (isBefore bool)
• func ParseLanguageFromText(text string) (lang string, remainingText string)
• func ParseTimeString(datestring string) (parsed time.Time)
• type ActionAPIBaseResponsePageInfo
• type ActionAPIGeneratorResponse
• type AnalyticsPageviews
• type MultiplePageResponseREST
• type Page
  • func FetchRelated(term string) (resp []Page, lang string, actualTerm string)
  • func FetchSearch(searchString string) (resp []Page, lang string, actualSearchString string)
  • func FetchSummary(title string) (resp []Page, lang string, actualTitle string)
• type PageResponseREST
• type PagelistPage
  • func FetchTopPageviews(datestring string, lang string) (resp []PagelistPage)

Functions

func FetchGetGeneralTerm

func FetchGetGeneralTerm(term string) (results []Page, related []Page, lang string, actualTitle string)

FetchGetGeneralTerm is a general method of fetching a term from Wikipedia, displaying related terms, and showing a general search result if a page isn't found. This is mostly meant to make sure that a user expects -- and gets -- sensible results from Wikipedia regardless of the "quirks" of the API mechanisms like expecting exact character chase or exact match for titles, and also suggesting related information rather than leaving the user with no result at all.

= Fallback mechanism The process performs the following with the given term - Always: Fetch the summary of the <term>

• If summary found:
• Fetch related articles for the <term>
• If summary not found:
• Fetch search results for the <term>
• If first result (by relevance) is equal (regardless of case) to the requested term, continue as if summary was found.
• Otherwise, return search results

= Return value The method returns a list of results, and a list of 'sub' results (related pages) so the consumer can display those differently if they wish.

func IsDateBeforeUTCToday

func IsDateBeforeUTCToday(requestedDate time.Time) (isBefore bool)

IsDateBeforeUTCToday checks whether the given date is before the official "today" date of UTC.

This is meant to see if there needs to be a conversion (going 'back' a day) when the user from a timezone that is ahead of UTC requests data for a certain date. For example, a user in San Francisco asking for results for June 5th, The UTC date may still be June 4th, which means the results from the remote API request (specifically analytics clusters, but others may as well) be unavailable. If that is the case, this gives the consumer a chance to change the date to a day before or alert the user that they should change their requested date themselves.

func ParseLanguageFromText

func ParseLanguageFromText(text string) (lang string, remainingText string)

ParseLanguageFromText looks for the lang=xx expression and outputs the language, or defaults to 'en' if language wasn't found.

func ParseTimeString

func ParseTimeString(datestring string) (parsed time.Time)

ParseTimeString normalizes and then parses the given string into a time object

Types

type ActionAPIBaseResponsePageInfo

type ActionAPIBaseResponsePageInfo struct {
	Pageid    int    `json:"pageid"`
	Ns        int    `json:"ns"`
	Title     string `json:"title"`
	Index     int    `json:"index"`
	Extract   string `json:"extract"`
	Thumbnail struct {
		Source string `json:"source"`
		Width  int    `json:"width"`
		Height int    `json:"height"`
	} `json:"thumbnail"`
	Pageimage            string    `json:"pageimage"`
	Contentmodel         string    `json:"contentmodel"`
	Pagelanguage         string    `json:"pagelanguage"`
	Pagelanguagehtmlcode string    `json:"pagelanguagehtmlcode"`
	Pagelanguagedir      string    `json:"pagelanguagedir"`
	Touched              time.Time `json:"touched"`
	Lastrevid            int       `json:"lastrevid"`
	Length               int       `json:"length"`
	Fullurl              string    `json:"fullurl"`
	Editurl              string    `json:"editurl"`
	Canonicalurl         string    `json:"canonicalurl"`
}

ActionAPIBaseResponsePageInfo is the structure of the individual pages in the ActionAPIGeneratorResponse. It is declared separately so that a map can be used, since the API returns an object with dynamic keys per page information.

type ActionAPIGeneratorResponse

type ActionAPIGeneratorResponse struct {
	Batchcomplete string `json:"batchcomplete"`
	Continue      struct {
		Gsroffset int    `json:"gsroffset"`
		Continue  string `json:"continue"`
	} `json:"continue"`
	Query struct {
		Pages map[string]ActionAPIBaseResponsePageInfo `json:"pages"`
	} `json:"query"`
}

ActionAPIGeneratorResponse is the structure expected from the Wikipedia action API using a generator

type AnalyticsPageviews

type AnalyticsPageviews struct {
	Title  string `json:"title"`
	Detail string `json:"detail"`
	Items  []struct {
		Project  string `json:"project"`
		Access   string `json:"access"`
		Year     string `json:"year"`
		Month    string `json:"month"`
		Day      string `json:"day"`
		Articles []struct {
			Article string `json:"article"`
			Views   int    `json:"views"`
			Rank    int    `json:"rank"`
		} `json:"articles"`
	} `json:"items"`
}

AnalyticsPageviews is the structure that is expected from the Wikipedia analytics API, like for pageviews requests

type MultiplePageResponseREST

type MultiplePageResponseREST struct {
	Pages []PageResponseREST `json:"pages"`
}

MultiplePageResponseREST is the wrapper around the response for multiple pages coming from the Wikipedia REST api

type Page

type Page struct {
	Title   string
	Extract string
	Image   string
	URL     string
	Rank    int
}

Page is a normalized structure for representing page data

func FetchRelated

func FetchRelated(term string) (resp []Page, lang string, actualTerm string)

FetchRelated fetches the related pages for the given term

func FetchSearch

func FetchSearch(searchString string) (resp []Page, lang string, actualSearchString string)

FetchSearch fetches search results from Wikipedia given the search string

func FetchSummary

func FetchSummary(title string) (resp []Page, lang string, actualTitle string)

FetchSummary fetches the summary of a specific Wikipedia page given by its title

type PageResponseREST

type PageResponseREST struct {
	Type         string `json:"type"`
	Title        string `json:"title"`
	Displaytitle string `json:"displaytitle"`
	Namespace    struct {
		ID   int    `json:"id"`
		Text string `json:"text"`
	} `json:"namespace"`
	WikibaseItem string `json:"wikibase_item"`
	Titles       struct {
		Canonical  string `json:"canonical"`
		Normalized string `json:"normalized"`
		Display    string `json:"display"`
	} `json:"titles"`
	Pageid    int `json:"pageid"`
	Thumbnail struct {
		Source string `json:"source"`
		Width  int    `json:"width"`
		Height int    `json:"height"`
	} `json:"thumbnail"`
	Originalimage struct {
		Source string `json:"source"`
		Width  int    `json:"width"`
		Height int    `json:"height"`
	} `json:"originalimage"`
	Lang              string `json:"lang"`
	Dir               string `json:"dir"`
	Revision          string `json:"revision"`
	Tid               string `json:"tid"`
	Timestamp         string `json:"timestamp"`
	Description       string `json:"description"`
	DescriptionSource string `json:"description_source"`
	Coordinates       struct {
		Lat float64 `json:"lat"`
		Lon float64 `json:"lon"`
	} `json:"coordinates"`
	ContentUrls struct {
		Desktop struct {
			Page      string `json:"page"`
			Revisions string `json:"revisions"`
			Edit      string `json:"edit"`
			Talk      string `json:"talk"`
		} `json:"desktop"`
		Mobile struct {
			Page      string `json:"page"`
			Revisions string `json:"revisions"`
			Edit      string `json:"edit"`
			Talk      string `json:"talk"`
		} `json:"mobile"`
	} `json:"content_urls"`
	APIUrls struct {
		Summary      string `json:"summary"`
		Metadata     string `json:"metadata"`
		References   string `json:"references"`
		Media        string `json:"media"`
		EditHTML     string `json:"edit_html"`
		TalkPageHTML string `json:"talk_page_html"`
	} `json:"api_urls"`
	Extract     string `json:"extract"`
	ExtractHTML string `json:"extract_html"`
}

PageResponseREST is the response expected from the Wikipedia REST API for page results. If the response is a single page, this is the structure expected from the API. If the response contains multiple pages, it will be wrapped with an object that is represented by the MultiplePageResponseREST struct

type PagelistPage

type PagelistPage struct {
	Title string
	URL   string
	Rank  int
	Info  string
}

PagelistPage represent normalized structure for an information for a page in a list

func FetchTopPageviews

func FetchTopPageviews(datestring string, lang string) (resp []PagelistPage)

FetchTopPageviews fetches the top articles by pageview for a given date. Lang parameter will dictate the Wikipedia that will be searched. If given empty string, will fall back on "en"