package module
Version: v0.0.0-...-887c23e Latest Latest

This package is not in the latest version of its module.

Go to latest
Published: Nov 19, 2019 License: MIT Imports: 16 Imported by: 0



GoDoc Reference Build Status Go Report Card Bugs Coverage Lines of Code Maintainability Rating Reliability Rating Security Rating Vulnerabilities


Package wikibrief is a golang package that provides utility functions for extracting information from Wikipedia dumps.


This package can be installed with the go get command:

go get


API documentation can be found in the associated godoc reference




View Source
const AnonimousUserID uint32 = 0

AnonimousUserID is the UserID value assumed by revisions done by an anonimous user


This section is empty.


func FanOut

func FanOut(ctx context.Context, in <-chan EvolvingPage, n int) (out []<-chan EvolvingPage)

FanOut creates n copies of the given EvolvingPage channel; if n < 1 panics.

func New

func New(ctx context.Context, fail func(err error) error, tmpDir, lang string, restrict bool) <-chan EvolvingPage

New digest the latest wikipedia dump of the specified language into the output channel. The revision channel of each page must be exhausted (or the context cancelled), doing otherwise may result in a deadlock. The ctx and fail together should behave in the same manner as if created with WithFail - The condition restrict restricts the digest to just one dump file, used for testing purposes.


type EvolvingPage

type EvolvingPage struct {
	PageID          uint32
	Title, Abstract string
	TopicID         uint32
	Revisions       <-chan Revision

EvolvingPage represents a wikipedia page that is being edited. Revisions is closed when there are no more revisions. Revision channel must be exhausted (or the context cancelled), doing otherwise may result in a deadlock.

type Revision

type Revision struct {
	ID, UserID uint32
	IsBot      bool
	Text, SHA1 string
	IsRevert   uint32
	Timestamp  time.Time

Revision represents a revision of a page.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL