wikibase

README

Simple library for talking to Wikibase in golang

This library has two functions:

• Provides a simple wrapper for common calls the to MediaWiki API for creating and protecting articles along with getting edit tokens.
• Provides a simple pseudo-ORM for working with Items and Properties on wikibase - you build your item as a tagged structure using an embedded header, and then you can sync that up to wikibase to write data there.

Currently this library is work in progress, with a bias on writing to wikibase rather than reading, as that's what has been required on the project this was developed for.

Basics

Currently this library assumes you have valid OAuth tokens for client and consumer. This will be resolved shortly.

For basic API usage there are a series of simple calls in wikibase.go. In general page IDs are used in preference of page titles, for consistency with items and property also referred to by IDs.

If you want to create items and properties, then you can create a structure with a ItemHeader embedded entry, which you can store the Item ID in, and then use the property annotation on all fields you want to be turned into a property (you can add additional fields to the structure, if they don't have a property tag then they will be safely ignored). The value of the property annotation should be the label of the property (not the P number, as that will change most likely between production and test servers, so labels are seen as useful abstractions for naming).

type ExampleWikibaseItem struct {
	wikibase.ItemHeader

    Name             string                 `property:"Name"`
    Birthday         time.Time              `property:"Date of birth"`
    NextOfKin        *wikibase.ItemProprety `property:"Next of kin,omitoncreate"`
    SkateboardsOwned int                    `property:"Skateboards owner"`
}

The library will manage some of the property formatting restrictions of Wikibase: Pointers with a nil value will be set as having no value in Wikibase, as will string properties with a zero length. Strings will automatically have whitespace formatting homogenised to keep Wikibase happy too.

The omitoncreate modified on the tag will tell the library not to attempt to set an initial value for that property when the item is being created. If you are uploading a set of items and then layer need to link them using ItemProperty fields then you may not wish to load them initially at create time and upload them later as a restricted subset (using the argument to the update call to say only add new items). Ideally this sort of thing wouldn't be necessary but the Wikibase API is relatively slow with even trivial amounts of data, so this lets you start to manage how much you actually do in each transaction.

Once you've defined your structure and created a client you first need to get the Client to loop up the actual P numbers of the properties using a call to MapPropertyAndItemConfiguration like so:

    client := wikibase.NewClient(...)
    err := client.MapPropertyAndItemConfiguration(ExampleWikibaseItem{}, true)

The boolean second argument tells the client to create property definitions if they don't already exist on the wikibase server.

If you want to fetch the Q numbers for specific items so you can store them in ItemProperty fields then you can call MapItemConfigurationByLabel, which also takes a second argument to say whether it should create the item if not found.

You can create a new Wikibase item as follows:

    person := ExampleWikibaseItem{Name: "Alice", Birthday: time.Now()}
    err := client.CreateItemInstance("person item", &person)

After this call, assuming successful, the person.ID field will be set to the Q number for the Item on Wikibase created, and the person.PropertyIDs field will map the P numbers of the properties to their GUID on wikibase. It is recommend you serialise these to JSON or some other format and restore them later if you wish to edit the same object on Wikibase across multiple invocations of your client.

You can similarly update the Wikibase Item you are modelling like so:

    person.SkateboardsOwned += 1
    err := client.UploadClaimsForItem(&person, true)

The boolean argument indicates if properties already uploaded should be updated or ignored. True here means updated, false would have been effectively a no-op. The API is like this as Wikibase API updates are relatively slow, and so having the fidelity to control how much up update can make for a much quicker client.

SPARQL Query Service

There is also limited support for using the query service if such a thing is running on the wikibase instance. You can provide a SPARQL query as a string, along with the URL for the REST API endpoint and call:

    res, err := MakeSPARQLQuery(URL_TO_API_ENDPOINT, SPARQL_QUERY)

The return type of SparqlResult is just a thing wrapper around the JSON SPARQL format, with results stored in a map of variable names as defined in the submitted query.

License

This library is copyright Content Mine Ltd 2018, and released under the Apache 2.0 License.

Dependencies

Relies on https://github.com/mrjones/oauth

Documentation

Overview

Package provides some mediawiki convenience functions along with a pseudo-ORM for creating wikibase items and their associated properties.

Index

• func StringClaimToAPIData(value string) (*string, error)
• type APIError
  • func (e *APIError) Error() string
• type AccessToken
• type Client
  • func NewClient(oauthClient NetworkClientInterface) *Client
  • func (c *Client) CreateClaimOnItem(item ItemPropertyType, property_id string, encoded_data []byte) (string, error)
  • func (c *Client) CreateItemInstance(label string, i interface{}) error
  • func (c *Client) CreateOrUpdateArticle(title string, body string) (int, error)
  • func (c *Client) FetchItemIDsForLabel(label string) ([]string, error)
  • func (c *Client) FetchPropertyIDsForLabel(label string) ([]string, error)
  • func (c *Client) GetEditingToken() (string, error)
  • func (c *Client) MapItemConfigurationByLabel(label string, create_if_not_there bool) error
  • func (c *Client) MapPropertyAndItemConfiguration(i interface{}, create_if_not_there bool) error
  • func (c *Client) ProtectPageByID(page_id int) error
  • func (c *Client) ProtectPageByTitle(title string) error
  • func (c *Client) UploadClaimsForItem(i interface{}, allow_refresh bool) error
• type ConsumerInformation
• type ItemClaim
  • func ItemClaimToAPIData(value ItemPropertyType) (ItemClaim, error)
• type ItemHeader
• type ItemPropertyType
• type NetworkClientInterface
• type OAuthInformation
  • func LoadOauthInformation(path string) (OAuthInformation, error)
• type OAuthNetworkClient
  • func NewOAuthNetworkClient(oauthInfo OAuthInformation, urlbase string) *OAuthNetworkClient
  • func (client *OAuthNetworkClient) Get(args map[string]string) (io.ReadCloser, error)
  • func (client *OAuthNetworkClient) Post(args map[string]string) (io.ReadCloser, error)
• type QuantityClaim
  • func QuantityClaimToAPIData(value int) (QuantityClaim, error)
• type SparqlHead
• type SparqlResponse
  • func MakeSPARQLQuery(service_url string, sparql string) (*SparqlResponse, error)
• type SparqlResult
• type SparqlResults
• type SparqlValue
• type TimeDataClaim
  • func TimeDataClaimToAPIData(value string) (TimeDataClaim, error)
• type WikiBaseType

Functions

func StringClaimToAPIData

func StringClaimToAPIData(value string) (*string, error)

Types

type APIError

type APIError struct {
	Code string `json:"code"`
	Info string `json:"info"`
}

Error as returned by MediaWiki API

func (*APIError) Error

func (e *APIError) Error() string

type AccessToken

type AccessToken struct {
	Token  string `json:"token"`
	Secret string `json:"secret"`
}

type Client

type Client struct {

	// Mapping of labels to IDs for Items and Properties.
	PropertyMap map[string]string
	ItemMap     map[string]ItemPropertyType
	// contains filtered or unexported fields
}

The Wikibase/media wiki client struct. Create this with a call to NewClient, passing it a valid network client.

func NewClient

func NewClient(oauthClient NetworkClientInterface) *Client

NewClient is a factory method for creating a new Client object.

func (*Client) CreateClaimOnItem

func (c *Client) CreateClaimOnItem(item ItemPropertyType, property_id string, encoded_data []byte) (string, error)

func (*Client) CreateItemInstance

func (c *Client) CreateItemInstance(label string, i interface{}) error

CreateItemInstance will take a pointer to a Go structure that has the embedded wikibase header and item and property tags on its fields and create a new item with the provided label. Any fields in the structure with a Property tag that does not contain the "omitoncreate" clause will also be created as item claims at the same time.

func (*Client) CreateOrUpdateArticle

func (c *Client) CreateOrUpdateArticle(title string, body string) (int, error)

CreateOrUpdateArticle will create a new mediawiki page if necessary, and set its content to the provided body text. The body should be in wikitext format, or if your Mediawiki instance supports it, parsoidHTML.

func (*Client) FetchItemIDsForLabel

func (c *Client) FetchItemIDsForLabel(label string) ([]string, error)

FetchPropertyIDsForLabel will find Wikibase items with the exact matching label and return them as a list of Q numbers.

func (*Client) FetchPropertyIDsForLabel

func (c *Client) FetchPropertyIDsForLabel(label string) ([]string, error)

FetchPropertyIDsForLabel will find Wikibase properties with the exact matching label and return them as a list of P numbers.

func (*Client) GetEditingToken

func (c *Client) GetEditingToken() (string, error)

GetEditingToken returns an already acquired editing token for this session, or fetches a new one if necessary. This method is thread safe.

func (*Client) MapItemConfigurationByLabel

func (c *Client) MapItemConfigurationByLabel(label string, create_if_not_there bool) error

MapItemConfigurationByLabel will attempt to find the item with the exact matching label on Wikibase and populate the Wikibase client structs internal map of labels to Item IDs. The client will use this when performing ORM like operations on structures to upload to Wikibase.

func (*Client) MapPropertyAndItemConfiguration

func (c *Client) MapPropertyAndItemConfiguration(i interface{}, create_if_not_there bool) error

MapPropertyAndItemConfiguration will take a pointer to a Go structure that has the embedded wikibase header and item and property tags on its fields and create a map that goes from the labels in the tags to the Item and Property IDs used by Wikibase.

func (*Client) ProtectPageByID

func (c *Client) ProtectPageByID(page_id int) error

ProtectPageByID will attempt to set the edit protection on a page with the given title to admin. Will fail if page does not exist.

func (*Client) ProtectPageByTitle

func (c *Client) ProtectPageByTitle(title string) error

ProtectPageByTitle will attempt to set the edit protection on a page with the given title to admin. Will fail if page does not exist.

func (*Client) UploadClaimsForItem

func (c *Client) UploadClaimsForItem(i interface{}, allow_refresh bool) error

UploadClaimsForItem will take a pointer to a Go structure that has the embedded wikibase header and item and property tags on its fields and set the claims on the item to match. The item must have been created already. If allow_refresh is set to true, all properties will be written, regardless of whether they've been uploaded before; if set to false only items with no existing Wikibase Property ID in the map will be updated.

type ConsumerInformation

type ConsumerInformation struct {
	Key    string `json:"key"`
	Secret string `json:"secret"`
}

type ItemClaim

type ItemClaim struct {
	EntityType string `json:"entity-type"`
	NumericID  int    `json:"numeric-id"`
}

func ItemClaimToAPIData

func ItemClaimToAPIData(value ItemPropertyType) (ItemClaim, error)

type ItemHeader

type ItemHeader struct {
	ID          ItemPropertyType  `json:"wikibase_id,omitempty"`
	PropertyIDs map[string]string `json:"wikibase_property_ids,omitempty"`
}

ItemHeader must be embedded in all structs that are to be uploaded to Wikibase. If you give this embedded struct a JSON Tag then you can save and restore the Wikibase ID state for the entire struct, which can be used to avoid creating the item multiple times when you run.

Each field that you want to sync as a property on the item in Wikibase must have a "property" tag, with the name of the label of a property on Wikibase. We use labels rather than P numbers as it's impossible to guarantee that production, staging, and test servers will use the same P numbers as they are allocated automatically by the Wikibase server; labels on the other hand can be managed by humans/bots. You should always call the client function MapPropertyAndItemConfiguration to populate it's internal map before attempting to create/update Items and their properties. If you add an "omitoncreate" clause then the Property will not be added to the item at create time, only later on during property sync.

type ItemPropertyType

type ItemPropertyType string

If you're trying to encode structs to properties then you should use these types to help guide the code in encoding things properly for the API

type NetworkClientInterface

type NetworkClientInterface interface {
	Get(args map[string]string) (io.ReadCloser, error)
	Post(args map[string]string) (io.ReadCloser, error)
}

type OAuthInformation

type OAuthInformation struct {
	Consumer ConsumerInformation `json:"consumer"`
	Access   *AccessToken        `json:"access,omitempty"`
}

func LoadOauthInformation

func LoadOauthInformation(path string) (OAuthInformation, error)

type OAuthNetworkClient

type OAuthNetworkClient struct {
	APIURL string

	AccessToken *oauth.AccessToken
	// contains filtered or unexported fields
}

func NewOAuthNetworkClient

func NewOAuthNetworkClient(oauthInfo OAuthInformation, urlbase string) *OAuthNetworkClient

func (*OAuthNetworkClient) Get

func (client *OAuthNetworkClient) Get(args map[string]string) (io.ReadCloser, error)

func (*OAuthNetworkClient) Post

func (client *OAuthNetworkClient) Post(args map[string]string) (io.ReadCloser, error)

type QuantityClaim

type QuantityClaim struct {
	Amount string `json:"amount"`
	Unit   string `json:"unit"`
}

func QuantityClaimToAPIData

func QuantityClaimToAPIData(value int) (QuantityClaim, error)

type SparqlHead

type SparqlHead struct {
	Vars []string `json:"vars"`
}

type SparqlResponse

type SparqlResponse struct {
	Head    SparqlHead    `json:"head"`
	Results SparqlResults `json:"results"`
}

func MakeSPARQLQuery

func MakeSPARQLQuery(service_url string, sparql string) (*SparqlResponse, error)

type SparqlResult

type SparqlResult map[string]SparqlValue

type SparqlResults

type SparqlResults struct {
	Bindings []SparqlResult `json:"bindings"`
}

type SparqlValue

type SparqlValue struct {
	Type     string `json:"type"`
	Value    string `json:"value"`
	DataType string `json:"datatype"`
}

type TimeDataClaim

type TimeDataClaim struct {
	Time          string `json:"time"`
	TimeZone      int    `json:"timezone"`
	Before        int    `json:"before"`
	After         int    `json:"after"`
	Precision     int    `json:"precision"`
	CalendarModel string `json:"calendarmodel"`
}

func TimeDataClaimToAPIData

func TimeDataClaimToAPIData(value string) (TimeDataClaim, error)

type WikiBaseType

type WikiBaseType string

const (
	WikiBaseProperty WikiBaseType = "property"
	WikiBaseItem     WikiBaseType = "item"
)