fetchers

Documentation

Index

• func Items(ids []string, pt vangogh_local_data.ProductType, httpClient *http.Client) error
• func NewIndexSetter(pt vangogh_local_data.ProductType, ids []string) (dolo.IndexSetter, error)
• func Pages(pt vangogh_local_data.ProductType, since int64, httpClient *http.Client, ...) error

Functions

func Items

func Items(
	ids []string,
	pt vangogh_local_data.ProductType,
	httpClient *http.Client) error

Items fetches all individual data items (details, api-products-v1/v2) using provided ids

func NewIndexSetter

func NewIndexSetter(pt vangogh_local_data.ProductType, ids []string) (dolo.IndexSetter, error)

func Pages

func Pages(pt vangogh_local_data.ProductType, since int64, httpClient *http.Client, tpw nod.TotalProgressWriter) error

Pages fetches all paged product type pages concurrently (using dolo.GetSet). To do that it downloads the first page, decodes that to get TotalPages, then constructs a slice of URLs and page ids to download all the remaining pages from 2nd to TotalPages using kvas index setter.