puba

Documentation

Overview

Package puba scrapes the Puba pornstar network at puba.com. The site has a JSON-API page at `/pornstarnetwork/index.php` that takes a few well-known query parameters:

• `section=538` — videos (vs. 539 = sites listing, 320 = images)
• `view=v` — videos view (when not filtering by `group`)
• `group={N}` — per-pornstar / per-sub-site filter
• `searching=Search` — required: the PHP backend treats the request as a search request only when this flag is present
• `start={offset}&count={N}` — pagination
• `format=json&resource=video` — switch from HTML to JSON

JSON response shape: `{total, page, num_pages, items:[{galid, secid, description, image_url, video_url, actors, time, favorite}]}`.

Items are sorted by `galid` descending (newest first), which makes incremental scrapes with `KnownIDs` early-stop reliable.

13 pornstar sub-sites map to `group={N}` filter IDs; the network's `?section=539` index lists them all as `<!-- {SiteName} --> … group=N` pairs. The parent `puba` scraper omits the filter and walks the full 2800+ video catalogue.

Index

• type Scraper
  • func New(cfg SiteConfig) *Scraper
  • func (s *Scraper) ID() string
  • func (s *Scraper) ListScenes(ctx context.Context, studioURL string, opts scraper.ListOpts) (<-chan scraper.SceneResult, error)
  • func (s *Scraper) MatchesURL(u string) bool
  • func (s *Scraper) Patterns() []string
• type SiteConfig

Types

type Scraper

type Scraper struct {
	// contains filtered or unexported fields
}

func New

func New(cfg SiteConfig) *Scraper

func (*Scraper) ID

func (s *Scraper) ID() string

func (*Scraper) ListScenes

func (s *Scraper) ListScenes(ctx context.Context, studioURL string, opts scraper.ListOpts) (<-chan scraper.SceneResult, error)

func (*Scraper) MatchesURL

func (s *Scraper) MatchesURL(u string) bool

func (*Scraper) Patterns

func (s *Scraper) Patterns() []string

type SiteConfig

type SiteConfig struct {
	ID       string
	SiteName string // shown in Scene.Series; empty for the parent network
	Group    int    // 0 = whole catalogue, N = pornstarnetwork group filter
	Patterns []string
	MatchRe  *regexp.Regexp
}

SiteConfig describes one Puba scraper — either the parent network (Group == 0) or a per-pornstar / per-sub-site filter (Group > 0).