ghcr

Documentation

Overview

Package ghcr is the GitHub Container Registry source for registry-stats. It collects per-package download counts by scraping the public github.com package pages (GHCR has no unauthenticated API for this data), handling wildcard owner expansion via the owner's packages listing page.

Contract boundaries kept intact through extraction:

• URL shapes (https://github.com/users/{owner}/packages, https://github.com/users/{owner}/packages/container/package/{name}) are unchanged; httpx.DockerGitHubRedirectPolicy (wired on the shared *http.Client in main.go) still enforces the SSRF allowlist for github.com hops.
• The HTML parsing heuristics (Total downloads marker, 500-byte title= search window, /users/{owner}/packages/container/package/ prefix matching) are preserved verbatim. Any change here is a behavior-change deferral, not a refactor.
• ErrHTMLFormatChanged is the single sentinel returned for any parse failure; callers in main.go compare via errors.Is to decide whether to emit the "format may be changing" WARN/ERROR logs.

The package exposes a *Client (for composition-root wiring via api.RegistrySource). The HTML-handling internals (fetchHTML, scrapePackageList, scrapeDownloads, buildPackageList) are unexported — all collection flows go through *Client.Collect. ParseDownloads and ParsePackageList stay exported as pure-core parse helpers so in-package and external parse-only tests can drive them without constructing a Client; ErrHTMLFormatChanged stays exported as the sentinel callers compare via errors.Is.

Index

• Constants
• Variables
• func ParseDownloads(html string) (int64, error)
• func ParsePackageList(html, owner string) ([]string, error)
• type Client
  • func NewClient(client *http.Client, retryOpts []httpx.Option, opts Options, ...) *Client
  • func (c *Client) Collect(ctx context.Context, refs []model.RepoRef) (entries []model.RegistryEntry, attempted int, healthy bool)
  • func (c *Client) Name() string
  • func (c *Client) Source() model.RegistrySource
• type Options

Constants

const (
	DefaultMinPacing    = 2 * time.Second
	DefaultPacingJitter = 3 * time.Second
)

DefaultMinPacing and DefaultPacingJitter are the production pacing values applied when an Options field is zero. collect() adds a uniformly distributed jitter in [0, DefaultPacingJitter) to DefaultMinPacing to space out consecutive GHCR scrape requests.

Variables

var ErrHTMLFormatChanged = errors.New("GHCR HTML format changed")

ErrHTMLFormatChanged is the sentinel returned for any GHCR HTML parse failure — missing "Total downloads" marker, missing title attribute, non-numeric or negative count, empty package list. Callers compare via errors.Is to distinguish parse drift from transport errors.

Functions

func ParseDownloads

func ParseDownloads(html string) (int64, error)

ParseDownloads extracts the download count from a single package page. Looks for the "Total downloads" marker, then scans forward at most maxTitleDistance bytes for the first title="N" attribute and parses N as a non-negative int64. Line boundaries are intentionally not meaningful — GitHub's HTML can reflow whitespace without breaking this parser.

func ParsePackageList

func ParsePackageList(html, owner string) ([]string, error)

ParsePackageList extracts package names from an owner's packages page HTML. Looks for /users/{owner}/packages/container/package/{name} links and filters names through urlsafe.IsSafeURLSegment so a crafted page cannot smuggle path traversal into downstream URL construction. Duplicate names (same page can list a package multiple times) are deduplicated in insertion order.

Types

type Client

type Client struct {
	// contains filtered or unexported fields
}

Client implements api.RegistrySource for the GitHub Container Registry. Construct via NewClient; the zero value is not usable.

func NewClient

func NewClient(client *http.Client, retryOpts []httpx.Option, opts Options, logger *slog.Logger) *Client

NewClient returns a Client that uses the provided *http.Client for all outbound requests, applying retryOpts to each call via httpx.Retry. opts configures GHCR-specific pacing; its zero value selects DefaultMinPacing + DefaultPacingJitter. A nil logger falls back to slog.Default.

func (*Client) Collect

func (c *Client) Collect(
	ctx context.Context,
	refs []model.RepoRef,
) (entries []model.RegistryEntry, attempted int, healthy bool)

Collect gathers download counts for every ref in refs. Wildcard refs are expanded via buildPackageList before scraping; explicit refs are scraped as-is. Packages whose scrape fails are NOT appended so a transient error cannot corrupt the daily-delta calculation.

entries carry only the GHCR-relevant fields (Name, DownloadCount); PullCount / LastUpdated / Tags stay zero-valued. attempted counts per-package scrape attempts (listing failures do not contribute to the per-package health ratio). healthy mirrors the legacy formula: no per-package failures OR fewer failures than successes.

func (*Client) Name

func (c *Client) Name() string

Name identifies this source in logs and in the per-source health ratio. Matches the "ghcr" const used on the HTTP API surface.

func (*Client) Source

func (c *Client) Source() model.RegistrySource

Source returns the typed model.RegistrySource the orchestrator uses to route entries into snap.GHCR without a string compare. model.SourceGHCR.String() must stay equal to Name() — both read as "ghcr" on the wire.

type Options

type Options struct {
	MinPacing    time.Duration
	PacingJitter time.Duration
}

Options configures GHCR-specific scraper policy. Its zero value selects production defaults (DefaultMinPacing + DefaultPacingJitter) so main.go can pass ghcr.Options{} and preserve the pre-c3 2-5s per-package pacing byte-for-byte.