registry

Documentation

Overview

Package registry fetches package metadata and archives from npm and PyPI.

Index

• Constants
• type Artifact
• type Client
• type File
• type Metadata
• type NPMClient
  • func NewNPM() *NPMClient
  • func (c *NPMClient) Fetch(ctx context.Context, name, version string) (*Artifact, error)
• type PyPIClient
  • func NewPyPI() *PyPIClient
  • func (c *PyPIClient) Fetch(ctx context.Context, name, version string) (*Artifact, error)

Constants

const (
	MaxArchiveBytes  = 80 << 20 // 80 MiB compressed download cap
	MaxFileBytes     = 2 << 20  // 2 MiB per extracted file (enough for source)
	MaxFilesPerPkg   = 4000     // hard cap of files inspected per package
	MaxTotalUnpacked = 256 << 20
)

Limits applied while downloading and extracting archives. They keep memory bounded and defend against decompression bombs shipped by hostile packages.

Types

type Artifact

type Artifact struct {
	Meta  Metadata
	Files []File
}

Artifact bundles everything the analyzer needs about one package version.

type Client

type Client interface {
	Fetch(ctx context.Context, name, version string) (*Artifact, error)
}

Client resolves a package version into an analyzable Artifact.

type File

type File struct {
	Path      string
	Content   []byte
	Truncated bool
}

File is one extracted file from a package archive, truncated to MaxFileBytes.

type Metadata

type Metadata struct {
	Name             string
	Version          string
	PublishedAt      time.Time // when this exact version was published
	FirstPublishedAt time.Time // when the package first appeared
	Maintainers      []string
	// InstallScripts are lifecycle hooks declared in the manifest
	// (npm: preinstall/install/postinstall — the Shai-Hulud entry point).
	InstallScripts map[string]string
	TarballURL     string
}

Metadata is registry-level information about a specific package version.

type NPMClient

type NPMClient struct {
	BaseURL string // default https://registry.npmjs.org
}

NPMClient talks to the public npm registry (or a compatible mirror).

func NewNPM

func NewNPM() *NPMClient

func (*NPMClient) Fetch

func (c *NPMClient) Fetch(ctx context.Context, name, version string) (*Artifact, error)

Fetch implements Client for the npm ecosystem.

type PyPIClient

type PyPIClient struct {
	BaseURL string // default https://pypi.org
}

PyPIClient talks to the public PyPI JSON API.

func NewPyPI

func NewPyPI() *PyPIClient

func (*PyPIClient) Fetch

func (c *PyPIClient) Fetch(ctx context.Context, name, version string) (*Artifact, error)

Fetch implements Client for PyPI. It prefers the sdist (closer to what executes at install time via setup.py) and falls back to the wheel.