pacing

Documentation

Overview

Package pacing provides anti-fingerprint request spacing primitives for stealth HTTP clients: uniform random jitter, per-key pacing, and symmetric percentage jitter for backoff. These are the canonical implementations for the go-* fleet — go-stealth, go-twitter, and go-kit's own retry/breaker packages delegate here instead of carrying ad-hoc copies.

Pacing is distinct from ratelimit: a rate limiter is the authoritative throughput ceiling; pacing adds human-like spacing under that ceiling to evade fingerprinting on request timing. Pacing never blocks a request that the rate limiter would allow — it only delays it.

Index

• Variables
• func ExponentialBackoff(initial, max time.Duration, multiplier, jitterPct float64, attempt int) time.Duration
• func SymmetricJitter(base time.Duration, pct float64) time.Duration
• type Jitter
  • func (j Jitter) Duration() time.Duration
  • func (j Jitter) Sleep(ctx context.Context) error
• type KeyedPacer
  • func NewKeyedPacer(minDelay, randomDelay time.Duration, opts ...PacerOption) *KeyedPacer
  • func (p *KeyedPacer) Allow(key string) bool
  • func (p *KeyedPacer) Wait(ctx context.Context, key string) error
• type PacerOption
  • func WithPacerClock(clock func() time.Time) PacerOption

Variables

var DefaultJitter = Jitter{
	Min: 500 * time.Millisecond,
	Max: 2500 * time.Millisecond,
}

DefaultJitter is 500ms–2.5s, suitable for most scraping. Callers with tighter or looser human-pace requirements should construct their own Jitter.

Functions

func ExponentialBackoff

func ExponentialBackoff(initial, max time.Duration, multiplier, jitterPct float64, attempt int) time.Duration

ExponentialBackoff returns the backoff delay for the given attempt (0-indexed) with symmetric jitter applied. Formula: initial * mult^attempt, capped at max, then ±pct jitter. This is the canonical exponential backoff with jitter — go-stealth/BackoffConfig.Duration and go-kit/breaker.computeBackoff delegate here.

func SymmetricJitter

func SymmetricJitter(base time.Duration, pct float64) time.Duration

SymmetricJitter applies ±pct% random variation to a base duration, returning a duration in [base*(1-pct), base*(1+pct)]. pct is a fraction in [0,1]: 0.25 means ±25%. Returns base unchanged when pct <= 0 or base <= 0. The result is clamped to be non-negative.

This is the canonical symmetric jitter for backoff/retry delay variation across the fleet — go-kit/retry, go-kit/breaker, and go-stealth/BackoffConfig all delegate here.

Types

type Jitter

type Jitter struct {
	Min time.Duration
	Max time.Duration
}

Jitter defines a uniform random delay range for anti-fingerprinting. A request spaced by Jitter fires at a uniformly random point in [Min, Max), which avoids the periodic burst pattern that a fixed delay produces.

func (Jitter) Duration

func (j Jitter) Duration() time.Duration

Duration returns a uniform random duration in [Min, Max). Returns 0 if Max <= Min (degenerate range).

func (Jitter) Sleep

func (j Jitter) Sleep(ctx context.Context) error

Sleep pauses for a uniform random duration in [Min, Max). Returns ctx.Err() if the context is cancelled during the wait. A degenerate range (Max <= Min) returns immediately without sleeping.

type KeyedPacer

type KeyedPacer struct {
	// contains filtered or unexported fields
}

KeyedPacer spaces consecutive requests for the SAME key by a minimum delay plus optional random jitter. Pacing is independent per key: a recent request on key A never delays key B. This is the per-account stealth pacer — keyed by account ID after the pool selects an account, so each account self-paces its own request rhythm without a single global gate that would starve a low-frequency caller. It deliberately carries NO window/rate limiter: the per-account-per-endpoint ratelimit.Limiter is the authoritative throughput ceiling; this only adds human-like spacing under that ceiling.

func NewKeyedPacer

func NewKeyedPacer(minDelay, randomDelay time.Duration, opts ...PacerOption) *KeyedPacer

NewKeyedPacer creates a per-key pacer. minDelay is the hard floor between consecutive same-key requests; randomDelay adds [0, randomDelay) jitter on top so realized spacing is human-variable. Both zero ⇒ pacing disabled (Allow always true, Wait always immediate).

func (*KeyedPacer) Allow

func (p *KeyedPacer) Allow(key string) bool

Allow reports whether a request for key may proceed now. When it returns true it arms the key's next-allowed time by sampling MinDelay+jitter ONCE, so the jitter is rolled exactly once per granted request (faithful spacing distribution), not re-rolled on every poll. The first request for any key is always allowed (no prior grant to space against).

func (*KeyedPacer) Wait

func (p *KeyedPacer) Wait(ctx context.Context, key string) error

Wait blocks until a request for key is allowed or ctx is cancelled. It polls Allow at pacerPollInterval. Returns ctx.Err() if the context is cancelled before the key becomes available.

type PacerOption

type PacerOption func(*KeyedPacer)

PacerOption configures a KeyedPacer.

func WithPacerClock

func WithPacerClock(clock func() time.Time) PacerOption

WithPacerClock injects the time source (default time.Now) so tests can advance time deterministically instead of sleeping.