proxypool

Documentation

Overview

Package proxypool provides a rotating proxy pool with passive health checking and circuit breaking for the httpc library.

A Pool holds a list of proxy URLs and selects one per request according to a strategy (round-robin or random). Connection-level failures are tracked passively: after a configurable number of consecutive failures a proxy's circuit opens and it is temporarily skipped, then automatically retried (half-open probe) after a cooldown.

HTTP status codes (e.g. 403) are NOT treated as proxy failures here — they are target-specific. Status-based rotation is handled at the retry layer by re-invoking Select, which naturally returns a different proxy under round-robin/random.

Index

• Variables
• type Config
• type Pool
  • func New(cfg Config) (*Pool, error)
  • func (p *Pool) Hosts() []string
  • func (p *Pool) Len() int
  • func (p *Pool) NextIndex() int
  • func (p *Pool) ReportFailure(host string)
  • func (p *Pool) ReportSuccess(host string)
  • func (p *Pool) Select(_ *http.Request) (*url.URL, error)
  • func (p *Pool) SelectIndex(attempt int) *url.URL
• type Strategy

Variables

var ErrNoProxies = errors.New("proxy pool is empty")

ErrNoProxies is returned when the pool is created with no proxy URLs.

Types

type Config

type Config struct {
	// Proxies is the list of proxy URLs (http, https, socks5, socks5h).
	// Entries sharing the same host:port are collapsed to the first occurrence.
	Proxies []string

	// Strategy selects how proxies are chosen. The zero value defaults to
	// StrategyRoundRobin.
	Strategy Strategy

	// FailureThreshold is the number of consecutive connection failures to a
	// proxy before its circuit opens. Zero defaults to 3.
	FailureThreshold int

	// Cooldown is how long a circuit stays open before the proxy is eligible
	// again (half-open probe). Zero defaults to 30s.
	Cooldown time.Duration
}

Config configures a proxy pool. Zero-value fields receive defaults.

type Pool

type Pool struct {
	// contains filtered or unexported fields
}

Pool is a concurrency-safe, rotating proxy pool with passive circuit breaking. It is safe for concurrent use by multiple goroutines. Create one with New.

func New

func New(cfg Config) (*Pool, error)

New creates a proxy pool from the given configuration. All proxy URLs are validated with the same validator the public Config layer uses, so scheme and host rules cannot drift. Entries sharing a host:port are collapsed to the first occurrence.

func (*Pool) Hosts

func (p *Pool) Hosts() []string

Hosts returns the host:port of every proxy in the pool, including those whose circuit is currently open. Used to seed SSRF exemptions so that proxy connections are never blocked by private-IP validation.

func (*Pool) Len

func (p *Pool) Len() int

Len returns the number of proxies in the pool.

func (*Pool) NextIndex

func (p *Pool) NextIndex() int

NextIndex atomically advances the round-robin cursor once and returns the resulting entry index (modulo pool size). Unlike Select, it does not check circuit-open proxies or return a URL — it simply reserves a starting position. Used by the retry layer to reserve a unique base proxy per request, ensuring inter-request rotation; each retry attempt then uses base + attempt as the SelectIndex argument for intra-request rotation.

func (*Pool) ReportFailure

func (p *Pool) ReportFailure(host string)

ReportFailure records a connection-level failure (dial/TLS) for the proxy at the given host:port. After FailureThreshold consecutive failures the proxy's circuit opens and it is temporarily skipped by Select for Cooldown.

Only connection-level failures should be reported here. HTTP status codes (e.g. 403) are target-specific and must NOT circuit-break a proxy; they are handled by retrying with a fresh Select (see Connection.ProxyRotateOnStatus).

func (*Pool) ReportSuccess

func (p *Pool) ReportSuccess(host string)

ReportSuccess resets a proxy's failure count and closes its circuit, marking it healthy for immediate reuse.

func (*Pool) Select

func (p *Pool) Select(_ *http.Request) (*url.URL, error)

Select returns a proxy URL for the given request, skipping any proxy whose circuit is currently open. If every circuit is open it returns the proxy closest to recovery (earliest open-unil timestamp) as a best-effort fallback rather than failing outright.

The request parameter is accepted to satisfy the http.Transport.Proxy signature; it is not used in the current implementation.

func (*Pool) SelectIndex

func (p *Pool) SelectIndex(attempt int) *url.URL

SelectIndex returns a proxy URL deterministically indexed by attempt, skipping any proxy whose circuit is currently open. Unlike Select it does NOT advance the round-robin counter, so the same attempt always lands on the same proxy regardless of how many times the transport calls Proxy within a single logical request (e.g. redirect-following).

This is used by the retry layer to guarantee that retry attempt N selects a DIFFERENT proxy than attempt N-1, even when redirect chains consume extra Select calls and would otherwise desynchronize the round-robin cursor.

type Strategy

type Strategy int

Strategy selects the algorithm for choosing a proxy from the pool.

const (
	// StrategyRoundRobin cycles through proxies in order. Each selection
	// advances the cursor, so a retry that re-invokes Select naturally lands
	// on a different proxy without any extra wiring.
	StrategyRoundRobin Strategy = iota
	// StrategyRandom picks a healthy proxy uniformly at random.
	StrategyRandom
)