Documentation
ΒΆ
Overview ΒΆ
Package aoni is a generic, middleware-driven HTTP and WebSocket client wrapper around the standard net/http package, specifically engineered for chaotic, unstable, and highly defensive network environments.
It operates under the philosophy: "In networks, chaos is the default. Let aoni be your ice-cold anchor." While standard clients are optimized for predictable internal cloud microservices, aoni serves as tactical "off-road armor" for uncooperative targetsβhandling aggressive firewalls, rotating unstable proxy pools, and evading deep-packet inspection (DPI).
Pipeline Philosophy ΒΆ
HTTP requests in aoni are modeled as fluid pipeline streams processed in four distinct phases:
- RequestModifiers: Declarative request decoration (Headers, URL Variables, Decoders, Body).
- Middlewares: Interception and control (Rate limiters, Concurrency, Circuit Breakers, Retries).
- Transport Layer: Multi-protocol execution (uTLS, HTTP/3, p0f Spoofing, ProxyRotator, LoadBalancer).
- Generic Decoders: Output binding with automated UTF-8 transcoding and multi-format decompression (Gzip, Brotli, Zstd).
Architecture Overview ΒΆ
- Client: The central, immutable client wrapping HTTPDoer with a fluent API. Every configuration call clones the client, maintaining thread-safety.
- LoadBalancer: A failover router distributing requests across backends with health checking, prewarming, and structured slog-compatible event logging.
- ProxyRotator: A proxy pool router supporting sticky sessions, proxy connection fault metrics, and automatic failover recovery with detailed logs.
- Decoder: An interface defining response deserialization strategies (JSON, XML, YAML, Raw).
- ProxyIsolatedCookieJar: An isolated cookie sandbox grouping cookie storage by active proxy URL to prevent session leakage and fingerprint anomalies.
Deep Evasion & Fingerprinting ΒΆ
- JA4+ Tracing: Real-time fingerprinting tracking TLS client (JA4) and HTTP request (JA4H) structures during live transactions.
- uTLS Integration: Custom TLS ClientHello emulation matching specific browser engines (Chrome, Firefox, Safari).
- TCP/IP p0f Spoofing: Low-level socket control mimicking target OS network stacks (TTL, Don't Fragment, Window Size).
- Header Ordering: Custom HTTP/1.1 header sequence preservation to match browser-specific HTTP signatures.
- Custom HTTP/2 Settings: Fine-tuning of h2 parameters (MaxHeaderListSize, InitialWindowSize) to prevent transport mismatches.
Resilience & Chaos Control ΒΆ
- Latency Hedging: Dual concurrent request dispatch to mitigate tail (p99) latencies under high load.
- Vegas Adaptive Concurrency: Dynamic concurrency limits based on Vegas TCP-style round-trip time (RTT) queuing.
- Circuit Breakers: Independent host circuit-health tracking preventing cascading downstream blockages.
- Robust Retry Engine: Retries with exponential backoff, custom jitter, and out-of-the-box policies for transient errors, gateway issues, and 429 Rate Limits (respecting Retry-After).
Modern Network Protocols ΒΆ
- HTTP/3 (QUIC): Built-in transport support for UDP-based connections via http3.Transport.
- DNS-over-HTTPS & DNS-over-TLS: Native DoH/DoT resolvers bypassing local ISP DNS interception.
- In-Memory DNS Caching: Custom TTL-based DNS caching avoiding redundant DNS query overhead.
- WebSocket over H2/uTLS: Native WebSocket dialing using uTLS/JA4 fingerprinting and modern HTTP/2 Extended CONNECT (RFC 8441) tunnels.
Automation Session Sync ΒΆ
- Headless Browser Sync: Native ExportCookies and ImportCookies functions to translate Cookie Jars into flat, serialization-ready structures for Playwright or Chromedp.
Basic Usage Example ΒΆ
package main
import (
"context"
"fmt"
"log"
"time"
"github.com/lemon4ksan/aoni"
)
type TimeResponse struct {
Time string `json:"time"`
}
func main() {
ctx := context.Background()
// Initialize client with fluent base URL and timeout configurations
client := aoni.NewClient(nil).
WithBaseURL("https://time.jsontest.com").
WithTimeout(5 * time.Second)
// Perform a generic, type-safe GET request
res, err := aoni.GetJSON[TimeResponse](ctx, client, "/time")
if err != nil {
log.Fatalf("Request failed: %v", err)
}
fmt.Println("Server time:", res.Time)
}
Index ΒΆ
- Constants
- Variables
- func CurlCommand(req *http.Request, body []byte) string
- func DefaultCircuitBreakerCondition(resp *http.Response, err error) bool
- func DefaultRedirectPolicy(maxRedirects int, sensitiveHeaders ...string) func(req *http.Request, via []*http.Request) error
- func Delete[Req, Resp any](ctx context.Context, path string, payload Req, mods ...RequestModifier) (*Resp, error)
- func DeleteJSON[Req, Resp any](ctx context.Context, c Requester, path string, payload Req, ...) (*Resp, error)
- func DialWebSocket(ctx context.Context, c *Client, targetURL string, mods ...RequestModifier) (net.Conn, *http.Response, error)
- func DialWebSocketWithConfig(ctx context.Context, c *Client, targetURL string, config DialWebSocketConfig, ...) (net.Conn, *http.Response, error)
- func Get[Resp any](ctx context.Context, path string, mods ...RequestModifier) (*Resp, error)
- func GetJSON[Resp any](ctx context.Context, c Requester, path string, mods ...RequestModifier) (*Resp, error)
- func ImportCookies(jar http.CookieJar, u *url.URL, cookies []CookieData)
- func NewFragmentedConn(conn net.Conn, cfg *FragmentConfig) net.Conn
- func NewProxyClient(cfg ProxyConfig) (*http.Client, error)
- func ParseSSE[T any](ctx context.Context, resp *StreamResponse) (<-chan T, <-chan error)
- func Patch[Req, Resp any](ctx context.Context, path string, payload Req, mods ...RequestModifier) (*Resp, error)
- func PatchJSON[Req, Resp any](ctx context.Context, c Requester, path string, payload Req, ...) (*Resp, error)
- func Post[Req, Resp any](ctx context.Context, path string, payload Req, mods ...RequestModifier) (*Resp, error)
- func PostForm[Req, Resp any](ctx context.Context, c Requester, path string, payload Req, ...) (*Resp, error)
- func PostJSON[Req, Resp any](ctx context.Context, c Requester, path string, payload Req, ...) (*Resp, error)
- func Put[Req, Resp any](ctx context.Context, path string, payload Req, mods ...RequestModifier) (*Resp, error)
- func PutJSON[Req, Resp any](ctx context.Context, c Requester, path string, payload Req, ...) (*Resp, error)
- func StreamNDJSON[T any](ctx context.Context, resp *StreamResponse) (<-chan T, <-chan error)
- func StreamSSE[T any](ctx context.Context, c Requester, path string, mods ...RequestModifier) (<-chan T, <-chan error, error)
- func StructToValues(s any) (url.Values, error)
- func Validate(s any) error
- type APIError
- type AdaptiveLimiter
- type Backend
- type BaseResponse
- type BaseResponseProvider
- type BoolInt
- type BrowserID
- type ChaosConfig
- type CircuitBreaker
- type CircuitBreakerConfig
- type CircuitState
- type Client
- func (c *Client) BaseResponse() BaseResponse
- func (c *Client) Clone() *Client
- func (c *Client) CloseIdleConnections()
- func (c *Client) HTTP() HTTPDoer
- func (c *Client) Logger() Logger
- func (c *Client) Request(ctx context.Context, method, path string, mods ...RequestModifier) (*http.Response, error)
- func (c *Client) Transport() *http.Transport
- func (c *Client) UserAgent() string
- func (c *Client) WithAfterResponse(hook func(resp *http.Response, err error)) *Client
- func (c *Client) WithBaseResponse(provider func() BaseResponse) *Client
- func (c *Client) WithBaseURL(raw string) *Client
- func (c *Client) WithBeforeRequest(hook func(req *http.Request)) *Client
- func (c *Client) WithConnectionPool(cfg ConnectionPoolConfig) *Client
- func (c *Client) WithCookieJar(jar http.CookieJar) *Client
- func (c *Client) WithDNSCache(ttl time.Duration) *Client
- func (c *Client) WithDNSResolver(resolver DNSResolver) *Client
- func (c *Client) WithDoTResolver(server, hostname string) *Client
- func (c *Client) WithFragmentation(cfg FragmentConfig) *Client
- func (c *Client) WithHTTP2Settings(settings HTTP2Settings) *Client
- func (c *Client) WithHTTP3() *Client
- func (c *Client) WithHappyEyeballs(delay time.Duration) *Client
- func (c *Client) WithHeader(key, value string) *Client
- func (c *Client) WithHedging(d time.Duration) *Client
- func (c *Client) WithHostRewrite(rules map[string]string) *Client
- func (c *Client) WithJA4Callback(fn func(ja4.Report)) *Client
- func (c *Client) WithLocalAddr(addr string) *Client
- func (c *Client) WithLocalAddrPool(addrs []string) *Client
- func (c *Client) WithLogger(l Logger) *Client
- func (c *Client) WithMaxResponseSize(size int64) *Client
- func (c *Client) WithModifiers(mods ...RequestModifier) *Client
- func (c *Client) WithMultiReadBody(threshold int64) *Client
- func (c *Client) WithOrigin(origin string) *Client
- func (c *Client) WithP0fSignature(sig *p0f.Signature) *Client
- func (c *Client) WithProxyIsolatedCookieJar(jar *ProxyIsolatedCookieJar) *Client
- func (c *Client) WithRedirectLimit(max int) *Client
- func (c *Client) WithSSRFGuard() *Client
- func (c *Client) WithTLSFingerprint(browser BrowserID) *Client
- func (c *Client) WithTimeout(d time.Duration) *Client
- func (c *Client) WithUserAgent(ua string) *Client
- type ClientWithProxy
- type ConnectionPoolConfig
- type CookieData
- type DNSResolver
- type Decoder
- type DecoderFunc
- type DialWebSocketConfig
- type DoHResolver
- type DoTResolver
- type DoerFunc
- type FallbackFunc
- type Float64String
- type FragmentConfig
- type HTTP2Settings
- type HTTPDoer
- type HostRewriteConfig
- type InMemoryDNSCache
- type Int64String
- type JitterStrategy
- type LoadBalancer
- type LoadBalancerConfig
- type LoadBalancingStrategy
- type Logger
- type Middleware
- func AdaptiveLimitMiddleware(limiter *AdaptiveLimiter) Middleware
- func ChaosMiddleware(cfg ChaosConfig) Middleware
- func CircuitBreakerMiddleware(cb *CircuitBreaker, isFailure func(*http.Response, error) bool) Middleware
- func FallbackMiddleware() Middleware
- func FallbackMiddlewareEx(isFailure func(*http.Response, error) bool) Middleware
- func RateLimitMiddleware(rps float64, burst int) Middleware
- func RecoveryMiddleware(onPanic func(any)) Middleware
- func RetryMiddleware(opts RetryOptions, condition RetryCondition) Middleware
- type NoResponse
- type ProgressFunc
- type ProxyConfig
- type ProxyIsolatedCookieJar
- type ProxyRotator
- func (r *ProxyRotator) Close() error
- func (r *ProxyRotator) Do(req *http.Request) (*http.Response, error)
- func (r *ProxyRotator) Prewarm(ctx context.Context, targetURL string)
- func (r *ProxyRotator) UpdateClients(clients ...ClientWithProxy)
- func (r *ProxyRotator) WithStickySessions(f StickyKeyFunc) *ProxyRotator
- type ProxyRotatorConfig
- type ReplayableBody
- type RequestModifier
- func AsCurl() RequestModifier
- func AsJSON() RequestModifier
- func AsRaw() RequestModifier
- func AsXML() RequestModifier
- func AsYAML() RequestModifier
- func CaptureResponse(target **http.Response) RequestModifier
- func Debug() RequestModifier
- func Trace(target *TraceInfo) RequestModifier
- func TraceJA4(target *TraceInfo) RequestModifier
- func WithALPN(protocols []string) RequestModifier
- func WithAccept(accept string) RequestModifier
- func WithBasicAuth(username, password string) RequestModifier
- func WithBearer(token string) RequestModifier
- func WithBody(r io.Reader) RequestModifier
- func WithContentType(ct string) RequestModifier
- func WithCookie(c *http.Cookie) RequestModifier
- func WithCookies(kv map[string]string) RequestModifier
- func WithDecoder(d Decoder) RequestModifier
- func WithDownloadProgress(onProgress ProgressFunc) RequestModifier
- func WithErrorModel(target any) RequestModifier
- func WithFallback(f FallbackFunc) RequestModifier
- func WithForceHTTP1() RequestModifier
- func WithForceHTTP2() RequestModifier
- func WithFragmentation(cfg FragmentConfig) RequestModifier
- func WithHeader(key, value string) RequestModifier
- func WithHedging(delay time.Duration) RequestModifier
- func WithHostRewrite(rules map[string]string) RequestModifier
- func WithJSONBody(payload any) RequestModifier
- func WithMultiReadBody(threshold int64) RequestModifier
- func WithMultipart(fields map[string]string, files map[string]io.Reader) RequestModifier
- func WithOrderedHeaders(order []string) RequestModifier
- func WithOrigin(origin string) RequestModifier
- func WithP0fSignature(sig *p0f.Signature) RequestModifier
- func WithQuery(query any) RequestModifier
- func WithStreamingMultipart(fields map[string]string, files map[string]io.Reader) RequestModifier
- func WithUploadProgress(onProgress ProgressFunc) RequestModifier
- func WithUserAgent(ua string) RequestModifier
- func WithVar(key string, value any) RequestModifier
- func WithVars(pairs ...any) RequestModifier
- type Requester
- type RetryCondition
- type RetryOptions
- type SSEEvent
- type SourceIPRotator
- type StdlibResolver
- type StickyKeyFunc
- type StreamResponse
- type TraceInfo
- type Uint64String
- type UnixTimestamp
- type Unwrapper
- type ValidationError
Constants ΒΆ
const DefaultUserAgent = "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.0.0 Safari/537.36"
DefaultUserAgent is the default User-Agent string used for HTTP requests.
Variables ΒΆ
var ( // ErrUnexpectedContentType is returned when the response content type does not match the expected format. // This often indicates a captive portal or unexpected network interception. ErrUnexpectedContentType = errors.New("aoni: unexpected content-type (possible captive portal or intercept)") // ErrCloudflareChallenge is returned when a Cloudflare JS challenge or CAPTCHA is detected in the response body. ErrCloudflareChallenge = errors.New("aoni: cloudflare challenge detected") // ErrResponseTooLarge is returned when the response size exceeds the configured maximum limit. // It is returned by [Client.Request] if response size checking is active. ErrResponseTooLarge = errors.New("aoni: response size limit exceeded") // ErrSSRFBlocked is returned when a request is blocked because it resolved to a private or local IP address. // It is returned by [Client.Request] if SSRF protection is enabled. ErrSSRFBlocked = errors.New("aoni: request blocked by SSRF guard") )
var DefaultClient = NewClient(nil)
DefaultClient is the shared default client instance used by global helper functions.
var DefaultSensitiveHeaders = []string{
"Authorization",
"Cookie",
"X-Session-ID",
"X-Access-Token",
"X-Access-Key",
"X-Api-Key",
"X-Auth-Token",
}
DefaultSensitiveHeaders is the list of sensitive headers that are scrubbed from requests on redirect.
Functions ΒΆ
func CurlCommand ΒΆ
CurlCommand generates a shell-compatible curl command string from an http.Request and body.
func DefaultCircuitBreakerCondition ΒΆ
DefaultCircuitBreakerCondition reports true for transport errors and HTTP status codes >= 500.
func DefaultRedirectPolicy ΒΆ
func DefaultRedirectPolicy( maxRedirects int, sensitiveHeaders ...string, ) func(req *http.Request, via []*http.Request) error
DefaultRedirectPolicy returns a redirect policy function that stops after maxRedirects, and scrubs sensitiveHeaders from the request. If sensitiveHeaders is empty, DefaultSensitiveHeaders are used.
func Delete ΒΆ
func Delete[Req, Resp any]( ctx context.Context, path string, payload Req, mods ...RequestModifier, ) (*Resp, error)
Delete performs a global DELETE request using DefaultClient and decodes the JSON response body.
func DeleteJSON ΒΆ
func DeleteJSON[Req, Resp any]( ctx context.Context, c Requester, path string, payload Req, mods ...RequestModifier, ) (*Resp, error)
DeleteJSON marshals the payload to JSON, executes a DELETE request, and decodes the response body. It automatically configures the request headers with Content-Type and Accept set to "application/json".
It validates the payload structure beforehand using Validate. Returns a ValidationError if validation fails.
func DialWebSocket ΒΆ
func DialWebSocket( ctx context.Context, c *Client, targetURL string, mods ...RequestModifier, ) (net.Conn, *http.Response, error)
DialWebSocket establishes a WebSocket connection using the same uTLS/JA4 pipeline as regular HTTP requests. It respects proxy configuration, source IP rotation, SSRF guards, and Happy Eyeballs dialing.
The returned net.Conn is a full-duplex byte stream over WebSocket. For wss:// connections, the TLS handshake uses the client's configured browser fingerprint (via Client.WithTLSFingerprint), and JA4 fingerprints are computed during the handshake.
Use TraceJA4 to capture both TLS (JA4) and HTTP (JA4H) fingerprints:
info := &aoni.TraceInfo{}
wsConn, resp, err := aoni.DialWebSocket(ctx, client, "wss://example.com/ws",
aoni.WithHeader("Origin", "https://example.com"),
aoni.TraceJA4(info),
)
fmt.Println(info.JA4.JA4) // TLS fingerprint from the WebSocket handshake
func DialWebSocketWithConfig ΒΆ
func DialWebSocketWithConfig( ctx context.Context, c *Client, targetURL string, config DialWebSocketConfig, mods ...RequestModifier, ) (net.Conn, *http.Response, error)
DialWebSocketWithConfig is like DialWebSocket but allows custom buffer sizes.
func Get ΒΆ
Get performs a global GET request using DefaultClient and decodes the JSON response body.
func GetJSON ΒΆ
func GetJSON[Resp any]( ctx context.Context, c Requester, path string, mods ...RequestModifier, ) (*Resp, error)
GetJSON performs a GET request and decodes the JSON response body into a new instance of Resp. It returns an APIError if the server responds with a non-2xx status code.
func ImportCookies ΒΆ
func ImportCookies(jar http.CookieJar, u *url.URL, cookies []CookieData)
ImportCookies imports cookies from the browser into aoni.CookieJar.
func NewFragmentedConn ΒΆ
func NewFragmentedConn(conn net.Conn, cfg *FragmentConfig) net.Conn
NewFragmentedConn wraps a net.Conn with fragmentation and delay settings.
func NewProxyClient ΒΆ
func NewProxyClient(cfg ProxyConfig) (*http.Client, error)
NewProxyClient initializes a standard http.Client configured with proxy transport. It prioritizes ProxyConfig.TransportFactory first, then ProxyConfig.Transport, and falls back to a default http.Transport if neither is provided.
If ProxyConfig.ProxyURL is empty, no proxy routing is applied. If ProxyConfig.Timeout is zero, a default 15-second timeout is set.
func ParseSSE ΒΆ
func ParseSSE[T any](ctx context.Context, resp *StreamResponse) (<-chan T, <-chan error)
ParseSSE parses a Server-Sent Event stream and returns a channel of parsed events and an error channel.
func Patch ΒΆ
func Patch[Req, Resp any]( ctx context.Context, path string, payload Req, mods ...RequestModifier, ) (*Resp, error)
Patch performs a global PATCH request using DefaultClient and decodes the JSON response body.
func PatchJSON ΒΆ
func PatchJSON[Req, Resp any]( ctx context.Context, c Requester, path string, payload Req, mods ...RequestModifier, ) (*Resp, error)
PatchJSON marshals the payload to JSON, executes a PATCH request, and decodes the response body. It automatically configures the request headers with Content-Type and Accept set to "application/json".
It validates the payload structure beforehand using Validate. Returns a ValidationError if validation fails.
func Post ΒΆ
func Post[Req, Resp any]( ctx context.Context, path string, payload Req, mods ...RequestModifier, ) (*Resp, error)
Post performs a global POST request using DefaultClient and decodes the JSON response body.
func PostForm ΒΆ
func PostForm[Req, Resp any]( ctx context.Context, c Requester, path string, payload Req, mods ...RequestModifier, ) (*Resp, error)
PostForm marshals the payload, performs a POST request with URL-encoded parameters, and decodes the resulting JSON response body into Resp.
It validates the payload structure beforehand using Validate. Returns a ValidationError if validation fails.
func PostJSON ΒΆ
func PostJSON[Req, Resp any]( ctx context.Context, c Requester, path string, payload Req, mods ...RequestModifier, ) (*Resp, error)
PostJSON marshals the payload to JSON, executes a POST request, and decodes the response body. It automatically configures the request headers with Content-Type and Accept set to "application/json".
It validates the payload structure beforehand using Validate. Returns a ValidationError if validation fails.
func Put ΒΆ
func Put[Req, Resp any]( ctx context.Context, path string, payload Req, mods ...RequestModifier, ) (*Resp, error)
Put performs a global PUT request using DefaultClient and decodes the JSON response body.
func PutJSON ΒΆ
func PutJSON[Req, Resp any]( ctx context.Context, c Requester, path string, payload Req, mods ...RequestModifier, ) (*Resp, error)
PutJSON marshals the payload to JSON, executes a PUT request, and decodes the response body. It automatically configures the request headers with Content-Type and Accept set to "application/json".
It validates the payload structure beforehand using Validate. Returns a ValidationError if validation fails.
func StreamNDJSON ΒΆ
func StreamNDJSON[T any](ctx context.Context, resp *StreamResponse) (<-chan T, <-chan error)
StreamNDJSON reads a newline-delimited JSON stream from the StreamResponse body. It parses values concurrently in a background goroutine and pushes them to the returned channel. It automatically closes channels and connection streams when done or on context cancellation.
func StreamSSE ΒΆ
func StreamSSE[T any]( ctx context.Context, c Requester, path string, mods ...RequestModifier, ) (<-chan T, <-chan error, error)
StreamSSE parses incoming Server-Sent Events from the StreamResponse body. It executes a background parsing loop and closes returned channels when done.
func StructToValues ΒΆ
StructToValues encodes a struct's fields into url.Values using "url" or "json" tags. It recursively expands inline structures and supports slices, arrays, and standard primitive types. Returns an error if the input kind is not a structure or pointer to a structure.
func Validate ΒΆ
Validate inspects a struct's fields for the "validate:required" tag. It performs a deep structural verification and returns ValidationError for the first missing required field.
Types ΒΆ
type APIError ΒΆ
type APIError struct {
// StatusCode is the HTTP status code returned by the server.
StatusCode int
// Body is the raw response body.
Body []byte
// Model is the deserialized error structure, if any.
Model any
}
APIError represents an unsuccessful HTTP response with a status code outside the 2xx range. It contains the raw response body and a deserialized error model if a custom error model was configured on the Client. Inspect this error using errors.As to handle API failures.
type AdaptiveLimiter ΒΆ
type AdaptiveLimiter struct {
// contains filtered or unexported fields
}
AdaptiveLimiter controls concurrency dynamically using a Vegas-style congestion algorithm. It measures round-trip times (RTT) to dynamically adjust the allowed concurrent request limit.
func NewAdaptiveLimiter ΒΆ
func NewAdaptiveLimiter(initialLimit float64) *AdaptiveLimiter
NewAdaptiveLimiter initializes an AdaptiveLimiter instance with default settings.
func (*AdaptiveLimiter) Acquire ΒΆ
func (l *AdaptiveLimiter) Acquire(ctx context.Context) error
Acquire blocks until a concurrent execution slot becomes available or context is cancelled. It returns context.Canceled or context.DeadlineExceeded if the context expires.
func (*AdaptiveLimiter) Limit ΒΆ
func (l *AdaptiveLimiter) Limit() float64
Limit returns the active dynamic concurrency limit.
func (*AdaptiveLimiter) Release ΒΆ
func (l *AdaptiveLimiter) Release(rtt time.Duration)
Release registers request completion, updates RTT metrics, and recalculates limits. It adjusts the concurrency limit based on Vegas queuing limits (alpha and beta thresholds).
type Backend ΒΆ
type Backend struct {
// URL is the string representation of the backend server address.
URL string
// Weight is the selection weight for the WeightedRoundRobin strategy.
Weight int
// contains filtered or unexported fields
}
Backend represents a single backend server configured in the load balancer.
type BaseResponse ΒΆ
type BaseResponse interface {
// IsSuccess reports whether the response indicates a successful operation.
IsSuccess() bool
// Error returns an error representation if IsSuccess returns false.
Error() error
// SetData registers the destination pointer where the payload is decoded.
SetData(data any)
}
BaseResponse defines the contract for structured response wrappers. It handles success status checking and decoding destination binding.
type BaseResponseProvider ΒΆ
type BaseResponseProvider interface {
BaseResponse() BaseResponse
}
BaseResponseProvider defines an optional interface for a Requester to provide a BaseResponse wrapper for structured decoding.
type BoolInt ΒΆ
type BoolInt bool
BoolInt handles booleans that Steam sends as 1 (true) or 0 (false). It also handles string variations ("1", "0", "true", "false").
func (*BoolInt) UnmarshalJSON ΒΆ
UnmarshalJSON implements json.Unmarshaler.
type BrowserID ΒΆ
type BrowserID int
BrowserID defines browser TLS configurations used for JA3 fingerprint evasion. Pass these identifiers to Client.WithTLSFingerprint.
const ( // BrowserNone disables TLS fingerprint emulation. BrowserNone BrowserID = iota // BrowserChrome emulates Google Chrome TLS fingerprints. BrowserChrome // BrowserFirefox emulates Mozilla Firefox TLS fingerprints. BrowserFirefox // BrowserSafari emulates Apple Safari TLS fingerprints. BrowserSafari )
type ChaosConfig ΒΆ
type ChaosConfig struct {
// FailureRate is the probability (0.0 to 1.0) of randomly returning a 503 error.
FailureRate float64
// LatencyMin is the minimum artificial delay duration applied to requests.
LatencyMin time.Duration
// LatencyMax is the maximum artificial delay duration applied to requests.
LatencyMax time.Duration
}
ChaosConfig defines metrics for injecting synthetic latency and failure rates.
type CircuitBreaker ΒΆ
type CircuitBreaker struct {
// contains filtered or unexported fields
}
CircuitBreaker tracks connection health per host to prevent cascading failures. It keeps state histories for each host and blocks requests during outages.
func NewCircuitBreaker ΒΆ
func NewCircuitBreaker(cfg CircuitBreakerConfig) *CircuitBreaker
NewCircuitBreaker initializes a CircuitBreaker instance. If the configuration values are zero, they default to 5 fails, 2 successes, and a 10-second cooldown.
type CircuitBreakerConfig ΒΆ
type CircuitBreakerConfig struct {
// FailureThreshold is the consecutive failure limit that triggers an open state.
FailureThreshold uint32
// SuccessThreshold is the trial success count required to close a half-open breaker.
SuccessThreshold uint32
// Cooldown is the duration the breaker remains open before transitioning to half-open.
Cooldown time.Duration
}
CircuitBreakerConfig defines performance thresholds for a CircuitBreaker.
type CircuitState ΒΆ
type CircuitState int
CircuitState represents the execution state of a host circuit breaker.
const ( // StateClosed indicates a healthy state allowing all requests to pass. StateClosed CircuitState = iota // StateOpen indicates a failing state where requests are blocked instantly. StateOpen // StateHalfOpen indicates a testing state permitting trial requests to verify recovery. StateHalfOpen )
type Client ΒΆ
type Client struct {
// contains filtered or unexported fields
}
Client is a thread-safe, immutable HTTP client implementing Requester. It manages base URLs, default headers, and custom transport options. Use NewClient to initialize a new instance.
func NewClient ΒΆ
NewClient initializes a new Client instance with DefaultUserAgent. If the provided httpClient is nil, a default http.Client is used. The default client has a 15-second timeout and scrubs sensitive headers on redirect using DefaultRedirectPolicy.
func UnwrapClient ΒΆ
UnwrapClient recursively unwraps any decorator or wrapper layers from the Requester interface, reaching down to the original concrete structure *Client. Returns nil if the passed object is a clean test mock.
func (*Client) BaseResponse ΒΆ
func (c *Client) BaseResponse() BaseResponse
BaseResponse returns a new BaseResponse wrapper if a provider is configured on the client. Returns nil if no provider is set.
func (*Client) Clone ΒΆ
Clone returns a deep copy of the Client instance. It is used internally to maintain immutability across configuration calls.
func (*Client) CloseIdleConnections ΒΆ
func (c *Client) CloseIdleConnections()
CloseIdleConnections closes any idle keep-alive connections maintained by the client. This only works if the underlying HTTPDoer is an http.Client.
func (*Client) Logger ΒΆ
Logger returns the logger used by the client. If no logger is set, a no-op logger is returned.
func (*Client) Request ΒΆ
func (c *Client) Request( ctx context.Context, method, path string, mods ...RequestModifier, ) (*http.Response, error)
Request constructs and executes an HTTP request using the configured transport. It resolves relative paths against the configured base URL.
Context cancellation or timeouts are fully propagated to the underlying transport. If SSRF guarding is enabled, requests resolving to private or loopback IPs return ErrSSRFBlocked. If response size limits are set, reading past the limit returns ErrResponseTooLarge.
If path is empty, it resolves directly to the base URL. Nil modifiers in the mods slice are safely ignored.
The method performs automatic decompression (brotli, zstd, gzip) and content-type charset transcoding to UTF-8. It registers a garbage collection finalizer on the response body to prevent socket leaks, and caches responses below the multi-read threshold.
func (*Client) Transport ΒΆ
Transport returns the underlying http.Transport of the client. Returns nil if the HTTPDoer is not an http.Client or its transport is not an http.Transport.
func (*Client) UserAgent ΒΆ
UserAgent returns the default User-Agent header value configured on the client.
func (*Client) WithAfterResponse ΒΆ
WithAfterResponse returns a new Client with the given response hook registered. After-response hooks are executed regardless of whether the request succeeded or failed.
func (*Client) WithBaseResponse ΒΆ
func (c *Client) WithBaseResponse(provider func() BaseResponse) *Client
WithBaseResponse returns a new Client utilizing the given provider for responses. Pass nil to clear any previously configured provider.
func (*Client) WithBaseURL ΒΆ
WithBaseURL returns a new Client with the specified base URL. An empty raw string clears the base URL. Relative paths in Client.Request are resolved against this base URL.
func (*Client) WithBeforeRequest ΒΆ
WithBeforeRequest returns a new Client with the given request hook registered. Before-request hooks are executed in the order they are registered.
func (*Client) WithConnectionPool ΒΆ
func (c *Client) WithConnectionPool(cfg ConnectionPoolConfig) *Client
WithConnectionPool returns a new Client with the transport pool tuned to the given configuration. It is only effective if the client is using an http.Client with an http.Transport. If config fields are <= 0, they are ignored and the original transport settings are kept.
func (*Client) WithCookieJar ΒΆ
WithCookieJar returns a new Client configured with the specified cookie jar. This configuration is only applied if the underlying HTTPDoer is an http.Client.
func (*Client) WithDNSCache ΒΆ
WithDNSCache returns a new Client that wraps the current DNS resolver with an in-memory cache using the specified TTL.
func (*Client) WithDNSResolver ΒΆ
func (c *Client) WithDNSResolver(resolver DNSResolver) *Client
WithDNSResolver returns a new Client with the given DNS resolver.
func (*Client) WithDoTResolver ΒΆ
WithDoTResolver returns a new Client configured to use DNS-over-TLS for resolution.
func (*Client) WithFragmentation ΒΆ
func (c *Client) WithFragmentation(cfg FragmentConfig) *Client
WithFragmentation returns a new Client configured with TLS fragmentation settings. When set, the TLS ClientHello is split into smaller chunks across multiple TCP segments.
func (*Client) WithHTTP2Settings ΒΆ
func (c *Client) WithHTTP2Settings(settings HTTP2Settings) *Client
WithHTTP2Settings returns a new Client configured with custom HTTP/2 parameters.
func (*Client) WithHappyEyeballs ΒΆ
WithHappyEyeballs returns a new Client configured with a Happy Eyeballs staggered delay. Setting delay <= 0 disables staggered dialing and uses a single connection attempt.
func (*Client) WithHeader ΒΆ
WithHeader returns a new Client with the given default header set. It overwrites any existing header with the same key.
func (*Client) WithHedging ΒΆ
WithHedging returns a new Client configured with the specified hedging delay. Hedging sends a secondary request if the primary request does not respond within the delay. A delay <= 0 disables request hedging.
func (*Client) WithHostRewrite ΒΆ
WithHostRewrite returns a new Client that rewrites hostnames to IP addresses while preserving the original hostname for TLS SNI.
func (*Client) WithJA4Callback ΒΆ
WithJA4Callback returns a new Client that invokes fn with the JA4 fingerprint after each TLS handshake. The callback receives both the TLS (JA4) fingerprint and, if TraceJA4 is used, the HTTP (JA4H) fingerprint.
This option requires [WithTLSFingerprint] to be enabled.
func (*Client) WithLocalAddr ΒΆ
WithLocalAddr returns a new Client bound to the specified local IP address. An invalid IP address string will prevent the custom dialer from binding. This option is only applied if the underlying HTTPDoer is an http.Client with an http.Transport.
func (*Client) WithLocalAddrPool ΒΆ
WithLocalAddrPool returns a new Client with the given local address pool.
func (*Client) WithLogger ΒΆ
WithLogger returns a new Client with the given logger.
func (*Client) WithMaxResponseSize ΒΆ
WithMaxResponseSize returns a new Client enforcing the specified maximum response size. Setting size <= 0 disables any response size limits.
func (*Client) WithModifiers ΒΆ
func (c *Client) WithModifiers(mods ...RequestModifier) *Client
WithModifiers returns a new Client with the given request modifiers applied by default.
func (*Client) WithMultiReadBody ΒΆ
WithMultiReadBody returns a new Client with the default response body caching threshold. Setting threshold <= 0 disables automatic response body caching.
func (*Client) WithOrigin ΒΆ
WithOrigin returns a new Client configured with the specified Origin header.
func (*Client) WithP0fSignature ΒΆ
WithP0fSignature configures TCP/IP spoofing based on a p0f signature. After the TCP connection is established, spoofable fields (TTL, DF, window) are applied via syscalls to make the connection appear as the specified OS to passive fingerprinters like p0f.
func (*Client) WithProxyIsolatedCookieJar ΒΆ
func (c *Client) WithProxyIsolatedCookieJar(jar *ProxyIsolatedCookieJar) *Client
WithProxyIsolatedCookieJar returns a new Client that uses per-proxy cookie isolation. Each proxy gets its own cookie jar, preventing cross-proxy cookie leakage.
func (*Client) WithRedirectLimit ΒΆ
WithRedirectLimit returns a new Client with a custom redirect handling limit. A max value of 0 disables redirects. A negative value restores default Go behavior.
func (*Client) WithSSRFGuard ΒΆ
WithSSRFGuard returns a new Client with SSRF protection enabled. When enabled, requests resolving to private or loopback IP ranges are blocked.
func (*Client) WithTLSFingerprint ΒΆ
WithTLSFingerprint returns a new Client configured to use uTLS for JA3 signature evasion. Passing BrowserNone disables TLS fingerprint emulation. This option is only effective if the client is using an http.Client with an http.Transport.
func (*Client) WithTimeout ΒΆ
WithTimeout returns a new Client configured with the specified request timeout. This configuration is only applied if the underlying HTTPDoer is an http.Client. A duration <= 0 represents no timeout.
type ClientWithProxy ΒΆ
ClientWithProxy is a wrapper around an HTTPDoer that includes a proxy URL.
type ConnectionPoolConfig ΒΆ
type ConnectionPoolConfig struct {
// MaxIdleConns is the maximum number of idle connections across all hosts.
MaxIdleConns int
// MaxIdleConnsPerHost is the maximum number of idle connections kept per host.
MaxIdleConnsPerHost int
// MaxConnsPerHost is the maximum total number of connections allowed per host.
MaxConnsPerHost int
// IdleConnTimeout is the maximum duration an idle connection is kept open.
IdleConnTimeout time.Duration
// ResponseHeaderTimeout is the maximum duration to wait for reading response headers.
ResponseHeaderTimeout time.Duration
}
ConnectionPoolConfig defines tuning parameters for the client's connection pool. Apply these settings to a client using Client.WithConnectionPool.
type CookieData ΒΆ
type CookieData struct {
Name string `json:"name"`
Value string `json:"value"`
Domain string `json:"domain"`
Path string `json:"path"`
Expires time.Time `json:"expires"`
HTTPOnly bool `json:"httpOnly"`
Secure bool `json:"secure"`
}
CookieData holds the data for a cookie.
func ExportCookies ΒΆ
func ExportCookies(jar http.CookieJar, u *url.URL) []CookieData
ExportCookies prepares cookies for loading into Playwright/Chromedp.
type DNSResolver ΒΆ
DNSResolver is an interface for DNS resolution.
type Decoder ΒΆ
type Decoder interface {
// Decode reads data from the reader and unmarshals it into the target destination.
// It returns an error if reading or decoding fails, or if target is incompatible.
Decode(r io.Reader, target any) error
}
Decoder defines the contract for decoding response bodies into target structures. Implementations of this interface (such as JSONDecoder or XMLDecoder) are used by Client to deserialize HTTP response payloads.
var JSONDecoder Decoder = DecoderFunc(func(r io.Reader, target any) error { return json.NewDecoder(r).Decode(target) })
JSONDecoder parses the response body using the standard encoding/json package. The target argument must be a pointer to the destination structure.
var RawDecoder Decoder = rawDecoder{}
RawDecoder reads the entire response body and returns it as a raw byte slice. The target argument must be a non-nil pointer to a byte slice (*[]byte). It returns an error if target is not a pointer to a byte slice or if the read fails.
var XMLDecoder Decoder = DecoderFunc(func(r io.Reader, target any) error { return xml.NewDecoder(r).Decode(target) })
XMLDecoder parses the response body using the standard encoding/xml package. The target argument must be a pointer to the destination structure.
var YAMLDecoder Decoder = DecoderFunc(func(r io.Reader, target any) error { return yaml.NewDecoder(r).Decode(target) })
YAMLDecoder parses the response body using the gopkg.in/yaml.v3 package. The target argument must be a pointer to the destination structure.
type DecoderFunc ΒΆ
DecoderFunc adapts a plain function to satisfy the Decoder interface. This is useful for defining inline or lightweight custom decoders.
type DialWebSocketConfig ΒΆ
type DialWebSocketConfig struct {
// ReadBufferSize sets the gorilla WebSocket read buffer (default 4096).
ReadBufferSize int
// WriteBufferSize sets the gorilla WebSocket write buffer (default 4096).
WriteBufferSize int
}
DialWebSocketConfig holds optional configuration for DialWebSocket.
type DoHResolver ΒΆ
type DoHResolver struct {
// contains filtered or unexported fields
}
DoHResolver resolves DNS via HTTPS, supporting both A and AAAA records.
func NewDoHResolver ΒΆ
func NewDoHResolver(endpoint string) *DoHResolver
NewDoHResolver creates a DoHResolver with the given endpoint URL.
func (*DoHResolver) LookupIPAddr ΒΆ
LookupIPAddr queries both A and AAAA records via DoH.
type DoTResolver ΒΆ
type DoTResolver struct {
Server string // e.g. "1.1.1.1:853"
Hostname string // TLS SNI, e.g. "cloudflare-dns.com"
Timeout time.Duration
}
DoTResolver resolves DNS over TLS, querying both A and AAAA records. Uses github.com/miekg/dns for reliable DNS packet construction and parsing.
func NewDoTResolver ΒΆ
func NewDoTResolver(server, hostname string) *DoTResolver
NewDoTResolver creates a DoTResolver with the specified server and TLS hostname.
func (*DoTResolver) LookupIPAddr ΒΆ
LookupIPAddr queries both A and AAAA records over TLS.
type FallbackFunc ΒΆ
FallbackFunc is a function to provide fallback responses on failure.
func FallbackJSON ΒΆ
func FallbackJSON(statusCode int, data any) FallbackFunc
FallbackJSON returns a FallbackFunc that generates a mock response with a JSON payload.
type Float64String ΒΆ
type Float64String float64
Float64String handles parsing float64 values received as string representations in JSON.
func (*Float64String) UnmarshalJSON ΒΆ
func (f *Float64String) UnmarshalJSON(b []byte) error
UnmarshalJSON parses JSON byte data into the Float64String target.
type FragmentConfig ΒΆ
FragmentConfig holds the configuration for fragmentation.
type HTTP2Settings ΒΆ
HTTP2Settings holds the settings for an HTTP/2 connection.
type HTTPDoer ΒΆ
HTTPDoer defines the interface for objects executing an http.Request. It is implemented by http.Client and can be customized via DoerFunc.
func Chain ΒΆ
func Chain(doer HTTPDoer, middlewares ...Middleware) HTTPDoer
Chain nests multiple Middleware layers around an HTTPDoer handler. Middlewares are applied from left to right (the first in the slice executes first). If the middlewares slice is empty, it returns the original doer unmodified.
type HostRewriteConfig ΒΆ
HostRewriteConfig holds the configuration for host rewrite.
type InMemoryDNSCache ΒΆ
type InMemoryDNSCache struct {
// contains filtered or unexported fields
}
InMemoryDNSCache is an in-memory DNS cache that implements the DNSResolver interface.
func NewInMemoryDNSCache ΒΆ
func NewInMemoryDNSCache(ttl time.Duration, r DNSResolver) *InMemoryDNSCache
NewInMemoryDNSCache creates a new InMemoryDNSCache with the given TTL and resolver.
func (*InMemoryDNSCache) LookupIPAddr ΒΆ
LookupIPAddr looks up the IP addresses for the given host using the cache or resolver.
type Int64String ΒΆ
type Int64String int64
Int64String handles parsing int64 values received as string representations in JSON. It parses raw integers, JSON null, or empty string representations safely.
func (*Int64String) UnmarshalJSON ΒΆ
func (i *Int64String) UnmarshalJSON(b []byte) error
UnmarshalJSON parses JSON byte data into the Int64String target.
type JitterStrategy ΒΆ
type JitterStrategy int
JitterStrategy defines the algorithm used to calculate retry delay noise.
const ( // JitterEqual calculates sleep time as exponential backoff plus +/- 10% random noise. JitterEqual JitterStrategy = iota // JitterFull calculates sleep time as a random duration between 0 and current backoff. JitterFull )
type LoadBalancer ΒΆ
type LoadBalancer struct {
// contains filtered or unexported fields
}
LoadBalancer distributes requests across multiple backend servers. It implements the HTTPDoer interface and can be passed to NewClient. Use NewLoadBalancer to initialize new instances.
func NewLoadBalancer ΒΆ
func NewLoadBalancer(cfg LoadBalancerConfig, backends ...string) (*LoadBalancer, error)
NewLoadBalancer initializes a new LoadBalancer with the given backends. It returns an error if the backends slice is empty. If MaxFails or RetryAfter configuration options are zero, they default to 3 and 30 seconds respectively.
func (*LoadBalancer) Close ΒΆ
func (lb *LoadBalancer) Close() error
Close stops background health check routines.
func (*LoadBalancer) Do ΒΆ
Do executes the HTTP request across healthy backends based on the configured strategy. It clones the original request and overwrites its Scheme and Host to target the chosen backend.
If the chosen backend fails the criteria of [LoadBalancer.isFault], it is marked as failed. The load balancer then retries other available backends sequentially. If all backends fail, it returns an error wrapping the last encountered failure.
func (*LoadBalancer) Prewarm ΒΆ
func (lb *LoadBalancer) Prewarm(ctx context.Context)
Prewarm preemptively opens TCP/TLS connections to all registered backends. It executes concurrent HEAD requests to each backend URL to warm up transport pools. The method respects the provided context for cancellation.
func (*LoadBalancer) UpdateBackends ΒΆ
func (lb *LoadBalancer) UpdateBackends(backends ...string)
UpdateBackends replaces the entire set of active backends. It resets the selection counter to zero. If the backends slice is empty, the method returns early and makes no changes.
func (*LoadBalancer) WithClients ΒΆ
func (lb *LoadBalancer) WithClients(clients ...HTTPDoer) *LoadBalancer
WithClients applies custom HTTPDoer clients to the corresponding backends. The clients are matched to backends by slice index order. If the number of clients is different from backends, any excess clients are ignored.
type LoadBalancerConfig ΒΆ
type LoadBalancerConfig struct {
// Strategy determines the selection algorithm for choosing a backend.
Strategy LoadBalancingStrategy
// MaxFails is the consecutive failure threshold before a backend is marked unhealthy.
MaxFails uint32
// RetryAfter is the offline cooldown duration for an unhealthy backend.
RetryAfter time.Duration
// HealthCheckURL is the server endpoint probed by background health checks.
HealthCheckURL string
// HealthCheckInterval is the period between sequential background health checks.
HealthCheckInterval time.Duration
}
LoadBalancerConfig defines the tuning and policy settings for a LoadBalancer.
type LoadBalancingStrategy ΒΆ
type LoadBalancingStrategy int
LoadBalancingStrategy defines the selection algorithm for backends.
const ( // RoundRobin selects backends sequentially in a cyclic order. RoundRobin LoadBalancingStrategy = iota // Random selects backends randomly for each request. Random // WeightedRoundRobin selects backends based on their assigned weights. WeightedRoundRobin )
type Logger ΒΆ
type Logger interface {
Debug(msg string, args ...any)
DebugContext(ctx context.Context, msg string, args ...any)
Info(msg string, args ...any)
InfoContext(ctx context.Context, msg string, args ...any)
Warn(msg string, args ...any)
WarnContext(ctx context.Context, msg string, args ...any)
Error(msg string, args ...any)
ErrorContext(ctx context.Context, msg string, args ...any)
}
Logger is an interface for logging messages.
type Middleware ΒΆ
Middleware wraps an HTTPDoer with additional logic.
func AdaptiveLimitMiddleware ΒΆ
func AdaptiveLimitMiddleware(limiter *AdaptiveLimiter) Middleware
AdaptiveLimitMiddleware returns a Middleware wrapping an AdaptiveLimiter. It locks execution slots during connection handshakes and updates metrics on release.
func ChaosMiddleware ΒΆ
func ChaosMiddleware(cfg ChaosConfig) Middleware
ChaosMiddleware returns a Middleware that injects synthetic latency and random 503 failures. It is useful in staging or testing environments to verify retries and breaker logic.
func CircuitBreakerMiddleware ΒΆ
func CircuitBreakerMiddleware(cb *CircuitBreaker, isFailure func(*http.Response, error) bool) Middleware
CircuitBreakerMiddleware returns a Middleware wrapping requests in a CircuitBreaker. If the circuit breaker for a host is in the open state, it returns an error instantly. Successful and failed attempts are recorded dynamically to manage the host's breaker state.
func FallbackMiddleware ΒΆ
func FallbackMiddleware() Middleware
FallbackMiddleware executes registered fallbacks on request failure.
func FallbackMiddlewareEx ΒΆ
func FallbackMiddlewareEx(isFailure func(*http.Response, error) bool) Middleware
FallbackMiddlewareEx executes fallbacks on custom failure conditions.
func RateLimitMiddleware ΒΆ
func RateLimitMiddleware(rps float64, burst int) Middleware
RateLimitMiddleware returns a Middleware that limits request rates. It uses a token bucket algorithm based on requests per second (rps) and burst limits.
func RecoveryMiddleware ΒΆ
func RecoveryMiddleware(onPanic func(any)) Middleware
RecoveryMiddleware catches panics occurring during request execution. It executes the optional onPanic callback and wraps the panic into a standard Go error.
func RetryMiddleware ΒΆ
func RetryMiddleware(opts RetryOptions, condition RetryCondition) Middleware
RetryMiddleware returns a Middleware that retries failed requests based on a condition. It fully reads and buffers the request body into memory to allow replay attempts.
The middleware automatically honors any standard "Retry-After" header received from the server. If no such header is found, it applies exponential backoff with randomized jitter.
type NoResponse ΒΆ
type NoResponse struct{}
NoResponse is a sentinel type used to indicate a request that does not return a response body.
type ProgressFunc ΒΆ
type ProgressFunc func(current, total int64)
ProgressFunc defines the callback signature for tracking transfer progress. The total parameter represents the content length, or -1 if unknown.
type ProxyConfig ΒΆ
type ProxyConfig struct {
// ProxyURL is the raw address of the proxy server (e.g. http://user:pass@ip:port).
ProxyURL string
// Timeout is the overall request timeout.
Timeout time.Duration
// InsecureSkipVerify controls whether SSL/TLS certificate verification is bypassed.
InsecureSkipVerify bool
// Transport is a custom round tripper that overrides the default transport settings.
Transport http.RoundTripper
// TransportFactory is a custom factory function used to initialize a new round tripper.
TransportFactory func(ProxyConfig) (http.RoundTripper, error)
}
ProxyConfig defines the configuration parameters for a proxy-supported client.
type ProxyIsolatedCookieJar ΒΆ
type ProxyIsolatedCookieJar struct {
// contains filtered or unexported fields
}
ProxyIsolatedCookieJar is an isolated cookie jar that stores cookies per proxy URL.
func NewProxyIsolatedCookieJar ΒΆ
func NewProxyIsolatedCookieJar() *ProxyIsolatedCookieJar
NewProxyIsolatedCookieJar creates a new ProxyIsolatedCookieJar.
func (*ProxyIsolatedCookieJar) Cookies ΒΆ
func (p *ProxyIsolatedCookieJar) Cookies(u *url.URL) []*http.Cookie
Cookies implements the http.CookieJar interface (as a fallback for standard calls).
func (*ProxyIsolatedCookieJar) SetCookies ΒΆ
func (p *ProxyIsolatedCookieJar) SetCookies(u *url.URL, cookies []*http.Cookie)
SetCookies implements the http.CookieJar interface (as a fallback for standard calls).
type ProxyRotator ΒΆ
type ProxyRotator struct {
// contains filtered or unexported fields
}
ProxyRotator distributes HTTP requests across a pool of proxy clients. It implements the HTTPDoer interface and can be passed to NewClient.
Use NewProxyRotator to initialize new instances. It supports sticky routing, health monitoring, and dynamic pool replacement.
func NewProxyRotator ΒΆ
func NewProxyRotator(config ProxyRotatorConfig, clients ...ClientWithProxy) (*ProxyRotator, error)
NewProxyRotator initializes a new ProxyRotator instance. It returns an error if the clients slice is empty. If MaxFails or RetryAfter configuration options are zero, they default to 3 and 30 seconds respectively.
func (*ProxyRotator) Close ΒΆ
func (r *ProxyRotator) Close() error
Close stops background health check and session cleanup routines.
func (*ProxyRotator) Do ΒΆ
Do performs an HTTP request using the next available client in the rotation pool.
It attempts sticky routing first. If the sticky client is offline or fails, it falls back to standard round-robin selection.
On proxy connection faults, the client is flagged and marked as failed. If all clients fail or are marked unhealthy, Do returns an error.
func (*ProxyRotator) Prewarm ΒΆ
func (r *ProxyRotator) Prewarm(ctx context.Context, targetURL string)
Prewarm preemptively opens TCP/TLS connections to targetURL through all proxy clients. It sends concurrent HEAD requests to the target URL to pre-populate transport connection pools.
func (*ProxyRotator) UpdateClients ΒΆ
func (r *ProxyRotator) UpdateClients(clients ...ClientWithProxy)
UpdateClients replaces the active proxy clients with a new pool. It resets all session-to-proxy mappings to prevent indexing errors. If the clients slice is empty, the method returns early and makes no changes.
func (*ProxyRotator) WithStickySessions ΒΆ
func (r *ProxyRotator) WithStickySessions(f StickyKeyFunc) *ProxyRotator
WithStickySessions enables session persistence using the provided key extractor. It returns a copy of the ProxyRotator configured with the sticky function.
type ProxyRotatorConfig ΒΆ
type ProxyRotatorConfig struct {
// MaxFails is the consecutive error limit before a client is marked unhealthy.
MaxFails uint32
// RetryAfter is the duration for which an unhealthy client is kept offline.
RetryAfter time.Duration
// HealthCheckURL is the endpoint probed by background health checks.
HealthCheckURL string
// HealthCheckInterval is the execution period of background health checks.
HealthCheckInterval time.Duration
}
ProxyRotatorConfig defines health-checking and recovery limits for a ProxyRotator.
type ReplayableBody ΒΆ
type ReplayableBody interface {
io.ReadCloser
Reset()
}
ReplayableBody represents a response stream that can be reset to the beginning for re-reading (for example, after previewing or logging).
func AsReplayable ΒΆ
func AsReplayable(rc io.ReadCloser) ReplayableBody
AsReplayable turns any io.ReadCloser into a ReplayableBody. If there's already a high-performance buffer (multiReadBody) under the hood, it will return it. If not, it will transparently create a lightweight in-memory buffer for repeated reading.
type RequestModifier ΒΆ
RequestModifier represents a function that alters an http.Request before execution.
func AsCurl ΒΆ
func AsCurl() RequestModifier
AsCurl returns a RequestModifier that dumps the equivalent curl command to a silent discard stream.
func AsJSON ΒΆ
func AsJSON() RequestModifier
AsJSON returns a RequestModifier that configures the client to use JSONDecoder. Clients use this modifier by default when no other decoder is specified.
func AsRaw ΒΆ
func AsRaw() RequestModifier
AsRaw returns a RequestModifier that configures the client to use RawDecoder.
func AsXML ΒΆ
func AsXML() RequestModifier
AsXML returns a RequestModifier that configures the client to use XMLDecoder.
func AsYAML ΒΆ
func AsYAML() RequestModifier
AsYAML returns a RequestModifier that configures the client to use YAMLDecoder.
func CaptureResponse ΒΆ
func CaptureResponse(target **http.Response) RequestModifier
CaptureResponse captures the resulting http.Response structure upon request completion.
func Debug ΒΆ
func Debug() RequestModifier
Debug returns a modifier that forces verbose logging of request and response details.
func Trace ΒΆ
func Trace(target *TraceInfo) RequestModifier
Trace returns a RequestModifier that registers a connection tracer on the active request. Timing metrics are recorded and populated inside the provided TraceInfo structure.
func TraceJA4 ΒΆ
func TraceJA4(target *TraceInfo) RequestModifier
TraceJA4 returns a RequestModifier that populates the JA4 field of the provided TraceInfo. It sets up a shared store in the request context so that Client.WithTLSFingerprint can write the TLS fingerprint during the handshake, and computes the HTTP fingerprint from request headers.
The JA4 report is fully populated after the request completes. The TLS fingerprint (JA4) requires Client.WithTLSFingerprint to be enabled.
Use this modifier alongside Trace for complete timing + fingerprint data:
info := &aoni.TraceInfo{}
client.Get(ctx, "/path", aoni.Trace(info), aoni.TraceJA4(info))
// After request: info.JA4 contains both JA4 and JA4H
func WithALPN ΒΆ
func WithALPN(protocols []string) RequestModifier
WithALPN returns a RequestModifier that sets custom ALPN protocols.
func WithAccept ΒΆ
func WithAccept(accept string) RequestModifier
WithAccept overrides the standard Accept header field.
func WithBasicAuth ΒΆ
func WithBasicAuth(username, password string) RequestModifier
WithBasicAuth applies Basic Authorization credentials.
func WithBearer ΒΆ
func WithBearer(token string) RequestModifier
WithBearer applies a Bearer Token authorization header.
func WithBody ΒΆ
func WithBody(r io.Reader) RequestModifier
WithBody replaces the request body stream with the provided reader.
func WithContentType ΒΆ
func WithContentType(ct string) RequestModifier
WithContentType overrides the standard Content-Type header field.
func WithCookie ΒΆ
func WithCookie(c *http.Cookie) RequestModifier
WithCookie attaches a single cookie to the request.
func WithCookies ΒΆ
func WithCookies(kv map[string]string) RequestModifier
WithCookies attaches multiple cookies from a key-value map.
func WithDecoder ΒΆ
func WithDecoder(d Decoder) RequestModifier
WithDecoder sets the response body Decoder strategy for the active request.
func WithDownloadProgress ΒΆ
func WithDownloadProgress(onProgress ProgressFunc) RequestModifier
WithDownloadProgress registers a progress callback for monitoring server downloads.
func WithErrorModel ΒΆ
func WithErrorModel(target any) RequestModifier
WithErrorModel configures a target structure to parse non-2xx API error responses.
func WithFallback ΒΆ
func WithFallback(f FallbackFunc) RequestModifier
WithFallback returns a RequestModifier registering a FallbackFunc for a single request.
func WithForceHTTP1 ΒΆ
func WithForceHTTP1() RequestModifier
WithForceHTTP1 returns a RequestModifier that forces ALPN to only http/1.1.
func WithForceHTTP2 ΒΆ
func WithForceHTTP2() RequestModifier
WithForceHTTP2 returns a RequestModifier that forces ALPN to only h2.
func WithFragmentation ΒΆ
func WithFragmentation(cfg FragmentConfig) RequestModifier
WithFragmentation returns a RequestModifier that sets fragmentation configuration on the request context.
func WithHeader ΒΆ
func WithHeader(key, value string) RequestModifier
WithHeader sets the key header field to the given value.
func WithHedging ΒΆ
func WithHedging(delay time.Duration) RequestModifier
WithHedging applies a request-specific hedging timeout.
func WithHostRewrite ΒΆ
func WithHostRewrite(rules map[string]string) RequestModifier
WithHostRewrite returns a RequestModifier that rewrites the host header based on the provided rules.
func WithJSONBody ΒΆ
func WithJSONBody(payload any) RequestModifier
WithJSONBody serializes the payload as JSON and configures it as the request body. It sets the Content-Type header to "application/json". Any marshaling errors are written to the request context via bodyErrorCtxKey.
func WithMultiReadBody ΒΆ
func WithMultiReadBody(threshold int64) RequestModifier
WithMultiReadBody returns a RequestModifier setting the body re-readability threshold. Passing a threshold <= 0 disables body caching for the request.
func WithMultipart ΒΆ
WithMultipart writes the provided text fields and files into a multipart/form-data body. It automatically computes and sets the Content-Type header with the boundary value. Any write errors during formatting cause the modifier to return early with an incomplete body.
func WithOrderedHeaders ΒΆ
func WithOrderedHeaders(order []string) RequestModifier
WithOrderedHeaders specifies the exact order of headers for HTTP/1.1 requests.
func WithOrigin ΒΆ
func WithOrigin(origin string) RequestModifier
WithOrigin overrides the standard Origin header field.
func WithP0fSignature ΒΆ
func WithP0fSignature(sig *p0f.Signature) RequestModifier
WithP0fSignature returns a RequestModifier that stores a p0f signature in the request context. When used with Client.WithP0fSignature, the TCP/IP fields (TTL, DF, window size) are spoofed to match the specified OS.
func WithQuery ΒΆ
func WithQuery(query any) RequestModifier
WithQuery encodes a struct or map as URL query parameters and appends them to the request URL. It safely checks for validation errors using Validate and serialization failures. Any encountered errors are saved to the request context via queryErrorCtxKey.
func WithStreamingMultipart ΒΆ
WithStreamingMultipart applies a streaming multipart request body.
func WithUploadProgress ΒΆ
func WithUploadProgress(onProgress ProgressFunc) RequestModifier
WithUploadProgress registers a progress callback for monitoring client uploads.
func WithUserAgent ΒΆ
func WithUserAgent(ua string) RequestModifier
WithUserAgent overrides the standard User-Agent header field.
func WithVar ΒΆ
func WithVar(key string, value any) RequestModifier
WithVar replaces a single placeholder (e.g. "{key}") in the path with an escaped value.
func WithVars ΒΆ
func WithVars(pairs ...any) RequestModifier
WithVars replaces multiple placeholder keys in the path with their respective values. It accepts alternating key-value arguments. If the argument list has an odd length, it returns early and performs no replacements.
type Requester ΒΆ
type Requester interface {
Request(
ctx context.Context,
method, path string,
mods ...RequestModifier,
) (*http.Response, error)
}
Requester defines the contract for executing raw HTTP requests. It supports relative paths joining, query encoding, and custom modifiers.
type RetryCondition ΒΆ
RetryCondition is a function that determines whether a request should be retried. It receives the response and error from the failed attempt.
func ProxyRetryCondition ΒΆ
func ProxyRetryCondition(rotator *ProxyRotator) RetryCondition
ProxyRetryCondition returns a RetryCondition that triggers retries on proxy failures.
func RetryOnErr ΒΆ
func RetryOnErr() RetryCondition
RetryOnErr returns a RetryCondition that retries requests based on a nil error condition.
func RetryOnGatewayErrors ΒΆ
func RetryOnGatewayErrors() RetryCondition
RetryOnGatewayErrors returns a RetryCondition that retries requests based on gateway errors (HTTP 502, 503, 504).
func RetryOnRateLimit ΒΆ
func RetryOnRateLimit() RetryCondition
RetryOnRateLimit returns a RetryCondition that retries requests based on rate limit errors (HTTP 429).
func RetryOnTransientErrors ΒΆ
func RetryOnTransientErrors() RetryCondition
RetryOnTransientErrors returns a RetryCondition that retries requests based on transient errors.
type RetryOptions ΒΆ
type RetryOptions struct {
// MaxRetries is the maximum retry limit before giving up.
MaxRetries uint32
// Backoff is the initial delay duration applied before the first retry attempt.
Backoff time.Duration
// JitterStrategy is the delay noise calculation algorithm.
JitterStrategy JitterStrategy
}
RetryOptions defines retry configuration parameters for RetryMiddleware.
type SSEEvent ΒΆ
type SSEEvent struct {
// Event is the event identifier string.
Event string
// Data is the data payload buffer string.
Data string
// ID is the unique event tracking ID.
ID string
// Retry is the reconnection timeout value in milliseconds.
Retry int
}
SSEEvent represents a parsed single Server-Sent Event frame.
type SourceIPRotator ΒΆ
type SourceIPRotator struct {
// contains filtered or unexported fields
}
SourceIPRotator is a struct that holds a pool of source IP addresses and rotates between them.
func NewSourceIPRotator ΒΆ
func NewSourceIPRotator(addrs []string) (*SourceIPRotator, error)
NewSourceIPRotator creates a new SourceIPRotator with the given IP addresses.
func (*SourceIPRotator) Next ΒΆ
func (r *SourceIPRotator) Next() net.IP
Next returns the next IP address in the pool and rotates to the next one.
type StdlibResolver ΒΆ
StdlibResolver wraps Go's standard net.Resolver to implement DNSResolver. This is the default resolver used when no custom resolver is configured.
func NewStdlibResolver ΒΆ
func NewStdlibResolver() *StdlibResolver
NewStdlibResolver creates a StdlibResolver with the default resolver.
func (*StdlibResolver) LookupIPAddr ΒΆ
LookupIPAddr delegates to the underlying net.Resolver.
type StickyKeyFunc ΒΆ
StickyKeyFunc extracts a session identifier from a request to support sticky routing. Returning an empty string falls back to default round-robin rotation.
type StreamResponse ΒΆ
type StreamResponse struct {
// contains filtered or unexported fields
}
StreamResponse wraps an http.Response and manages connection reading streams. Callers are responsible for calling StreamResponse.Close after read operations complete.
func Stream ΒΆ
func Stream( ctx context.Context, c Requester, path string, mods ...RequestModifier, ) (*StreamResponse, error)
Stream executes a GET request and returns the resulting connection body as StreamResponse. Callers must ensure the returned stream is closed when done.
func StreamWithBody ΒΆ
func StreamWithBody( ctx context.Context, c Requester, method, path string, body io.Reader, mods ...RequestModifier, ) (*StreamResponse, error)
StreamWithBody executes an HTTP request with the provided body and returns a raw StreamResponse.
func (*StreamResponse) Close ΒΆ
func (s *StreamResponse) Close() error
Close closes the underlying network response body stream.
func (*StreamResponse) ContentLength ΒΆ
func (s *StreamResponse) ContentLength() int64
ContentLength returns the response body content length, or -1 if unknown.
func (*StreamResponse) ContentType ΒΆ
func (s *StreamResponse) ContentType() string
ContentType returns the Content-Type header field value.
func (*StreamResponse) Read ΒΆ
func (s *StreamResponse) Read(p []byte) (n int, err error)
Read reads connection body data into p.
func (*StreamResponse) Response ΒΆ
func (s *StreamResponse) Response() *http.Response
Response returns the underlying raw http.Response structure.
func (*StreamResponse) StatusCode ΒΆ
func (s *StreamResponse) StatusCode() int
StatusCode returns the HTTP status code of the response.
type TraceInfo ΒΆ
type TraceInfo struct {
// DNSLookup is the duration spent resolving the server's IP address.
DNSLookup time.Duration
// TCPConn is the duration spent establishing the TCP connection.
TCPConn time.Duration
// TLSHandshake is the duration spent completing the SSL/TLS handshake.
TLSHandshake time.Duration
// ServerProcessing is the duration from connection establishment to receiving the first response byte.
ServerProcessing time.Duration
// ContentTransfer is the duration spent transferring the response body data.
ContentTransfer time.Duration
// Total is the total execution time for the request.
Total time.Duration
// RequestSize is the request payload size in bytes.
RequestSize int64
// ResponseSize is the response payload size in bytes.
ResponseSize int64
// JA4 holds the JA4+ fingerprints computed during the request.
// Populated only when [TraceJA4] is used as a request modifier.
JA4 *ja4.Report
}
TraceInfo holds detailed network layer timing metrics for a request. The timing fields are fully populated only after the response body is completely read.
type Uint64String ΒΆ
type Uint64String uint64
Uint64String handles parsing uint64 values received as string representations in JSON. It parses raw integers, JSON null, or empty string representations safely.
func (*Uint64String) UnmarshalJSON ΒΆ
func (u *Uint64String) UnmarshalJSON(b []byte) error
UnmarshalJSON parses JSON byte data into the Uint64String target.
type UnixTimestamp ΒΆ
UnixTimestamp handles Unix timestamps that Steam sends as strings or numbers.
func (*UnixTimestamp) Time ΒΆ
func (t *UnixTimestamp) Time() time.Time
Time returns the underlying time.Time.
func (*UnixTimestamp) UnmarshalJSON ΒΆ
func (t *UnixTimestamp) UnmarshalJSON(b []byte) error
UnmarshalJSON implements json.Unmarshaler.
type Unwrapper ΒΆ
type Unwrapper interface {
Unwrap() Requester
}
Unwrapper is an interface for objects that can be unwrapped to reveal their underlying Requester implementation.
type ValidationError ΒΆ
type ValidationError struct {
// Field is the name of the missing or invalid struct field.
Field string
}
ValidationError is returned when a request structure fails validation. Inspect this error using errors.As to determine which field triggered the validation failure.
func (*ValidationError) Error ΒΆ
func (e *ValidationError) Error() string
Source Files
ΒΆ
Directories
ΒΆ
| Path | Synopsis |
|---|---|
|
examples
|
|
|
01_basic_get
command
Example: Basic GET request with typed JSON response.
|
Example: Basic GET request with typed JSON response. |
|
02_path_and_query
command
Example: Path variables and query parameters.
|
Example: Path variables and query parameters. |
|
03_post_json
command
Example: POST with JSON body and custom error model.
|
Example: POST with JSON body and custom error model. |
|
04_headers_auth
command
Example: Bearer token, Basic auth, and custom headers.
|
Example: Bearer token, Basic auth, and custom headers. |
|
05_error_handling
command
Example: Error handling with APIError, ErrCloudflareChallenge, and status codes.
|
Example: Error handling with APIError, ErrCloudflareChallenge, and status codes. |
|
06_proxy_rotation
command
Example: Proxy rotation with sticky sessions and retry middleware.
|
Example: Proxy rotation with sticky sessions and retry middleware. |
|
07_retry_middleware
command
Example: Retry middleware with RetryOptions and retry conditions.
|
Example: Retry middleware with RetryOptions and retry conditions. |
|
08_circuit_breaker
command
Example: Circuit breaker middleware.
|
Example: Circuit breaker middleware. |
|
09_tracing
command
Example: Request tracing and curl command generation.
|
Example: Request tracing and curl command generation. |
|
10_streaming
command
Example: Streaming responses with Stream, StreamSSE, and StreamNDJSON.
|
Example: Streaming responses with Stream, StreamSSE, and StreamNDJSON. |
|
11_ja4_fingerprint
command
Example: JA4 fingerprinting with TLS impersonation.
|
Example: JA4 fingerprinting with TLS impersonation. |
|
12_websocket
command
Example: WebSocket connection and binary message round-trip.
|
Example: WebSocket connection and binary message round-trip. |
|
13_tls_evasion
command
Example: Full TLS evasion stack.
|
Example: Full TLS evasion stack. |
|
14_load_balancer
command
Example: Load balancer with weighted round-robin strategy.
|
Example: Load balancer with weighted round-robin strategy. |
|
15_browser_impersonation
command
Example: Browser impersonation with Chrome and Firefox profiles.
|
Example: Browser impersonation with Chrome and Firefox profiles. |
|
16_p0f_fingerprint
command
Example: p0f fingerprinting
|
Example: p0f fingerprinting |
|
Package ja4 implements JA4+ network fingerprinting algorithms in pure Go.
|
Package ja4 implements JA4+ network fingerprinting algorithms in pure Go. |