requestsutls

README

requests-utls

Go engine for profile-driven TLS, concurrent Sessions, and ordered HTTP/1.1 and HTTP/2 headers, including repeated fields. The separate Python client bundles this engine through CFFI; this repository owns the Go API and native ABI.

Quick start

Requires Go 1.26+; native libraries and race tests also require a C compiler. Run from the repository root:

make check
go run ./cmd/requests-utls validate -profile profiles/chrome_152.json
make shared

See the Go API example and native ABI for embedding the engine.

Behavior

• Immutable TLS profiles, request-level header order, and concurrent Sessions.
• HTTP/2 and HTTP/1.1 with ALPN fallback, connection reuse and TLS resumption.
• Authenticated HTTP CONNECT proxies and certificate verification by default.
• Bounded buffered responses with gzip, deflate, Brotli and Zstandard decoding.
• No automatic redirects or Cookie state management. Responses retain Location and Set-Cookie; callers decide subsequent requests and Cookie updates. Connection pooling, cancellation and cleanup remain engine responsibilities.

A matching fingerprint does not imply complete browser behavior. Some profile extensions are advertisement-only; see profile capabilities. Streaming, HTTP/3 and HTTPS/SOCKS proxy connections are not supported.

Documentation

Task	Guide
Go API, header order, profiles and CLI	Usage
Bind another language	ABI, native artifacts
Understand validation evidence	Verification
Develop or maintain the engine	Contributing, maintenance
Review changes or report a vulnerability	Changelog, security

Documentation

Overview

Package requestsutls provides a concurrent, profile-driven HTTP client. This Go prototype deliberately has no mutable default headers or cookie jar.

Index

• Variables
• type HeaderField
• type Options
• type ProxyAuth
• type Request
• type Response
• type Session
  • func NewSession(o Options) (*Session, error)
  • func (s *Session) Close() error
  • func (s *Session) Do(ctx context.Context, input Request) (*Response, error)
  • func (s *Session) ProfileHash() string
• type StageError
  • func (e *StageError) Error() string
  • func (e *StageError) Unwrap() error

Variables

var (
	ErrSessionClosed    = errors.New("requests-utls: session closed")
	ErrQueueFull        = errors.New("requests-utls: request queue full")
	ErrResponseTooLarge = errors.New("requests-utls: response body exceeds limit")
	ErrInvalidRequest   = errors.New("requests-utls: invalid request")
)

Types

type HeaderField

type HeaderField struct {
	Name  string `json:"name"`
	Value string `json:"value"`
}

HeaderField is one field occurrence. Slices retain duplicates and their order.

type Options

type Options struct {
	// Connect/TLS/Proxy timeouts default to 10s when zero. Header/body zero disables
	// the phase limit. The caller context always bounds the whole operation.
	ConnectTimeout           time.Duration
	ProxyConnectTimeout      time.Duration
	TLSHandshakeTimeout      time.Duration
	ResponseHeaderTimeout    time.Duration
	BodyTimeout              time.Duration
	Profile                  *profile.Profile
	ProxyURL                 string     // Explicit http:// CONNECT proxy; empty means direct.
	ProxyAuth                *ProxyAuth // Optional separate Basic credentials; cannot combine with URL userinfo.
	RootCAs                  *x509.CertPool
	InsecureSkipVerify       bool
	DisableSessionResumption bool  // Default false: cache TLS tickets within this Session.
	ForceHTTP1               bool  // Advertise only HTTP/1.1 and use its ordered wire transport.
	RandomJA3                bool  // Shuffle eligible TLS extensions for each new connection.
	DisableContentDecoding   bool  // Return the compressed response bytes unchanged.
	MaxConcurrentRequests    int   // 0 means 64; includes body consumption.
	MaxPendingRequests       int   // 0 means no waiting queue.
	MaxResponseBytes         int64 // 0 means 32 MiB.
	MaxUnprocessedRetries    int   // 0 means 3; -1 disables; maximum 32. Only proven-unprocessed requests.
}

Options are snapshotted by NewSession. A Session's transport never changes.

type ProxyAuth

type ProxyAuth struct {
	Username string
	Password string
}

ProxyAuth is snapshotted when the Session is created. Credentials are sent only to the proxy in the CONNECT request, never as origin request headers.

type Request

type Request struct {
	Method  string        `json:"method"`
	URL     string        `json:"url"`
	Headers []HeaderField `json:"headers"`
	// HeadersOrder matches regular header names case-insensitively. A name listed once
	// groups all its values; repeated names specify each occurrence's position.
	// For a present name, repeated entries must match its number of values.
	// Absent names are ignored, unlisted fields follow in their original order,
	// and an empty list preserves field order. Generated fields also participate.
	// Content-Length is calculated from Body. HTTP/1 retains field casing;
	// HTTP/2 lowercases wire names. Pseudo-header order is part of the profile.
	HeadersOrder []string `json:"headers_order,omitempty"`
	Body         []byte   `json:"body,omitempty"`
}

Request owns the input for a single operation. Do snapshots its slices; callers must not mutate them concurrently with Do taking that snapshot.

type Response

type Response struct {
	StatusCode int           `json:"status_code"`
	Headers    []HeaderField `json:"headers"`
	Body       []byte        `json:"body"`
	Protocol   string        `json:"protocol"`
	Decoded    bool          `json:"decoded"`
}

Response contains the final response's ordered regular fields and body. Content decoding is enabled by default; Decoded identifies transformed bodies while Headers always retain their original wire values. Redirects are not followed.

type Session

type Session struct {
	// contains filtered or unexported fields
}

Session is safe for concurrent Do and Close calls. Create another Session to change the profile or trust configuration; connections cannot cross Sessions.

func NewSession

func NewSession(o Options) (*Session, error)

func (*Session) Close

func (s *Session) Close() error

Close cancels all admitted requests, closes active/idle connections and waits for Do calls to release resources. Concurrent/repeated calls are idempotent.

func (*Session) Do

func (s *Session) Do(ctx context.Context, input Request) (*Response, error)

func (*Session) ProfileHash

func (s *Session) ProfileHash() string

ProfileHash identifies the immutable configuration used for this Session.

type StageError

type StageError struct {
	Stage   string
	Elapsed time.Duration
	Err     error
}

StageError identifies the operation that failed. Unwrap preserves cancellation, deadline and size-limit classification. The underlying error remains inspectable.

func (*StageError) Error

func (e *StageError) Error() string

func (*StageError) Unwrap

func (e *StageError) Unwrap() error