headless

README

headless-client

headless-client is a Go library. It makes a Go client send the same network fingerprint as Chrome.

A Go client has a fingerprint of its own, from the standard library and from the packages it builds on:

• crypto/tls sends a Go TLS ClientHello.
• x/net/http2 sends a Go HTTP/2 SETTINGS frame and sorts headers alphabetically.
• pion sends a pion DTLS ClientHello.
• net.Dialer enables a TCP keepalive timer.

Any of these identifies the client as a Go process. This library replaces them with the Chrome equivalents.

Status

The library is in development. The API is unstable and changes without notice. One profile is available: Chrome 151 on Windows.

Installation

Go 1.26.1 or later is required.

go get github.com/kulikov0/headless-client

import "github.com/kulikov0/headless-client"

The package is named headless.

Use cases

• Network security research. Measure which surfaces let a passive observer distinguish a browser from a non-browser client.
• Crawling and scraping. The HTTP part covers JA3 and JA4, HTTP/2 SETTINGS, header order, the priority header and connection reuse, and it works without the WebRTC part.
• Testing services that handle non-browser clients differently.

What it covers

• TLS: Chrome ClientHello through utls, post-quantum signature algorithms, session resumption with the pre_shared_key extension.
• HTTP/1.1 and HTTP/2: header order per request destination, Chrome SETTINGS and window sizes, pseudo-header order, the RFC 9218 priority header, a connection pool with Chrome's per-host limit and idle timeout.
• Headers: user agent, client hints, Accept, Accept-Encoding, Sec-Fetch-*.
• WebSocket: the Chrome upgrade handshake.
• WebRTC: DTLS ClientHello extension shuffling, optional DTLS 1.3 mimicry, ServerHello extension order on both DTLS versions, no HelloVerifyRequest, handshake fragments sized so the datagram fills the MTU, SRTP profile order, RTP header extension set, identifiers and order, ICE credential shape, ICE keepalive interval.
• TCP: keepalive disabled, as in Chrome.

Usage

HTTP

HTTPClient returns an http.Client with the Chrome TLS, HTTP/2 and connection pool settings applied.

client := headless.ChromeWindows.HTTPClient()

Every call with the same profile returns a client backed by the same transport, so connections are pooled across call sites. client.CloseIdleConnections() drops that pool.

Headers returns the header set for a request destination.

request.Header = headless.ChromeWindows.Headers(headless.DestEmpty)

The destinations are DestDocument, DestScript, DestEmpty and DestWebSocket. The destination selects the Accept value, the Sec-Fetch-* values and the priority value.

Transport returns a new transport on every call and takes dial options.

transport := headless.ChromeWindows.Transport(headless.TLSOptions{
	DialContext: proxyDialer.DialContext,
	ServerName:  "example.com",
})

TLSOptions has three fields. DialContext replaces the TCP dial, which is where a proxy goes. ServerName overrides the SNI value. InsecureSkipVerify disables certificate verification.

HTTPClient takes no options because it keys its shared transport on the profile value, and a function field cannot be part of a map key. Use Transport when options are needed.

WebSocket

dialer := headless.ChromeWindows.WebSocketDialer(headless.TLSOptions{})

The dialer ignores the HTTP_PROXY and HTTPS_PROXY environment variables. Pass a proxy through TLSOptions.DialContext.

WebRTC

SettingEngine returns a webrtc.SettingEngine with the profile applied.

settingEngine, err := headless.ChromeWindows.SettingEngine()
if err != nil {
	return err
}
settingEngine.DetachDataChannels()

api := webrtc.NewAPI(webrtc.WithSettingEngine(settingEngine))

Apply caller settings after the call. A pion setting applied later overwrites an earlier one.

RegisterHeaderExtensions registers Chrome's RTP header extensions on a webrtc.MediaEngine. pion registers none by default.

mediaEngine := &webrtc.MediaEngine{}
if err := mediaEngine.RegisterDefaultCodecs(); err != nil {
	return err
}
if err := headless.ChromeWindows.RegisterHeaderExtensions(mediaEngine); err != nil {
	return err
}

api := webrtc.NewAPI(
	webrtc.WithSettingEngine(settingEngine),
	webrtc.WithMediaEngine(mediaEngine),
)

The method registers four extensions for audio and thirteen for video. The identifiers and the order of the a=extmap lines match Chrome. They hold across renegotiation.

Interceptor helpers such as RegisterDefaultInterceptors register some of the same extensions. Registration is keyed on the URI, so the second registration reuses the first entry.

Profiles

headless.ChromeWindows is the only profile. Each builder returns a copy with one part changed and leaves the receiver alone.

profile := headless.ChromeWindows.
	WithAcceptLanguage("en-US,en;q=0.9").
	WithDTLS13Mimicry()

• WithDTLS13Mimicry is for peers that negotiate DTLS 1.3. It replaces the cipher suite list with Chrome's, keeps only the key shares Chrome offers, sets Chrome's signature algorithms, and adds psk_key_exchange_modes when pion leaves it out. The extensions are still shuffled.
• WithDTLSGREASE adds two RFC 8701 GREASE extensions to the DTLS ClientHello, an empty one first and a one byte one last. It is off by default because Chrome sends neither. Use it only against a rule that matches on the extension set.
• WithClientHelloID selects a different utls parrot.
• WithUserAgent and WithAcceptLanguage replace those header values.
• WithName sets the string returned by Name.

UserAgent and ClientHelloID return the current values.

A Profile is a comparable value. Two profiles with the same fields share one connection pool.

TLS key log

If SSLKEYLOGFILE is set, the library appends TLS session keys to that file, so Wireshark can decrypt the capture. The file is opened once per process.

Method

Values come from Chromium and libwebrtc source. Packet captures verify them.

A value copied from a capture, such as a JA3 string or a sec-ch-ua header, describes one Chrome build. The next Chrome release changes it. This library ports the code that produces the value. Changing measuredChromeMajorVersion in profile.go regenerates the sec-ch-ua brand list. The priority values are derived from the blink-to-net priority chain, so they cover request destinations that were never captured.

The TLS ClientHello starts from the utls Chrome 133 parrot, the newest Chrome parrot utls ships. Chrome 151 sends three signature algorithms that the parrot does not, 0x0904, 0x0905 and 0x0906, so the library prepends them. The rest of the ClientHello needs no patch. Against a Chrome 151 capture the extension set, the groups and the cipher list match, and the JA4 fingerprints are equal.

libwebrtc calls SSL_CTX_set_permute_extensions on its SSL context. BoringSSL then permutes the extension table when it builds a ClientHello, and does not consult the permutation when it builds a ServerHello. This library shuffles the ClientHello extensions on every handshake and sends a fixed order in the ServerHello. Measured: 13 ClientHellos with 13 distinct orders, 12 ServerHellos with one order.

libwebrtc does not call SSL_CTX_set_grease_enabled. Chrome sends no GREASE extension in DTLS, so WithDTLSGREASE is off by default.

The stand directory contains a Docker capture stand. It runs a pinned Chromium and a client built with this library in separate network namespaces, and diffs their traffic. A page with two RTCPeerConnection objects connected to each other produces a Chrome DTLS handshake in the browser loopback capture. It needs no call and no account.

Tests

go test ./...

The tests check the values the library produces.

• The client hint tests reproduce Chromium's brand generator across versions.
• The header tests check the order and the values for each request destination.
• The DTLS tests run the ClientHello hook over a pion ClientHello and read back the extension order and the cipher suite list. One test builds a real pion client against a socket that never answers and checks the extension set the mimicry produces.
• The ICE test builds a peer connection and reads the credential lengths out of the offer.
• The transport tests run against a local server and check connection reuse.

Every vendored patch has a guard test that fails when a regeneration loses it. internal/chromehttp2 keeps its guards inside the tree, and chromehttp2.sh copies them back after each run. The guards for internal/dtls, internal/ice and websocket are in the root package, because those scripts delete every test file in the tree they rewrite.

Known gaps

The following gaps are scheduled. Gaps that will not be addressed are under Out of scope.

HTTP

• HTTP/3 and QUIC are not implemented.
• Accept-Language is fixed to ru-RU. Chrome reads this value from a per-locale resource. It does not derive the value, so a table is required.
• sec-ch-ua-platform is the only client hint value that was not read from Chromium source. The code branch that produces it is confirmed. The Windows spelling is not confirmed.

WebRTC

• The SDP has pion's shape. The CNAME is derived from the stream ID, where Chrome uses a random value. The codec set and payload types are pion defaults. The attribute order is pion's. A server that reads the offer can detect all of this.
• No STUN keepalive is sent to the STUN server. Chrome sends one every 10 s in addition to the peer keepalive. Over a 170 s capture this library sent two packets to its STUN servers, both during gathering, and nothing after.
• RTCP feedback format and cadence have not been audited.
• The ICE candidate priority is one number that packs the candidate type, a local preference and the component. pion always writes 65535 as the local preference, so a host candidate gets 2130706431 where the reference capture shows 2122260223. A server that reads the offer sees it. Anyone reading a STUN binding request sees it as well. Chrome's local preference there was 32542. Where that value comes from was not established, so there is no target to patch pion to yet.

Behavior

• The profile reports Windows. The TCP and IP layers report the real host OS. Run the client on the operating system that the profile names. Profiles for other platforms are planned.
• The client sends no speculative traffic. Chrome preconnects, prefetches, and requests favicons and revocation lists. The planned fix is to replay a real page load, using the request destinations and priorities in this library.

Out of scope

• Headers added by the caller sort after the Chrome headers. Chrome's order for an arbitrary header set follows a hash bucket order, which cannot be reproduced from a table.
• TURN over DTLS uses upstream pion without a ClientHello hook, so that handshake is not mimicked. libwebrtc implements this transport, so a server that offers it would see a pion handshake where Chrome sends its own. The branch runs only on a turns: URL with transport=udp, which no server in the measured traffic hands out, and a bare turns: resolves to TCP.
• Host ICE candidates are not hidden behind mDNS .local names. Chrome hides them only when the origin has no media permission, so the behaviour differs between calls. One reference capture carries 214 candidates and none is mDNS. In another, the remote peer offered two .local candidates and this library resolved them over mDNS. Hiding our own unconditionally would be wrong for the first case, and the condition that decides it is not visible from our side.

Open questions

The ICE keepalive interval is set to 2656 ms, a measured value. libwebrtc sets kStrongAndStableWritableConnectionPingInterval to 2500 ms, and reading the scheduler in wrapping_active_ice_controller.cc predicts 2500 ms. The 156 ms difference has no explanation.

Four causes were ruled out by measurement.

• RTT. The value is the same at 0.05 ms and at 60 ms.
• Capture noise. The standard deviation of the period is smaller than the RTT spread of the path.
• Timer drift. The offset is the same on a Mac and in a Linux container.
• A bimodal distribution hidden by the mean. The medians agree.

The measurement covers five datasets, two machines and three services, with medians within 0.5 ms. All reference captures are Chrome on Linux, and the profile reports Windows.

This library is configured for 2656 ms. The measured median is 2657 ms, over 65 intervals on each of two peer connections. The ICE task loop resets its timer after the task runs, which adds the extra millisecond.

Layout

internal/dtls, internal/ice, internal/chromehttp2, webrtc, and websocket are vendored. They are copies of upstream packages with fingerprint patches applied. Do not edit them by hand. Run the scripts in update-deps to regenerate them. Guard tests fail if a regenerated tree loses a patch.

internal/chromehttp1 is not vendored. It is an HTTP/1.1 request writer and connection pool written for this library, because net/http sorts header names.

See update-deps/README.md and stand/README.md.

Documentation

Index

• Variables
• type Profile
  • func (p Profile) ClientHelloID() utls.ClientHelloID
  • func (p Profile) HTTPClient() *http.Client
  • func (p Profile) Headers(dest RequestDest) http.Header
  • func (p Profile) Name() string
  • func (p Profile) RegisterHeaderExtensions(mediaEngine *webrtc.MediaEngine) error
  • func (p Profile) SettingEngine() (webrtc.SettingEngine, error)
  • func (p Profile) Transport(options TLSOptions) http.RoundTripper
  • func (p Profile) UserAgent() string
  • func (p Profile) WebSocketDialer(options TLSOptions) *websocket.Dialer
  • func (p Profile) WithAcceptLanguage(acceptLanguage string) Profile
  • func (p Profile) WithClientHelloID(clientHelloID utls.ClientHelloID) Profile
  • func (p Profile) WithDTLS13Mimicry() Profile
  • func (p Profile) WithDTLSGREASE() Profile
  • func (p Profile) WithName(name string) Profile
  • func (p Profile) WithUserAgent(userAgent string) Profile
• type RequestDest
• type TLSOptions

Variables

var ChromeWindows = Profile{
	// contains filtered or unexported fields
}

Types

type Profile

type Profile struct {
	// contains filtered or unexported fields
}

func (Profile) ClientHelloID

func (p Profile) ClientHelloID() utls.ClientHelloID

func (Profile) HTTPClient

func (p Profile) HTTPClient() *http.Client

func (Profile) Headers

func (p Profile) Headers(dest RequestDest) http.Header

func (Profile) Name

func (p Profile) Name() string

func (Profile) RegisterHeaderExtensions

func (p Profile) RegisterHeaderExtensions(mediaEngine *webrtc.MediaEngine) error

func (Profile) SettingEngine

func (p Profile) SettingEngine() (webrtc.SettingEngine, error)

func (Profile) Transport

func (p Profile) Transport(options TLSOptions) http.RoundTripper

func (Profile) UserAgent

func (p Profile) UserAgent() string

func (Profile) WebSocketDialer

func (p Profile) WebSocketDialer(options TLSOptions) *websocket.Dialer

func (Profile) WithAcceptLanguage

func (p Profile) WithAcceptLanguage(acceptLanguage string) Profile

func (Profile) WithClientHelloID

func (p Profile) WithClientHelloID(clientHelloID utls.ClientHelloID) Profile

func (Profile) WithDTLS13Mimicry

func (p Profile) WithDTLS13Mimicry() Profile

func (Profile) WithDTLSGREASE

func (p Profile) WithDTLSGREASE() Profile

func (Profile) WithName

func (p Profile) WithName(name string) Profile

func (Profile) WithUserAgent

func (p Profile) WithUserAgent(userAgent string) Profile

type RequestDest

type RequestDest int

const (
	DestDocument RequestDest = iota
	DestScript
	DestEmpty
	DestWebSocket
)

type TLSOptions

type TLSOptions struct {
	DialContext        func(ctx context.Context, network, address string) (net.Conn, error)
	ServerName         string
	InsecureSkipVerify bool
}