site-mimic

module
v0.2.1-0...-f5a6213 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Sep 6, 2026 License: MIT

README

site-mimic

Русский · 中文 · HANDOFF

Make a Go HTTP client present the same wire-level identity a target site expects from a real browser — TLS ClientHello (JA3/JA4), ALPN, HTTP/2 and the captured header set. Stock Go TLS is trivially distinguishable, and ClientHello-level signatures are exactly what DPI middleboxes block first.

site-mimic packages a production-proven uTLS transport plus the methodology that makes browser-fitting repeatable: an AI-agent skill that walks a new site from capture to verified request, capture/verification tooling, and two fully worked examples.

Verified (2026-08-30, against vk.ru)

Client JA4 (stand, pinned Chromium) Result
uTLS chrome_auto t13d1516h2_8daaf6152771_d8a2da3f94cd 200 OK over HTTP/2; near-match (extension set differs)
chrome_exact (delegates TLS to headless-client) t13d1516h2_8daaf6152771_806a8c22fdea byte-exact JA4 match with the browser
Real phone (Samsung S21 Ultra) Chrome 149 t13d1516h2_8daaf6152771_d8a2da3f94cd (phone reference) mobile reference; android_149 profile matches field-for-field
Real phone Chrome 152 (updated 2026-09-01) t13d1518h2_8daaf6152771_e2d80978ab2e — ML-DSA + 0xCA24 on mobile newest mobile reference
Desktop Chrome 152 stable t13d1517h2_8daaf6152771_cb7bf5808d99 desktop 152 reference
Windows Chrome 151 headed (AdGuard on) t13d1516h2_8daaf6152771_806a8c22fdea, TTL 128 examples/vk-ru-windows

The complete by-platform/by-condition table (AdGuard, headless vs headed, BrowserOS, Schannel, insecure-context caveats) lives in docs/fingerprint-matrix.md.

Also verified: stream.wb.ru returns the same 498 wbaas anti-bot challenge a fresh real browser gets — transport parity is correct, the JS challenge is app-layer (see examples/stream-wb-ru).

Credits: headless-client is the more accurate transport

kulikov0/headless-client is better at the transport layer, and site-mimic builds on it. Its ClientHello is hand-measured against the current stable Chrome (including post-quantum signature algorithms), which is why it matches the browser byte-for-byte, while the off-the-shelf uTLS chrome_auto profile lags slightly behind. Its HTTP part additionally covers header order, HTTP/2 SETTINGS framing and connection reuse, and its capture stand (stand/) diffs your binary against a real Chromium on the wire — the verification loop we now recommend everywhere.

site-mimic's own value is the layer around the transport: the fit-a-site skill for AI agents, capture→profile→verify tooling, site profiles and worked examples. tls_client_hello: "chrome_exact" delegates the TLS/HTTP2 layers to headless-client (MIT, thank you), chrome_auto and friends stay available as pure-uTLS fallbacks.

Install

go get github.com/megamen32/site-mimic/mimic

Start in minutes

git clone https://github.com/megamen32/site-mimic && cd site-mimic/examples/vk-ru
go run . -dump ch.json
python3 ../../tools/parse_clienthello.py ch.json   # JA3/JA4 of our ClientHello

Expect status: 200 OK, proto: HTTP/2.0 (server: kittenx). Then fit a new site with skill/SKILL.md.

Learn more

Honest limits

With chrome_exact the TLS layer is byte-exact, and the header wire order plus Chrome-shaped HTTP/2 framing come from headless-client too (see docs/methodology.md for what is closed vs still Go-shaped on the uTLS-only path). QUIC/DTLS are not covered. Anti-bot JavaScript challenges are out of scope by design — replay support for browser-harvested cookies is in docs/anti-bot.md.

MIT licensed. Not affiliated with VK or Wildberries.

Directories

Path Synopsis
cmd
fetchd command
fetchd is a local HTTP daemon that executes one-off requests through the mimic library (uTLS Chrome ClientHello / chrome_exact) — the single truth for outbound Rutube traffic from non-Go callers.
fetchd is a local HTTP daemon that executes one-off requests through the mimic library (uTLS Chrome ClientHello / chrome_exact) — the single truth for outbound Rutube traffic from non-Go callers.
fpd command
Command fpd is the live full-fingerprint display for the site-mimic verification stand (test.auto-gram.ru).
Command fpd is the live full-fingerprint display for the site-mimic verification stand (test.auto-gram.ru).
receiver command
Command receiver is the public byte-capturing target for site-mimic verification runs: deploy it behind nginx (TLS terminates at nginx via the shared 127.0.0.1:8444 SNI listener) and it logs the application view of every request as JSONL — client IP (X-Forwarded-For chain), method, path, protocol, header names and values — and answers with an echo JSON.
Command receiver is the public byte-capturing target for site-mimic verification runs: deploy it behind nginx (TLS terminates at nginx via the shared 127.0.0.1:8444 SNI listener) and it logs the application view of every request as JSONL — client IP (X-Forwarded-For chain), method, path, protocol, header names and values — and answers with an echo JSON.
examples
stand-probe command
Command stand-probe is a kulikov0/headless-client stand-compatible probe: it fetches a URL N times with the site-mimic client so the capture stand can diff its wire fingerprint against a real Chromium running next to it.
Command stand-probe is a kulikov0/headless-client stand-compatible probe: it fetches a URL N times with the site-mimic client so the capture stand can diff its wire fingerprint against a real Chromium running next to it.
stream-wb-ru command
Command stream-wb-ru fetches https://stream.wb.ru/ the way a desktop Chrome browser does on first load.
Command stream-wb-ru fetches https://stream.wb.ru/ the way a desktop Chrome browser does on first load.
vk-ru command
Command vk-ru fetches https://vk.ru/ the way a desktop Chrome browser does: uTLS chrome_auto ClientHello, ALPN-negotiated HTTP/2, and the header set captured from a real navigation.
Command vk-ru fetches https://vk.ru/ the way a desktop Chrome browser does: uTLS chrome_auto ClientHello, ALPN-negotiated HTTP/2, and the header set captured from a real navigation.
vk-ru-mobile command
Command vk-ru-mobile fetches https://m.vk.ru/ the way a mobile Chrome browser does: the android_149 ClientHello (uTLS Chrome spec wire-verified field-for-field against a real Android Chrome 149 — identical JA4 t13d1516h2_8daaf6152771_d8a2da3f94cd and per-connection extension shuffling, see docs/phone-reference.md) and the header set captured from a real Android Chrome 149 navigation on a phone (Samsung S21 Ultra, mobile 4G).
Command vk-ru-mobile fetches https://m.vk.ru/ the way a mobile Chrome browser does: the android_149 ClientHello (uTLS Chrome spec wire-verified field-for-field against a real Android Chrome 149 — identical JA4 t13d1516h2_8daaf6152771_d8a2da3f94cd and per-connection extension shuffling, see docs/phone-reference.md) and the header set captured from a real Android Chrome 149 navigation on a phone (Samsung S21 Ultra, mobile 4G).
vk-ru-windows command
Command vk-ru-windows fetches https://vk.ru/ presenting as a real Windows desktop Chrome: the chrome_exact transport (byte-exact ClientHello class verified against a live Windows Chrome 151 — t13d1516h2_8daaf6152771_806a8c22fdea, captured headed on a Win11 23H2 machine, see docs/phone-reference.md), the Windows header set and sec-ch-ua brand list, and the Windows IP TTL (128) stamped on every connection via the profile's ip_ttl field.
Command vk-ru-windows fetches https://vk.ru/ presenting as a real Windows desktop Chrome: the chrome_exact transport (byte-exact ClientHello class verified against a live Windows Chrome 151 — t13d1516h2_8daaf6152771_806a8c22fdea, captured headed on a Win11 23H2 machine, see docs/phone-reference.md), the Windows header set and sec-ch-ua brand list, and the Windows IP TTL (128) stamped on every connection via the profile's ip_ttl field.
Package mimic provides an HTTP client whose transport-level fingerprint — TLS ClientHello (JA3/JA4), ALPN, HTTP/2 — follows a site profile captured from a real browser, so a Go program presents the same wire-level identity the site expects.
Package mimic provides an HTTP client whose transport-level fingerprint — TLS ClientHello (JA3/JA4), ALPN, HTTP/2 — follows a site profile captured from a real browser, so a Go program presents the same wire-level identity the site expects.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL