rpcprobe

README

rpcprobe

dig/mtr for EVM JSON-RPC providers. One command tells you, for a set of endpoints, who is lagging, who is serving a different chain history, who throttles you, and who does not support the methods your application needs.

$ rpcprobe check --endpoints examples/endpoints.txt

rpcprobe is a provider due-diligence and SLO-monitoring tool for wallet and dApp teams choosing providers, node operators comparing their own nodes against public gateways, and SREs who need an exit code or a Prometheus metric to alert on. It is explicitly not a load-testing tool; if you want to saturate a node, use something like Chainbench.

Why this exists

• Provider benchmarks are usually published by providers. rpcprobe runs locally, from your network, against the URLs you actually use (including your API keys), and has no stake in the result.
• "Supports Ethereum" hides the details that break applications: eth_getLogs capped at 2,000 blocks, batches capped at 3 requests, no debug_*, state only for the last 128 blocks, a wss:// URL that never delivers a header. rpcprobe probes each of those explicitly.
• Lag and consistency only make sense relative to peers. rpcprobe compares every endpoint against the others of the same chain id: highest head wins, block hash at head - N is decided by majority vote.
• It is one static binary with a stable JSON schema and non-zero exit codes, so it drops into CI jobs, cron and alerting without glue.

Install

go install github.com/meldrov/rpcprobe/cmd/rpcprobe@latest

or from a checkout:

git clone https://github.com/meldrov/rpcprobe
cd rpcprobe
make build          # -> bin/rpcprobe

Prebuilt static binaries for Linux, macOS and Windows (amd64/arm64) are on the Releases page, with a checksums.txt to verify downloads.

Requires Go 1.26+. The only dependency outside the standard library is github.com/coder/websocket (for the subscription probe).

Quickstart

Real output against public Ethereum mainnet gateways (2026-08-21, from a residential connection in Europe; long lines trimmed):

$ rpcprobe check --endpoints examples/endpoints.txt
probing 5 endpoint(s)...
rpcprobe check: 5 endpoint(s), chain 1: max head 25801187, consistency block 25801177

ENDPOINT    CHAIN  HEAD      LAG  P50    P95    MAX    ERR   CONSISTENCY  BURST     BATCH  LOGS RANGE  ARCHIVE    WS
publicnode  1      25801187  0    361ms  371ms  371ms  0/10  consistent   ok 25/25  >=100  error       head-only  ok 5.66s
drpc        1      25801187  0    205ms  206ms  206ms  0/10  consistent   ok 25/25  2      <=2000      archive    -
flashbots   1      25801187  0    307ms  778ms  778ms  0/10  consistent   ok 25/25  5      <=5000      unknown    -
1rpc        -      -         -    -      -      -      -     -            -         -      -           -          -
merkle      -      -         -    -      -      -      -     -            -         -      -           -          -

METHOD                    publicnode   drpc    flashbots    1rpc  merkle
eth_getLogs               error        <=2000  <=5000       -     -
eth_feeHistory            ok           ok      ok           -     -
eth_maxPriorityFeePerGas  ok           ok      ok           -     -
eth_getBlockReceipts      ok           ok      ok           -     -
eth_call(stateOverride)   ok           ok      unsupported  -     -
debug_traceCall           unsupported  ok      unsupported  -     -
trace_block               unsupported  ok      unsupported  -     -
eth_getProof              ok           ok      timeout      -     -

publicnode:
  - client: Geth/v1.17.1-stable-16783c16/linux-amd64/go1.25.7
  - eth_getLogs: error: rpc error -32701: Please specify an address in your request or, to remove restrictions, order a dedicated full node ...
drpc:
  - client: Geth/v10.0.0/drpc
  - eth_getLogs: 2000-block range ok, 5000-block range failed: rpc error -32602: block range extends beyond current head block
  - batch: size 5 failed: http 500: rpc error 31: Batch of more than 3 requests are not allowed on free plan, to use this feature register ...
flashbots:
  - client: reth/v1.11.2-793a3d5/x86_64-unknown-linux-gnu
  - eth_getLogs: 5000-block range ok, 10000-block range failed: Post "https://rpc.flashbots.net": context deadline exceeded
  - batch: size 10 failed: rpc error -32014: too many RPC calls in batch request
  - eth_getProof: timeout: Post "https://rpc.flashbots.net": context deadline exceeded
1rpc:
  - unreachable: eth_chainId: rpc error -32001: You've reached the usage limit for your current plan. To continue with higher limits ...
merkle:
  - unreachable: eth_chainId: http 429: {"type":"https://developers.cloudflare.com/support/troubleshooting/http-status-codes/cloudflare-1xxx-errors/error-1015/"...
summary: 3/5 reachable, 0 lagging, 0 inconsistent, 0 rate-limited, 2 violation(s)
violations:
  - 1rpc: unreachable: eth_chainId: rpc error -32001: You've reached the usage limit for your current plan. ...
  - merkle: unreachable: eth_chainId: http 429: ... error-1015 ...
$ echo $?
1

Things this single run surfaced: publicnode's free gateway serves no historical state at all (head-only) and rejects address-less eth_getLogs; drpc's free plan caps batches at 3 and getLogs somewhere between 2,000 and 5,000 blocks; flashbots has no state override / debug / trace and its eth_getProof timed out; 1rpc and merkle had throttled this IP after a couple of earlier runs. The exit code is 1 because two endpoints were unreachable.

Mixed chains are fine; lag and consistency are computed per chain id:

$ rpcprobe check base=https://mainnet.base.org arbitrum=https://arb1.arbitrum.io/rpc
rpcprobe check: 2 endpoint(s), chain 8453: max head 50249052, consistency block 50249042, chain 42161: max head 496747329, consistency block 496747319

ENDPOINT  CHAIN  HEAD       LAG  P50    P95    MAX    ERR   CONSISTENCY  BURST     BATCH  LOGS RANGE  ARCHIVE  WS
base      8453   50249052   0    291ms  300ms  300ms  0/10  n/a          ok 25/25  10     <=5000      archive  -
arbitrum  42161  496747329  0    352ms  841ms  841ms  0/10  n/a          ok 25/25  >=100  >=10000     recent   -

METHOD                    base         arbitrum
eth_getLogs               <=5000       >=10000
eth_feeHistory            ok           ok
eth_maxPriorityFeePerGas  ok           ok
eth_getBlockReceipts      unsupported  ok
eth_call(stateOverride)   ok           ok
debug_traceCall           unsupported  unsupported
trace_block               unsupported  unsupported
eth_getProof              error        ok

base:
  - client: reth/v2.3.0-9384bc5/x86_64-unknown-linux-gnu/base/v1.1.1
  - eth_getLogs: 5000-block range ok, 10000-block range failed: http 413: rpc error -32614: eth_getLogs is limited to a 10,000 range
  - batch: size 25 failed: rpc error -32014: maximum 10 calls in 1 batch
  - eth_getProof: error: rpc error -32603: no state found for block number 50249053
arbitrum:
  - client: nitro/v3.11.3-rc.9-beb2108/linux-arm64/go1.25.12
summary: 2/2 reachable, 0 lagging, 0 inconsistent, 0 rate-limited, 0 violation(s)

Continuous mode:

$ rpcprobe watch --interval 5s publicnode=https://ethereum-rpc.publicnode.com drpc=https://eth.drpc.org flashbots=https://rpc.flashbots.net
2026-08-21T04:48:05Z  round 1  interval 5s  chain 1: head 25801212, consistency block 25801202
ENDPOINT    CHAIN  UP  HEAD      LAG  P50    P95    ERR  ERR TOTAL  CONSISTENCY
publicnode  1      up  25801212  0    386ms  391ms  0    0          consistent
drpc        1      up  25801212  0    270ms  280ms  0    0          consistent
flashbots   1      up  25801212  0    306ms  318ms  0    0          consistent

A quick sanity bench:

$ rpcprobe bench --rps 5 --duration 5s --method eth_getBlockByNumber --params '["latest", false]' https://eth.drpc.org
bench https://eth.drpc.org eth_getBlockByNumber
  duration      5.32s
  target rps    5.00
  achieved rps  4.70
  requests      25 (errors 0, 0.0%, dropped 0)
  latency       min 218ms  p50 233ms  p95 830ms  p99 858ms  max 858ms

Endpoints

Endpoints are given as arguments or through --endpoints FILE (repeatable, - for stdin). Each is url or label=url, optionally followed by a WebSocket URL:

# examples/endpoints.txt
publicnode=https://ethereum-rpc.publicnode.com wss://ethereum-rpc.publicnode.com
drpc=https://eth.drpc.org
# comments and blank lines are ignored

A ws:///wss:// URL given as the main URL is used for the subscription probe and converted to http(s):// for the JSON-RPC calls (this matches most providers; if yours differs, give both URLs). --ws label=wss://... or a positional --ws wss://... attaches a WebSocket URL from the command line. Labels default to the hostname; duplicates get #2, #3, ...

Commands

rpcprobe check [flags] <endpoint>...

One-shot probe. Endpoints are probed concurrently; inside one endpoint the probes run sequentially (except the WebSocket probe) so rpcprobe does not trip a provider's rate limiter before the burst probe does it on purpose.

Flag	Default	Meaning
--endpoints FILE		endpoint file, repeatable, - = stdin
--ws URL		label=wss://... or positional ws URL, repeatable
--samples N	10	sequential eth_blockNumber calls for latency
--consistency-depth N	10	compare block hash at maxHead - N
--burst N	25	concurrent requests for the rate-limit probe
--timeout D	10s	per-call timeout
--ws-timeout D	30s	wait for the first newHeads notification
--logs-ranges LIST	10000,5000,2000,1000,500,100,50,10	eth_getLogs range ladder
--batch-sizes LIST	2,5,10,25,50,100	batch size ladder
--quick		skip method matrix, getLogs, archive, batch and burst
--no-burst		skip only the burst probe
--no-ws		skip the WebSocket probe
--json / --format text|json|markdown	text	output format
--max-lag N	off	exit 1 if any endpoint is more than N blocks behind its chain's max head
--max-p95 D	off	exit 1 if any endpoint's p95 exceeds D
--require LIST		exit 1 unless every endpoint supports these; matrix names plus batch, ws, archive, consistent

A full run takes roughly 15-40 s per endpoint depending on the provider (the WebSocket probe waits for a real block; on mainnet that is up to 12 s).

rpcprobe watch [flags] <endpoint>...

Re-probes latency (--samples, default 3), head, lag, error count and consistency every --interval (default 15s) and redraws a table. When stdout is not a terminal, or with --no-tty, it prints one timestamped block per round instead. --json-lines prints one JSON object per round (see schema below). --prometheus ADDR serves /metrics. --rounds N stops after N rounds (useful for cron). Errors are counted cumulatively per endpoint.

rpcprobe bench [flags] <endpoint>

Open-loop load: a request is started every 1/--rps seconds (default 10) for --duration (default 10s) regardless of outstanding requests, calling --method with --params (JSON array). Reports requests, errors by kind (rate-limited, timeout, http-NNN, rpc-CODE, transport), achieved RPS and min/p50/p95/p99/max. --max-inflight N caps concurrency (ticks beyond the cap are counted as dropped). Exit 1 if every request failed.

rpcprobe version

Prints the version (set with -ldflags "-X main.version=..."), Go version and platform.

Exit codes

Code	Meaning
0	all endpoints reachable and no threshold violated
1	at least one endpoint unreachable, or a --max-lag / --max-p95 / --require violation (check); all requests failed (bench); runtime error
2	usage error (bad flag, no endpoints, unreadable file)

Violations are listed at the end of the text report and in the violations array of the JSON report, so a CI job can fail with a readable reason:

rpcprobe check --endpoints prod.txt --max-lag 2 --max-p95 750ms --require eth_getLogs,eth_feeHistory,batch

JSON schema (check --json)

Durations are milliseconds as decimal numbers. Status strings are ok | unsupported | rate-limited | timeout | error | skipped.

{
  "version": "0.1.0",
  "generated_at": "2026-08-21T04:44:28Z",
  "chains": [                      // one entry per chain id seen among reachable endpoints
    {
      "chain_id": 1,
      "endpoints": 3,
      "max_head": 25801194,
      "consistency_block": 25801184,  // max_head - consistency-depth (0 if chain too short)
      "majority_hash": "0xe2e7..."
    }
  ],
  "endpoints": [
    {
      "label": "publicnode",
      "url": "https://ethereum-rpc.publicnode.com",
      "ws_url": "wss://ethereum-rpc.publicnode.com",   // omitted when none
      "reachable": true,
      "error": "",                    // set when unreachable (eth_chainId failed)
      "chain_id": 1,
      "client_version": "Geth/v1.17.1-...",
      "net_version": "1",
      "syncing": false,
      "head": 25801194,
      "lag": 0,                       // blocks behind the chain's max_head; -1 if unknown
      "latency": { "samples": 10, "errors": 0, "min_ms": 272.87, "p50_ms": 276.53, "p95_ms": 285.41, "max_ms": 285.41 },
      "consistency": {
        "block": 25801184,
        "hash": "0xe2e7...",
        "status": "consistent",       // consistent | inconsistent | behind | unknown | n/a
        "detail": ""                  // e.g. "differs from majority 0x..."
      },
      "methods": {                    // keyed by matrix row name
        "eth_getLogs":              { "status": "error", "detail": "rpc error -32701: ...", "latency_ms": 0 },
        "eth_feeHistory":           { "status": "ok", "latency_ms": 281.2 },
        "eth_maxPriorityFeePerGas": { "status": "ok", "latency_ms": 270.1 },
        "eth_getBlockReceipts":     { "status": "ok", "latency_ms": 602.9 },
        "eth_call(stateOverride)":  { "status": "ok", "latency_ms": 290.5 },
        "debug_traceCall":          { "status": "unsupported", "detail": "rpc error -32601: ...", "latency_ms": 271.0 },
        "trace_block":              { "status": "unsupported", "detail": "http 403: ...", "latency_ms": 269.8 },
        "eth_getProof":             { "status": "ok", "latency_ms": 297.3 }
      },
      "get_logs":   { "status": "ok", "max_range": 2000, "failed_at": 5000, "detail": "last error" },
      "batch":      { "status": "ok", "max_size": 2, "failed_at": 5, "detail": "last error" },
      "rate_limit": { "burst": 25, "ok": 25, "limited": 0, "errors": 0, "status": "ok", "detail": "" },   // ok | rate-limited | error | skipped
      "archive": {
        "level": "head-only",         // archive | deep | recent | full | head-only | unknown | skipped
        "checks": [ { "name": "head-128", "block": 25801066, "status": "unsupported", "detail": "..." }, ... ]
      },
      "ws": {                         // omitted when the endpoint has no WebSocket URL
        "url": "wss://ethereum-rpc.publicnode.com",
        "status": "ok",               // ok | timeout | error | unsupported
        "connect_ms": 412.1,
        "first_header_ms": 5660.3,
        "head": 25801195,
        "error": ""
      },
      "probe_duration_ms": 31234.5,
      "warnings": [ "net_version: ..." ]   // omitted when empty
    }
  ],
  "violations": [ { "endpoint": "1rpc", "rule": "unreachable", "message": "..." } ]   // rule: unreachable | max-lag | max-p95 | require
}

watch --json-lines emits per round:

{
  "time": "2026-08-21T04:48:12Z",
  "round": 1,
  "chains": [ { "chain_id": 1, "max_head": 25801213, "consistency_block": 25801203, "majority_hash": "0x..." } ],
  "endpoints": [
    { "label": "publicnode", "chain_id": 1, "up": true, "head": 25801213, "lag": 0,
      "p50_ms": 370.7, "p95_ms": 485.9, "errors": 0, "errors_total": 0,
      "consistency": "consistent", "hash": "0x...", "error": "" }
  ]
}

Prometheus metrics (watch --prometheus :9090)

Text exposition format, no client library. Labels: endpoint (the label), chain (chain id; 0 until learned).

Metric	Type	Meaning
rpcprobe_up{endpoint,chain}	gauge	1 if eth_blockNumber answered at least once in the last round
rpcprobe_head_block{endpoint,chain}	gauge	head reported by the endpoint (up endpoints only)
rpcprobe_lag_blocks{endpoint,chain}	gauge	blocks behind the chain's max head
rpcprobe_latency_seconds{endpoint,chain,quantile="0.5"|"0.95"}	gauge	latency quantiles of the last round
rpcprobe_errors_total{endpoint,chain}	counter	failed probe calls since start
rpcprobe_consistent{endpoint,chain}	gauge	1 if the block hash matches the majority (also 1 when there is nothing to compare against), 0 if inconsistent, absent if unknown
rpcprobe_max_head_block{chain}	gauge	highest head among watched endpoints of the chain
rpcprobe_round	counter	completed rounds

Example alert rules:

- alert: RpcEndpointLagging
  expr: rpcprobe_lag_blocks > 3
  for: 2m
- alert: RpcEndpointInconsistent
  expr: rpcprobe_consistent == 0
- alert: RpcEndpointDown
  expr: rpcprobe_up == 0
  for: 1m

How the probes work

• Reachability / identity. eth_chainId must succeed; otherwise the endpoint is unreachable and nothing else runs. web3_clientVersion, net_version and eth_syncing are tolerated when unsupported.
• Latency. --samples sequential eth_blockNumber calls over a warm keep-alive connection; min/p50/p95/max with the nearest-rank method (with 10 samples p95 equals max, use --samples 20 or more for a real p95). The head is the highest block number seen in those samples.
• Lag. Reachable endpoints are grouped by chain id; lag = max head of the group - head.
• Consistency. eth_getBlockByNumber(maxHead - depth) from every endpoint of the chain; the most common hash is the majority, the rest are inconsistent. behind means the endpoint's head was below the block, unknown means the fetch failed, n/a means there was nothing to compare against (single endpoint or chain shorter than the depth). It runs before the heavy probes so the burst cannot disturb it.
• eth_getLogs range. Tries the ladder largest first with fromBlock = head - range + 1, toBlock = "latest" and a sparse filter (ERC-20 Transfer with both indexed addresses equal to zero) so result- count limits do not interfere; the first rung that succeeds is max_range, the smallest rung that failed is failed_at. >=10000 means the largest rung succeeded. A hard -32601/404 or a rate-limit error stops the ladder early; any other error is treated as range-dependent and the next rung is tried.
• Method matrix. eth_feeHistory(4, latest, [25,75]), eth_maxPriorityFeePerGas, eth_getBlockReceipts(latest), trace_block(head), eth_getProof(0x0, [], latest), debug_traceCall({to}, latest, {tracer: callTracer}). State override is detected by calling eth_call against an address without code while overriding its code with a contract returning 42: a node that silently ignores the third parameter returns 0x and is reported unsupported. Errors are classified into unsupported (-32601, "not supported/allowed/ whitelisted/available", HTTP 401/403/404/405/501), rate-limited (HTTP 429, -32005/-32016/-32029-style codes with a rate-limit message, messages mentioning rate/request limits or quotas), timeout, or error.
• Archive depth. eth_getBalance(0x0, block) at head-128, head-1000, head-100000 and block 1. Levels: archive (block 1 works), deep (>= 100k blocks), recent (>= 1000), full (>= 128, geth's default in-memory window), head-only (all historical lookups refused), unknown (errors that are not refusals, e.g. timeouts).
• Batch. JSON arrays of eth_blockNumber of the ladder sizes, smallest first; a size counts as accepted when the response is an array with one non-error entry per request. max_size is the largest accepted, failed_at the first rejected size.
• Rate limit. --burst concurrent eth_blockNumber calls; HTTP 429 and rate-limit-shaped JSON-RPC errors are counted as limited. It runs last.
• WebSocket. Dial, eth_subscribe newHeads, wait up to --ws-timeout for the first eth_subscription notification; reports connect time, time to first header and the header's block number. Runs in parallel with the HTTP probes.

Every probe is error-isolated: a failing call records its status and detail and the rest of the report is still produced.

Limitations

• Range and batch limits are found on a ladder, so the report says <=2000 (2000 worked, the next rung did not), not the exact cap. Tighten the ladder with --logs-ranges / --batch-sizes if you need the exact value.
• Provider behaviour varies with load balancing, plan, key and source IP; a single run is a sample. Use watch or run check a few times before drawing conclusions. Repeated runs from one IP will themselves trip per-IP limits on some public gateways (seen with 1rpc and merkle above).
• The getLogs probe measures the block-range cap, not result-count caps or address restrictions (publicnode's "specify an address" policy shows up as error with the message).
• The WebSocket probe only tests newHeads; it does not test reconnect behaviour, logs subscriptions or RPC-over-WebSocket calls.
• Archive detection uses the zero address; a gateway that special-cases it could mislead the classification.
• The latency numbers include your network path; run from where your application runs.
• bench is open-loop without coordinated-omission correction beyond counting dropped ticks; it is a smoke test, not a benchmark suite.

Roadmap

• Non-EVM JSON-RPC profiles (Solana getSlot/getBlock, Bitcoin Core).
• WebSocket reconnect and logs subscription testing.
• Multi-region runs (aggregate reports from several vantage points).
• Grafana dashboard JSON for the Prometheus metrics.
• Exact cap search (binary search between ladder rungs).
• Optional per-endpoint headers (for providers that want keys in headers).

Contributing

Issues and pull requests are welcome. Before sending a PR:

make lint      # gofmt + go vet
make test      # unit tests, no network needed
make integration   # RPCPROBE_INTEGRATION=1, hits public endpoints

Keep dependencies to the standard library (plus the WebSocket client), keep every probe error-isolated, and add a fake-server test under internal/rpctest for new classification rules.

License

MIT, see LICENSE. Copyright (c) 2026 Meldrov.