caddyx402

package module
v0.0.0-...-160485e Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Feb 27, 2026 License: BSD-3-Clause Imports: 23 Imported by: 0

README

caddy-x402

A Caddy middleware that charges AI crawlers real money (USDC stablecoin) to access your website content, using the x402 protocol by Coinbase.

The problem

AI companies send crawlers (GPTBot, ClaudeBot, etc.) to scrape your website and use your content to train their models or power their products. You get nothing. robots.txt is a suggestion they can ignore. This plugin makes them pay.

How the money flows

AI Crawler (GPTBot, ClaudeBot, etc.)
    |
    |  1. Crawls your page
    v
Your Caddy Server (this plugin)
    |
    |  2. Returns HTTP 402 "Pay me $0.01 in USDC to see this"
    v
AI Crawler
    |
    |  3. Signs a USDC payment authorization (EIP-3009)
    |     and sends it in the X-PAYMENT header
    v
Your Caddy Server
    |
    |  4. Forwards the signed payment to the Facilitator
    v
x402 Facilitator (x402.org, run by Coinbase)
    |
    |  5. Verifies the signature is valid
    |  6. Submits the transaction on-chain (Base L2)
    |  7. USDC moves from crawler's wallet -> your wallet
    v
Your Caddy Server
    |
    |  8. Confirms payment, serves the content
    v
AI Crawler gets the page content

You receive USDC (a dollar-pegged stablecoin) directly in your wallet. No middlemen hold your money. The facilitator just relays the transaction to the blockchain — the payment goes straight from the crawler's wallet to yours.

What you need

  1. An EVM wallet address to receive payments. This is a standard Ethereum/Base wallet address (starts with 0x). You can create one with:

  2. A server running Caddy (your web server)

  3. xcaddy to build Caddy with this plugin: go install github.com/caddyserver/xcaddy/cmd/xcaddy@latest

That's it. No API keys needed for the default facilitator. No crypto knowledge required beyond having a wallet address.

Quick start

1. Build Caddy with the plugin
xcaddy build --with github.com/paolobietolini/caddy-x402

This produces a caddy binary with the x402 middleware baked in.

2. Configure your Caddyfile
yoursite.com {
    x402 {
        pay_to    0xYOUR_WALLET_ADDRESS
        price     0.01
        network   base

        # Don't block these paths (crawlers need robots.txt, search engines need favicons)
        exempt /robots.txt
        exempt /favicon.ico
        exempt /.well-known/*
    }

    # Your normal site config below
    reverse_proxy localhost:3000
    # or: file_server, php_fastcgi, etc.
}
3. Replace your current Caddy binary and restart
sudo systemctl stop caddy
sudo cp caddy /usr/bin/caddy
sudo chmod +x /usr/bin/caddy
sudo systemctl start caddy
4. Verify it works
# Normal browser request — should get your page normally
curl https://yoursite.com/

# AI crawler request — should get 402 Payment Required
curl -A "GPTBot/1.0" https://yoursite.com/

The 402 response looks like:

{
  "x402Version": 1,
  "error": "Payment required to access this resource",
  "accepts": [{
    "scheme": "exact",
    "network": "base",
    "maxAmountRequired": "10000",
    "asset": "0x833589fCD6eDb6E08f4c7C32D4f71b54bdA02913",
    "payTo": "0xYOUR_WALLET_ADDRESS",
    "resource": "https://yoursite.com/your-page",
    "maxTimeoutSeconds": 60,
    "extra": {"name": "USDC", "version": "2"}
  }]
}

How to get paid

Once the plugin is running, payments happen automatically:

  1. An AI crawler hits your site
  2. It gets a 402 response telling it the price and your wallet
  3. If the crawler supports x402 (and has funds), it signs a payment and retries
  4. The payment is settled on Base (an Ethereum L2 — fast and cheap)
  5. USDC appears in your wallet

To check your balance, look up your wallet address on basescan.org. The USDC will show under the "Token" tab.

To convert USDC to real money in your bank, you can:

  • Transfer USDC to Coinbase, sell for EUR/USD, withdraw to your bank
  • Use any exchange that supports USDC on Base
Which crawlers support x402?

As of early 2025, x402 is new. Crawlers that support it will pay automatically. Crawlers that don't will simply get a 402 error and not see your content — which is the point. You're blocking them unless they pay.

The plugin detects these crawlers (and more):

Company Crawler
OpenAI GPTBot, ChatGPT-User, OAI-SearchBot
Google Google-Extended, Googlebot-Extended
Anthropic ClaudeBot, Claude-Web, anthropic-ai
Meta FacebookBot, Meta-ExternalAgent
Perplexity PerplexityBot
Amazon Amazonbot
Bytedance Bytespider
Common Crawl CCBot

Normal users (browsers, curl, etc.) are never affected.

Testing on testnet first

If you want to test without real money, use base-sepolia (the test network):

yoursite.com {
    x402 {
        pay_to    0xYOUR_WALLET_ADDRESS
        price     0.01
        network   base-sepolia
    }
}

You can get free testnet USDC from Coinbase's faucet.

Dry run mode

To see what would happen without any blockchain interaction:

x402 {
    pay_to    0xYOUR_WALLET_ADDRESS
    price     0.01
    network   base
    dry_run   true
}

This logs everything but skips payment verification. Crawlers that send an X-PAYMENT header will get through as if they paid.

Configuration reference

x402 {
    pay_to              0xADDRESS           # (required) Your EVM wallet address
    price               0.01                # (required) Price in USDC per request
    network             base                # Blockchain: "base" (mainnet) or "base-sepolia" (testnet)

    facilitator_url     https://x402.org/facilitator   # Payment processor endpoint
    facilitator_key     {env.CDP_API_KEY}              # API key (if using a private facilitator)

    exempt              /robots.txt         # Path to skip (repeatable)
    exempt              /favicon.ico
    exempt              /.well-known/*

    cloudflare_compat   true                # Emit crawler-price/crawler-charged headers
    db_path             /var/lib/caddy/x402-payments.db   # SQLite audit log
    dry_run             false               # Skip real payments (testing)
    description         "My site content"   # Description shown in 402 response
    max_timeout         60                  # Max payment timeout in seconds
    crawler_pattern     MyCustomBot         # Extra regex pattern to match (repeatable)
}
Exempt path patterns
  • Exact match: exempt /robots.txt
  • Wildcard (prefix): exempt /.well-known/* matches /.well-known/anything/nested
  • Glob: exempt /static/*.css
Audit trail

Every payment attempt (successful or not) is logged to a SQLite database. You can query it:

sqlite3 /var/lib/caddy/x402-payments.db "SELECT timestamp, path, crawler_name, payer, amount, success, tx_hash FROM payments ORDER BY timestamp DESC LIMIT 20"

Pricing strategy

The price is per request in USDC. Some examples:

Price What it means
0.001 $0.001 per page — cheap, mostly symbolic
0.01 $0.01 per page — reasonable for blog posts
0.10 $0.10 per page — premium content
1.00 $1.00 per page — high-value data

A crawler scraping 1,000 pages at $0.01 each = $10. At $0.10 each = $100.

Deployment

Using the Makefile
# Build
make build

# Run tests
make test

# Deploy to your server (set environment variables)
DEPLOY_HOST=user@yourserver.com DEPLOY_KEY=~/.ssh/your_key make deploy
Manual deployment
# Build locally
xcaddy build --with github.com/paolobietolini/caddy-x402=./ --output ./caddy-x402

# Copy to server
scp ./caddy-x402 user@server:/tmp/caddy-x402

# On the server
sudo systemctl stop caddy
sudo cp /usr/bin/caddy /usr/bin/caddy.backup
sudo cp /tmp/caddy-x402 /usr/bin/caddy
sudo chmod +x /usr/bin/caddy
sudo systemctl start caddy

Development

# Run tests
go test ./... -v

# Run tests with race detector
go test ./... -race

# Format code
go fmt ./...

How x402 works (the protocol)

x402 is an open protocol by Coinbase that gives meaning to HTTP status code 402 ("Payment Required"). It works like this:

  1. Server returns 402 with a JSON body describing what payment it accepts (amount, currency, wallet address, network)
  2. Client signs a payment using EIP-3009 TransferWithAuthorization — this authorizes a USDC transfer without actually executing it yet
  3. Client retries the request with the signed payment in the X-PAYMENT header (base64-encoded)
  4. Server forwards to a facilitator (x402.org) which verifies the signature and submits the on-chain transaction
  5. USDC moves on-chain from the client's wallet to the server's wallet
  6. Server serves the content and includes an X-PAYMENT-RESPONSE header with the transaction hash

The payment happens on Base, an Ethereum L2 built by Coinbase. Transaction fees are fractions of a cent.

License

BSD-3-Clause

Documentation

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

This section is empty.

Types

type Config 

type Config struct {
	// PayTo is the EVM address that receives USDC payments.
	PayTo string `json:"pay_to"`

	// Price in USDC (human-readable, e.g. "0.01").
	Price string `json:"price"`

	// Network is the blockchain network identifier (e.g. "base", "base-sepolia").
	Network string `json:"network"`

	// FacilitatorURL is the base URL of the x402 facilitator service.
	FacilitatorURL string `json:"facilitator_url,omitempty"`

	// FacilitatorKey is the API key for the facilitator (optional, for authenticated facilitators).
	// Redacted from JSON serialization via custom MarshalJSON on the parent struct.
	FacilitatorKey string `json:"-"`

	// ExemptPaths are path patterns that bypass the paywall.
	ExemptPaths []string `json:"exempt_paths,omitempty"`

	// CloudflareCompat enables crawler-price/crawler-charged header emission.
	CloudflareCompat bool `json:"cloudflare_compat"`

	// DBPath is the path to the SQLite audit trail database.
	DBPath string `json:"db_path,omitempty"`

	// DryRun when true skips actual settlement (useful for testing).
	DryRun bool `json:"dry_run"`

	// Description is a human-readable description of the protected resource.
	Description string `json:"description,omitempty"`

	// MaxTimeoutSeconds is the max time allowed for payment completion.
	MaxTimeoutSeconds int `json:"max_timeout_seconds,omitempty"`

	// CustomCrawlerPatterns are additional regex patterns to match crawler UAs.
	CustomCrawlerPatterns []string `json:"custom_crawler_patterns,omitempty"`
}

Config holds all directive settings parsed from the Caddyfile.

type CrawlerDetector 

type CrawlerDetector struct {
	// contains filtered or unexported fields
}

CrawlerDetector holds compiled regex patterns for matching AI crawler User-Agents.

func NewCrawlerDetector 

func NewCrawlerDetector(customPatterns []string) (*CrawlerDetector, error)

NewCrawlerDetector creates a detector with built-in patterns plus any custom ones.

func (*CrawlerDetector) IsCrawler 

func (d *CrawlerDetector) IsCrawler(userAgent string) bool

IsCrawler returns true if the User-Agent matches any known AI crawler pattern.

func (*CrawlerDetector) MatchedPattern 

func (d *CrawlerDetector) MatchedPattern(userAgent string) string

MatchedPattern returns the first matching pattern name, or empty string.

type FacilitatorClient 

type FacilitatorClient struct {
	// contains filtered or unexported fields
}

FacilitatorClient handles HTTP communication with the x402 facilitator service.

func NewFacilitatorClient 

func NewFacilitatorClient(baseURL, apiKey string) *FacilitatorClient

NewFacilitatorClient creates a new facilitator HTTP client.

func (*FacilitatorClient) Settle 

func (c *FacilitatorClient) Settle(ctx context.Context, payload *PaymentPayload, reqs PaymentRequirements) (*SettlementResponse, error)

Settle calls the facilitator /settle endpoint.

func (*FacilitatorClient) Verify 

func (c *FacilitatorClient) Verify(ctx context.Context, payload *PaymentPayload, reqs PaymentRequirements) (*VerifyResponse, error)

Verify calls the facilitator /verify endpoint.

type PaymentPayload 

type PaymentPayload struct {
	X402Version int             `json:"x402Version"`
	Scheme      string          `json:"scheme"`
	Network     string          `json:"network"`
	Payload     json.RawMessage `json:"payload"`
}

PaymentPayload is the decoded X-PAYMENT header (base64 JSON).

type PaymentRecord 

type PaymentRecord struct {
	ID          int64     `json:"id"`
	Timestamp   time.Time `json:"timestamp"`
	Path        string    `json:"path"`
	UserAgent   string    `json:"user_agent"`
	CrawlerName string    `json:"crawler_name"`
	Payer       string    `json:"payer"`
	Amount      string    `json:"amount"`
	Network     string    `json:"network"`
	TxHash      string    `json:"tx_hash"`
	Success     bool      `json:"success"`
	ErrorReason string    `json:"error_reason,omitempty"`
	RemoteAddr  string    `json:"remote_addr"`
	DryRun      bool      `json:"dry_run"`
}

PaymentRecord represents a single payment event in the audit trail.

type PaymentRequirements 

type PaymentRequirements struct {
	Scheme            string                 `json:"scheme"`
	Network           string                 `json:"network"`
	MaxAmountRequired string                 `json:"maxAmountRequired"`
	Asset             string                 `json:"asset"`
	PayTo             string                 `json:"payTo"`
	Resource          string                 `json:"resource"`
	Description       string                 `json:"description"`
	MimeType          string                 `json:"mimeType,omitempty"`
	OutputSchema      interface{}            `json:"outputSchema"`
	MaxTimeoutSeconds int                    `json:"maxTimeoutSeconds"`
	Extra             map[string]interface{} `json:"extra"`
}

PaymentRequirements defines a single acceptable payment method.

type PaymentRequirementsResponse 

type PaymentRequirementsResponse struct {
	X402Version int                   `json:"x402Version"`
	Error       string                `json:"error"`
	Accepts     []PaymentRequirements `json:"accepts"`
}

PaymentRequirementsResponse is the 402 response body per x402 spec v1.

type PaymentStore 

type PaymentStore interface {
	// Init sets up the store (create tables, etc.).
	Init() error

	// Record inserts a payment record.
	Record(rec PaymentRecord) error

	// Close shuts down the store.
	Close() error
}

PaymentStore is the interface for persisting payment audit records.

type SQLiteStore 

type SQLiteStore struct {
	// contains filtered or unexported fields
}

SQLiteStore implements PaymentStore using pure-Go SQLite.

func NewSQLiteStore 

func NewSQLiteStore(dbPath string) *SQLiteStore

NewSQLiteStore creates a new SQLite-backed payment store.

func (*SQLiteStore) Close 

func (s *SQLiteStore) Close() error

Close closes the database connection. Safe to call multiple times.

func (*SQLiteStore) Init 

func (s *SQLiteStore) Init() error

Init opens the database and creates the payments table if needed.

func (*SQLiteStore) Record 

func (s *SQLiteStore) Record(rec PaymentRecord) error

Record inserts a payment record into the database.

type SettleRequest 

type SettleRequest = VerifyRequest

SettleRequest is the request body for POST /settle (same structure as verify).

type SettlementResponse 

type SettlementResponse struct {
	Success     bool   `json:"success"`
	ErrorReason string `json:"errorReason,omitempty"`
	Transaction string `json:"transaction"`
	Network     string `json:"network"`
	Payer       string `json:"payer"`
}

SettlementResponse is included in the X-PAYMENT-RESPONSE header.

type VerifyRequest 

type VerifyRequest struct {
	PaymentPayload      *PaymentPayload     `json:"paymentPayload"`
	PaymentRequirements PaymentRequirements `json:"paymentRequirements"`
}

VerifyRequest is the request body for POST /verify.

type VerifyResponse 

type VerifyResponse struct {
	IsValid       bool   `json:"isValid"`
	InvalidReason string `json:"invalidReason,omitempty"`
	Payer         string `json:"payer,omitempty"`
}

VerifyResponse is the response from POST /verify.

type X402 

type X402 struct {
	Config
	// contains filtered or unexported fields
}

X402 is a Caddy HTTP middleware that implements the x402 payment protocol, requiring AI crawlers to pay USDC for content access.

func (X402) CaddyModule 

func (X402) CaddyModule() caddy.ModuleInfo

CaddyModule returns the Caddy module information.

func (*X402) Cleanup 

func (x *X402) Cleanup() error

Cleanup releases resources.

func (*X402) Provision 

func (x *X402) Provision(ctx caddy.Context) error

Provision sets up the module.

func (*X402) ServeHTTP 

func (x *X402) ServeHTTP(w http.ResponseWriter, r *http.Request, next caddyhttp.Handler) error

ServeHTTP implements caddyhttp.MiddlewareHandler.

func (*X402) UnmarshalCaddyfile 

func (x *X402) UnmarshalCaddyfile(d *caddyfile.Dispenser) error

UnmarshalCaddyfile implements caddyfile.Unmarshaler. 

x402 {
    pay_to      0xADDRESS
    price       0.01
    network     base
    facilitator_url https://x402.org/facilitator
    facilitator_key {env.CDP_API_KEY}
    exempt /robots.txt
    exempt /.well-known/*
    cloudflare_compat on
    db_path /var/lib/caddy/x402-payments.db
    dry_run off
    description "My site content"
    max_timeout 60
    crawler_pattern MyBot
}

func (*X402) Validate 

func (x *X402) Validate() error

Validate ensures the configuration is valid.

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.