httpsink

README

HTTP Request Sink 🚀

A lightweight HTTP request inspection tool that captures, displays, and forwards incoming HTTP requests in real-time. Built with Go and featuring a beautiful Material Design 3 web interface.

Features

• Real-time Request Capture - Monitor incoming HTTP requests via Server-Sent Events (SSE)
• Request/Response Inspection - View complete request and response data including:
  • Headers
  • Cookies
  • Query parameters
  • Body content with multiple format views (Raw, JSON, YAML, HTML, Hex, Base64)
  • Multipart form data with file previews
  • Images (displayed inline)
  • PDFs (embedded viewer)
• Multiple Response Modes:
  • ack - Acknowledge receipt (200 OK)
  • echo - Echo request body back
  • proxy - Forward requests to another server
• Persistent Storage - Requests saved to localStorage for offline viewing
• Search & Filter - Search requests by URL
• Copy to Clipboard - Easy copying of body content
• Console Output - Formatted table output in terminal (with -verbose flag)

Installation

Build from Source

go build -o httpsink .

Using Docker

Pull the latest image:

docker pull ghcr.io/hhxiao/httpsink:latest

Or pull a specific version:

docker pull ghcr.io/hhxiao/httpsink:v1.0.0

Usage

Running with Docker

Basic usage:

docker run --rm -p 9000:9000 ghcr.io/hhxiao/httpsink:latest

With custom port:

docker run --rm -p 8080:8080 ghcr.io/hhxiao/httpsink:latest -port 8080

With configuration file:

docker run --rm -p 9000:9000 \
  -v $(pwd)/httpsink.yaml:/app/httpsink.yaml \
  ghcr.io/hhxiao/httpsink:latest

In proxy mode:

docker run --rm -p 9000:9000 \
  ghcr.io/hhxiao/httpsink:latest \
  -mode proxy -proxy-target https://api.example.com

With verbose output:

docker run --rm -p 9000:9000 \
  ghcr.io/hhxiao/httpsink:latest \
  -verbose

Using Makefile:

make docker-build
make docker-run

Usage

Basic Usage

./httpsink -port 9000

Command Line Options

-port int
    Port to listen on (default 9000)
-mode string
    Sink mode: ack, echo, or proxy (default "ack")
-proxy-target string
    Proxy target URL (for proxy mode)
-verbose
    Enable verbose logging to console
-ack-status int
    Status code to return in ack mode (default 200)
-filter-path string
    Only capture requests matching these paths (comma-separated, supports * wildcard)
-exclude-path string
    Exclude requests matching these paths (comma-separated, supports * wildcard)
-filter-method string
    Only capture requests with these methods (comma-separated, e.g., GET,POST)
-exclude-method string
    Exclude requests with these methods (comma-separated, e.g., GET,OPTIONS)
-config string
    Configuration file (YAML or JSON)
-show-config
    Show effective configuration and exit

Configuration Files

Instead of using many command-line flags, you can use a configuration file.

Supported Formats

• YAML (.yaml, .yml)
• JSON (.json)

Default Locations

httpsink automatically loads configuration from (in order):

1. ./httpsink.yaml, ./httpsink.yml, ./httpsink.json (current directory)
2. ~/.httpsink.yaml, ~/.httpsink.yml, ~/.httpsink.json (home directory)
3. /etc/httpsink.yaml, /etc/httpsink.yml, /etc/httpsink.json (system-wide)

Example Configuration (YAML)

# httpsink.yaml

# Server settings
port: 9000
mode: echo
verbose: true

# Response settings (for ack mode)
ack-status: 200

# Request filtering
filter-path: "/api/*"
exclude-path: "/health,/metrics,/ready"
filter-method: "POST,PUT,DELETE"
exclude-method: "OPTIONS"

# Proxy settings (only used when mode is "proxy")
proxy-target: "https://backend.example.com"

Example Configuration (JSON)

{
  "port": 9000,
  "mode": "echo",
  "verbose": true,
  "ack-status": 200,
  "filter-path": "/api/*",
  "exclude-path": "/health,/metrics,/ready",
  "filter-method": "POST,PUT,DELETE",
  "exclude-method": "OPTIONS"
}

Using Configuration Files

Use a specific config file:

./httpsink -config=httpsink.yaml

Show effective configuration:

./httpsink -show-config

Override config with command-line flags:

# Config sets port 9000, but flag overrides to 8080
./httpsink -config=httpsink.yaml -port=8080

Configuration Precedence

1. Command-line flags (highest priority)
2. Configuration file
3. Default values (lowest priority)

Example Configurations

Development setup:

# ~/.httpsink.yaml
port: 9000
mode: echo
verbose: true
exclude-method: "OPTIONS"

Production proxy:

# /etc/httpsink.yaml
port: 80
mode: proxy
proxy-target: https://api.production.com
verbose: false
filter-path: "/api/*"
exclude-path: "/health,/metrics"

Testing setup:

# ./httpsink.yaml
port: 9000
mode: ack
ack-status: 202
filter-method: "POST"

Examples

Run in acknowledgment mode (default):

./httpsink -port 9000

Note: In ack mode, the server only acknowledges receipt with 200 OK and no response body. Use echo mode to see response body content.

Return custom status code in ack mode:

./httpsink -port 9000 -ack-status 202

Example: Use -ack-status 202 for "Accepted", -ack-status 404 for testing error handling, etc.

Filter requests by path:

./httpsink -port 9000 -filter-path "/api/*"

Example: Only capture requests to paths starting with /api/

Exclude health check endpoints:

./httpsink -port 9000 -exclude-path "/health,/ready,/metrics"

Example: Reduce noise by excluding frequent health check requests

Filter by HTTP method:

./httpsink -port 9000 -filter-method "POST,PUT"

Example: Only capture write operations

Exclude OPTIONS requests:

./httpsink -port 9000 -exclude-method "OPTIONS,GET"

Example: Skip CORS preflight and read-only requests

Combine multiple filters:

./httpsink -port 9000 \
  -filter-path "/api/*" \
  -exclude-path "/api/internal/*" \
  -exclude-method "OPTIONS"

Example: Capture API requests except internal endpoints and CORS preflights

Run in echo mode (returns request body):

./httpsink -port 9000 -mode echo

Tip: Use echo mode for testing to see both request and response bodies in the web interface.

Run in proxy mode (forward to another server):

./httpsink -port 9000 -mode proxy -proxy-target https://api.example.com

Enable verbose console output:

./httpsink -port 9000 -verbose

Request Filtering

Use filtering options to control which requests are captured, reducing noise in high-traffic environments.

Path Filtering

Include only specific paths:

./httpsink -filter-path "/api/*"

Exclude specific paths:

./httpsink -exclude-path "/health,/metrics,/ready"

Wildcard patterns:

• * matches any sequence of characters
• /api/* matches /api/users, /api/users/123, etc.
• /api/v* matches /api/v1, /api/v2/users, etc.

Method Filtering

Include only specific methods:

./httpsink -filter-method "POST,PUT"

Exclude specific methods:

./httpsink -exclude-method "OPTIONS,GET"

Filter Priority

Filters are applied in this order:

1. Exclude methods (highest priority) - Requests with these methods are always excluded
2. Exclude paths - Requests matching these paths are excluded
3. Filter methods - Only requests with these methods are included
4. Filter paths - Only requests matching these paths are included

If no filters are specified, all requests are captured.

Examples

Capture only API write operations:

./httpsink -filter-path "/api/*" -filter-method "POST,PUT,DELETE"

Exclude health checks and CORS preflights:

./httpsink -exclude-path "/health,/ready,/metrics" -exclude-method "OPTIONS"

Debug specific endpoint:

./httpsink -filter-path "/api/users/*/orders" -filter-method "POST"

Web Interface

Open your browser and navigate to http://localhost:9000 to access the web interface.

Web Interface Screenshot

The web interface showing request list (left) and request/response details (right)

Features

• Request List (left panel) - Shows all captured requests with method, path, and timestamp
• Request Details (right panel) - Displays detailed information with tabs:
  • Body - Request/response body with format switching
  • Query - Query parameters
  • Headers - HTTP headers
  • Cookies - Request/response cookies
• Format Options (for body content):
  • Raw - Plain text view with line numbers
  • JSON - Collapsible tree view
  • YAML - Collapsible tree view
  • HTML - Rendered in iframe
  • Image - Displayed inline with dimensions
  • PDF - Embedded viewer
  • Multipart - Individual part display with headers
  • Hex - Traditional hex dump with ASCII
  • Base64 - Base64 encoded view with copy button

Response Section

The response section shows:

• Status code with color-coded badge (green=2xx, blue=3xx, orange=4xx, red=5xx)
• Response headers
• Response cookies
• Response body (with same format options as request)

Console Output

When run with the -verbose flag, httpsink displays captured requests in a formatted table in the terminal.

Console Output Screenshot

Console output showing request and response data in formatted tables

Console Features

• Formatted table layout with proper borders and alignment
• Separate sections for request and response
• Color-coded method badges (GET=green, POST=blue, PUT=yellow, etc.)
• Support for displaying:
  • Headers and cookies
  • Query parameters
  • Body content (text, JSON, multipart, binary)
  • Images (ASCII art preview)
  • PDFs (first page text preview)

API Endpoints

GET /

Returns the web interface HTML and handles Server-Sent Events (SSE) for real-time request updates. The same endpoint serves both the initial HTML page and the SSE stream (based on the Accept header).

/* (any other path)

Captures and processes incoming HTTP requests

Data Structure

Requests are stored with the following structure:

{
  "id": "1708675200000000000",
  "timestamp": "2024-02-23T12:00:00Z",
  "request": {
    "method": "POST",
    "url": "/test",
    "path": "/test",
    "headers": {...},
    "cookies": [...],
    "query": {...},
    "contentType": "application/json",
    "body": {...},
    "rawBody": "{\"key\":\"value\"}",
    "isBinary": false
  },
  "response": {
    "statusCode": 200,
    "statusLine": "200 OK",
    "headers": {...},
    "cookies": [...],
    "contentType": "application/json",
    "body": {...},
    "rawBody": "{\"result\":\"success\"}",
    "isBinary": false
  }
}

Development

Project Structure

httpsink/
├── main.go           # Main application and HTTP handlers
├── dump/
│   ├── console_dump.go   # Console table output
│   ├── http_dump.go      # HTTP handler for web interface
│   ├── data.go           # Data structures and parsing
│   └── index.html        # Web interface (embedded)
├── go.mod
├── go.sum
└── README.md

Building

go build -o httpsink .

Using as a Library

You can integrate httpsink's request capturing functionality into your own Go applications.

Basic Integration

package main

import (
    "net/http"
    "github.com/hhxiao/httpsink/dump"
)

var consoleDumper *dump.ConsoleDumper
var httpDumper *dump.HttpDumper

func main() {
    // Initialize dumpers
    consoleDumper = dump.NewConsoleDumper(
        dump.WithVerbose(true),
        dump.WithTableWidth(82),
    )
    httpDumper = dump.NewHttpDumper()

    // Wrap your handlers
    http.HandleFunc("/api/", func(w http.ResponseWriter, r *http.Request) {
        // Your handler logic here
        w.WriteHeader(http.StatusOK)
    })

    // Capture requests at root
    http.HandleFunc("/", func(w http.ResponseWriter, r *http.Request) {
        if r.URL.Path == "/" {
            httpDumper.Serve(w, r)
            return
        }
        // Your default handler
        w.WriteHeader(http.StatusNotFound)
    })

    http.ListenAndServe(":9000", nil)
}

Custom Request Wrapper

func wrapHandler(handler http.HandlerFunc) http.HandlerFunc {
    return func(w http.ResponseWriter, r *http.Request) {
        // Read request body
        data, _ := io.ReadAll(r.Body)
        r.Body = io.NopCloser(bytes.NewBuffer(data))

        // Call your handler
        handler(w, r)

        // Create and broadcast dump
        d, err := dump.NewDump(r, &http.Response{
            StatusCode: 200,
            Header:     w.Header(),
        }, data, nil)
        
        if err == nil {
            httpDumper.Dump(d)
            consoleDumper.Dump(d)
        }
    }
}

// Usage
http.HandleFunc("/api/users", wrapHandler(usersHandler))

Accessing Request Data

The dump.Dump struct provides structured access to request/response data:

d, _ := dump.NewDump(req, res, requestBody, responseBody)

// Request data
fmt.Println(d.RequestDump.Method)
fmt.Println(d.RequestDump.URL)
fmt.Println(d.RequestDump.Headers)
fmt.Println(d.RequestDump.Body)

// Response data
fmt.Println(d.ResponseDump.StatusCode)
fmt.Println(d.ResponseDump.Headers)
fmt.Println(d.ResponseDump.Body)

Custom Console Dumper Options

// Enable verbose output
dumper := dump.NewConsoleDumper(dump.WithVerbose(true))

// Set table width
dumper := dump.NewConsoleDumper(dump.WithTableWidth(100))

// Combine options
dumper := dump.NewConsoleDumper(
    dump.WithVerbose(true),
    dump.WithTableWidth(100),
)

License

MIT License

Contributing

1. Fork the repository
2. Create a feature branch
3. Make your changes
4. Submit a pull request

Support

For issues and feature requests, please open an issue on the GitHub repository.

Documentation

Overview

Package main implements the HTTP Request Sink application. It captures incoming HTTP requests and displays them via console and web interface.