httpreq

README

httpreq

JSON-over-HTTP convenience layer on top of net/http. Stdlib-only. One function, one options pattern, typed errors, and dependency-free observability. No surprises.

import "github.com/ubgo/httpreq"

var resp UserResponse
_, err := httpreq.Do(ctx, "https://api.example.com/users",
    httpreq.WithMethod(http.MethodPost),
    httpreq.WithJSONBody(req),
    httpreq.WithBearerToken(token),
    httpreq.WithTimeout(5 * time.Second),
    httpreq.WithResponseInto(&resp),
)

What it does

• Builds the request from options.
• Adds bearer or basic auth, Content-Type, a default User-Agent, custom headers, and query params.
• Encodes the body as JSON, URL-encoded form, or raw bytes.
• Sends with a per-request timeout and your context.
• Reads the body once.
• Returns *HTTPError for non-2xx (with the raw body captured), and can decode a structured error shape.
• Routes the response into a JSON target, a raw []byte, or both.

What it doesn't do

• Retries / backoff / circuit breaking — install those at the transport layer (http.RoundTripper).
• GraphQL helpers — those live in separate packages.
• A global default client — pass WithHTTPClient if you need pooling.

Why httpreq?

The standard library is the right foundation, but the most common service call — marshal a body, set headers, send with a timeout, decode JSON, turn non-2xx into an error — is ~15 lines of boilerplate every time. Full-featured clients solve that by pulling in a dependency tree and a large API surface. httpreq takes the opposite bet: a single Do call, a handful of composable options, and zero third-party dependencies — so it never conflicts with your other modules and never surprises you at upgrade time.

	httpreq	net/http (raw)	resty / req
Third-party dependencies	0	0	several
Lines for a JSON POST + decode	~5	~15	~5
Typed non-2xx error with raw body	✅ HTTPError	❌ manual	⚠️ varies
Built-in observability (trace/slog/timing)	✅ dependency-free	❌	✅
API surface to learn	tiny	n/a	large
Connection pooling	✅ via WithHTTPClient	✅	✅

Reach for a full client when you need retries, rate limiting, or protocol helpers out of the box. Reach for httpreq when you want the stdlib with the boilerplate removed and nothing else added.

Recipes

Common day-to-day patterns.

OAuth2 token request — HTTP Basic client auth plus a form-encoded body:

form := url.Values{}
form.Set("grant_type", "client_credentials")
form.Set("scope", "read write")

var tok struct {
    AccessToken string `json:"access_token"`
    ExpiresIn   int    `json:"expires_in"`
}
_, err := httpreq.Do(ctx, "https://auth.example.com/oauth/token",
    httpreq.WithMethod(http.MethodPost),
    httpreq.WithBasicAuth(clientID, clientSecret),
    httpreq.WithFormBody(form),
    httpreq.WithResponseInto(&tok),
)

Fetch a non-JSON body — HTML, text, XML, or binary, captured verbatim:

var raw []byte
_, err := httpreq.Do(ctx, "https://example.com/page.html",
    httpreq.WithResponseBytes(&raw),
)
// raw holds the exact response bytes, whatever the content type.

Handle a structured error body — decode the API's error shape on non-2xx while still getting the typed HTTPError:

var apiErr struct {
    Code    string `json:"code"`
    Message string `json:"message"`
}
_, err := httpreq.Do(ctx, "https://api.example.com/users",
    httpreq.WithMethod(http.MethodPost),
    httpreq.WithJSONBody(newUser),
    httpreq.WithResponseInto(&created),
    httpreq.WithErrorInto(&apiErr),   // populated only on 4xx/5xx
)
var herr *httpreq.HTTPError
if errors.As(err, &herr) {
    log.Printf("api rejected: %d %s — %s", herr.StatusCode, apiErr.Code, apiErr.Message)
}

Identify your client — set a User-Agent (some hosts reject an empty one):

_, err := httpreq.Do(ctx, url, httpreq.WithUserAgent("my-service/1.4.2"))
// Unset, httpreq sends "httpreq/<version>"; pass "" to send none.

Download a large file — stream to disk (and checksum) without buffering in memory:

f, _ := os.Create("backup.zip")
defer f.Close()
h := sha256.New()
_, err := httpreq.Do(ctx, "https://example.com/backup.zip",
    httpreq.WithResponseWriter(io.MultiWriter(f, h)),
)
// f holds the file; h.Sum(nil) is its checksum — one pass, constant memory.

Sign a request — compute an auth header over the fully assembled request:

_, err := httpreq.Do(ctx, url,
    httpreq.WithJSONBody(payload),
    httpreq.WithRequest(func(req *http.Request) error {
        req.Header.Set("X-Signature", sign(req)) // sees final method, path, headers, body
        return nil                               // returning an error aborts before send
    }),
)

Conditional request — treat 304 Not Modified as success, not an error:

_, err := httpreq.Do(ctx, url,
    httpreq.WithHeader("If-None-Match", etag),
    httpreq.WithExpectStatus(http.StatusNotModified),
)
// err is nil on 304; check the response status to branch on cache hit.

Options

Option	Effect
WithMethod(string)	HTTP method. Default: GET.
WithHeader(k, v)	Add a header. Repeat for multi-value.
WithHeaders(http.Header)	Add a whole header set at once.
WithUserAgent(string)	Set User-Agent. Default: httpreq/<version>. Pass "" to suppress.
WithBearerToken(string)	Authorization: Bearer <t>. No-op when empty.
WithBasicAuth(user, pass)	Authorization: Basic <base64>. Overrides any earlier auth.
WithQueryParam(k, v)	Append a query string parameter. Repeat for multi-value.
WithQuery(url.Values)	Append a whole set of query parameters at once.
WithJSONBody(any)	Marshal body as JSON, set Content-Type. nil clears.
WithFormBody(url.Values)	URL-encoded application/x-www-form-urlencoded body. nil clears.
WithRawBody([]byte)	Send bytes verbatim. Caller sets Content-Type.
WithTimeout(time.Duration)	Default: 30s. Set to 0 to use ctx deadline only.
WithHTTPClient(*http.Client)	Override the underlying client.
WithResponseInto(any)	JSON-decode response into v (must be a pointer).
WithResponseBytes(*[]byte)	Capture the raw response body (any status, any content type).
WithResponseWriter(io.Writer)	Stream a successful body to a writer instead of buffering (large downloads).
WithErrorInto(any)	JSON-decode a structured error body on non-2xx into v.
WithExpectStatus(...int)	Treat listed non-2xx codes (e.g. 304) as success.
WithRequest(func(*http.Request) error)	Mutate the built request before send (signing, Host, cookies). Repeatable.
WithObserver(func(ctx, Trace))	Callback fired once per attempt with metadata (see Observability). Repeatable.
WithConnTrace()	Fill DNS/Connect/TLS/TTFB timings on the Trace via httptrace.
WithCurl(func(curl string))	Callback fired with the request as a runnable curl command, just before send.

Dump as curl

Get the exact request as a runnable curl command — to print, log, drop in a bug report, or replay on the command line. Pick by what you have in hand:

1. You're using httpreq's options → Curl — get the string without sending.

Build the command from the same options you'd pass to Do, but nothing goes on the wire:

cmd, _ := httpreq.Curl(ctx, "https://api/users",
    httpreq.WithMethod(http.MethodPost),
    httpreq.WithJSONBody(payload),
)
fmt.Println(cmd)
// curl -X POST -H 'Content-Type: application/json' --data-raw '{...}' 'https://api/users'

2. You're calling Do → WithCurl — log exactly what gets sent.

The callback fires just before the request goes out, so there's no option duplication and no separate build step:

_, _ = httpreq.Do(ctx, "https://api/users",
    httpreq.WithJSONBody(payload),
    httpreq.WithCurl(func(cmd string) { log.Println(cmd) }),
)

3. You already hold a plain *http.Request → RequestCurl — render that.

This is the low-level primitive the other two are built on. Use it when the request came from somewhere else — a middleware, a custom http.RoundTripper, another library — and you're not going through Do at all:

req, _ := http.NewRequest(http.MethodPost, "https://api/users", body)
req.Header.Set("Authorization", "Bearer "+token)

cmd, _ := httpreq.RequestCurl(req)

If you only ever call httpreq.Do, reach for WithCurl and you'll never need RequestCurl directly.

In all three, headers are emitted in sorted order and every value is shell-quoted, so the command survives special characters and is stable across runs. The request body is read without consuming it, so a request rendered mid-Do still sends normally.

Security: the rendered command is a faithful, full dump — it includes the Authorization header, cookies, and body. That's the point, but it means secrets land in whatever you do with the string. Redact before writing to a shared log. (This is the opposite trade-off from Trace, which is metadata-only by design.)

Error types

Non-2xx responses surface *HTTPError:

_, err := httpreq.Do(ctx, "https://api/x")
var herr *httpreq.HTTPError
if errors.As(err, &herr) {
    log.Printf("status=%d body=%s", herr.StatusCode, herr.Body)
}

Transport errors (DNS, connection, timeout, ctx-cancel) come back as wrapped errors from http.Client. Decode errors are wrapped JSON errors.

Observability

Register an observer to receive a Trace once per request attempt — on success and on every failure path (non-2xx, network error, decode error). The Trace carries metadata only (method, final URL, status, request/response byte counts, total duration, the typed error, attempt number). It never contains bodies or headers, so nothing sensitive leaks into your logs by accident. This is the single hook for all three observability pillars: route the Trace to a logger, a metrics recorder, or a span.

_, err := httpreq.Do(ctx, "https://api/x",
    httpreq.WithObserver(func(ctx context.Context, t httpreq.Trace) {
        // metrics, logging, spans — your call
    }),
)

Structured logging (log/slog)

SlogObserver is a batteries-included adapter — still stdlib-only. Failures log at ERROR regardless of the level you pass.

logger := slog.Default()
_, err := httpreq.Do(ctx, "https://api/x",
    httpreq.WithObserver(httpreq.SlogObserver(logger, slog.LevelInfo)),
)

Connection-phase timing (net/http/httptrace)

Add WithConnTrace() to populate DNS/Connect/TLS/TTFB on the Trace — the phase breakdown you actually need when debugging "why is this call slow." Connect and TLS stay zero on a reused keep-alive connection because no new dial or handshake happened.

_, err := httpreq.Do(ctx, "https://api/x",
    httpreq.WithConnTrace(),
    httpreq.WithObserver(func(_ context.Context, t httpreq.Trace) {
        log.Printf("dns=%s connect=%s tls=%s ttfb=%s total=%s",
            t.DNS, t.Connect, t.TLS, t.TTFB, t.Duration)
    }),
)

Metrics (Prometheus, etc.)

There is no stdlib metrics API, so the Trace callback is the metrics seam — no dependency is imported on your behalf. Record straight from the callback:

httpreq.WithObserver(func(_ context.Context, t httpreq.Trace) {
    reqDuration.WithLabelValues(t.Method, strconv.Itoa(t.StatusCode)).Observe(t.Duration.Seconds())
})

Distributed tracing (OpenTelemetry)

OTel is not a dependency of this package. Because your context flows through Do, wire tracing at the transport with otelhttp and spans nest correctly — no httpreq-specific glue needed:

client := &http.Client{Transport: otelhttp.NewTransport(http.DefaultTransport)}
_, err := httpreq.Do(ctx, "https://api/x", httpreq.WithHTTPClient(client))

FAQ

Does httpreq have any third-party dependencies? No. go.mod is stdlib-only and stays that way — that's a hard rule, enforced in CI. Everything, including the observability layer, is built on net/http, log/slog, and net/http/httptrace.

How do I add retries or a circuit breaker? Install them at the transport layer and pass the client with WithHTTPClient. Because httpreq wraps a standard *http.Client, any http.RoundTripper-based middleware (retry, tracing, rate limiting) composes without httpreq needing to know about it.

How do I get request logging, metrics, or tracing? Register WithObserver to receive a Trace (method, status, byte counts, duration, typed error) once per request. Use the built-in SlogObserver for log/slog logging, feed the Trace into a Prometheus histogram, or add WithConnTrace() for DNS/TLS/TTFB timing. OpenTelemetry works via a transport — see Observability. No dependency is added on your behalf.

Will anything sensitive end up in my logs? No. The Trace passed to observers carries metadata only — never request/response bodies and never headers — so tokens and cookies can't leak by accident. If you need header or body content, install a custom transport where you own the redaction.

How do I see the raw request as a curl command? Use WithCurl(func(cmd string){ ... }) while calling Do to log exactly what's sent, or Curl(ctx, url, opts...) to get the string without sending. If you already hold a plain *http.Request from elsewhere, RequestCurl(req) renders it. See Dump as curl. The output is a full, faithful dump — including Authorization — so redact before logging to a shared sink.

Can I send non-JSON bodies? Yes. Use WithFormBody(url.Values) for application/x-www-form-urlencoded posts, or WithRawBody([]byte) for protobuf or any pre-encoded payload (set Content-Type with WithHeader for raw).

How do I read a non-JSON response? Use WithResponseBytes(&raw) to capture the exact response bytes regardless of content type — HTML, text, XML, binary. It works alongside WithResponseInto and is populated on error responses too. See Recipes.

Can it handle responses larger than memory? Yes. WithResponseWriter(w) streams the body straight to any io.Writer (a file, a hash, an io.MultiWriter) via io.Copy, so memory stays constant regardless of size. It applies to successful responses; error bodies are still buffered into the HTTPError. See Recipes.

Is the API stable? The module is pre-1.0. The surface is small and unlikely to change much, but breaking changes are possible before v1.0.0; after that they require a major version bump.

License

Apache-2.0 — see LICENSE.

---

_{Go HTTP client · JSON API client for Go · net/http wrapper · stdlib-only · zero-dependency · typed HTTP errors · request observability · slog HTTP logging · httptrace timing}

Documentation

Overview

Package httpreq is a small JSON-over-HTTP client convenience layer on top of net/http.

The standard library is fine but verbose for the most common service call: marshal a request body, set headers, send with a timeout, decode a JSON response, and turn non-2xx into an error. This package collapses that into a single options-pattern call:

var resp UserResponse
httpResp, err := httpreq.Do(ctx, "https://api/users",
    httpreq.WithMethod(http.MethodPost),
    httpreq.WithJSONBody(req),
    httpreq.WithBearerToken(token),
    httpreq.WithTimeout(5*time.Second),
    httpreq.WithResponseInto(&resp),
)

Observability is built in and dependency-free: register a callback with WithObserver to receive a Trace (method, status, byte counts, duration, typed error) once per attempt, add WithConnTrace for DNS/TLS/TTFB phase timing via net/http/httptrace, or drop in SlogObserver for structured logging through log/slog. Traces carry metadata only — no bodies, no headers — so nothing sensitive leaks into your logs by accident.

Beyond the basics it covers common day-to-day needs without leaving the standard library: bearer and basic auth (WithBearerToken, WithBasicAuth), JSON / form / raw bodies (WithJSONBody, WithFormBody, WithRawBody), and flexible response handling — decode JSON (WithResponseInto), grab the raw bytes for non-JSON payloads (WithResponseBytes), stream large downloads to a writer (WithResponseWriter), or decode a structured error body on non-2xx (WithErrorInto). WithExpectStatus accepts extra status codes as success (e.g. 304), and WithRequest is an escape hatch to mutate or sign the built request before it is sent.

For debugging, render any request as a runnable curl command: WithCurl hands the string to a callback just before Do sends, Curl returns it as a value without sending, and RequestCurl renders an arbitrary *http.Request.

What's deliberately NOT here:

• Retries, circuit breaking, rate limiting. Use a dedicated transport or library for those concerns.
• GraphQL helpers, Shopify-specific extensions, etc. The lace.io ancestor of this package mixed those in; they have been removed.
• A global default client. Pass WithHTTPClient explicitly if you want to share connection pools.

Index

• Constants
• func Curl(ctx context.Context, endpoint string, opts ...Option) (string, error)
• func Do(ctx context.Context, endpoint string, opts ...Option) (*http.Response, error)
• func RequestCurl(req *http.Request) (string, error)
• func SlogObserver(l *slog.Logger, level slog.Level) func(context.Context, Trace)
• type HTTPError
  • func (e *HTTPError) Error() string
• type Option
  • func WithBasicAuth(user, password string) Option
  • func WithBearerToken(token string) Option
  • func WithConnTrace() Option
  • func WithCurl(fn func(curl string)) Option
  • func WithErrorInto(v any) Option
  • func WithExpectStatus(codes ...int) Option
  • func WithFormBody(values url.Values) Option
  • func WithHTTPClient(c *http.Client) Option
  • func WithHeader(key, value string) Option
  • func WithHeaders(h http.Header) Option
  • func WithJSONBody(body any) Option
  • func WithMethod(m string) Option
  • func WithObserver(fn func(context.Context, Trace)) Option
  • func WithQuery(q url.Values) Option
  • func WithQueryParam(key, value string) Option
  • func WithRawBody(body []byte) Option
  • func WithRequest(fn func(*http.Request) error) Option
  • func WithResponseBytes(b *[]byte) Option
  • func WithResponseInto(v any) Option
  • func WithResponseWriter(w io.Writer) Option
  • func WithTimeout(d time.Duration) Option
  • func WithUserAgent(ua string) Option
• type Trace

Examples

• Curl
• Do
• HTTPError
• RequestCurl
• SlogObserver
• WithBasicAuth
• WithConnTrace
• WithCurl
• WithErrorInto
• WithExpectStatus
• WithFormBody
• WithHeaders
• WithObserver
• WithQuery
• WithRequest
• WithResponseBytes
• WithResponseWriter
• WithUserAgent

Constants

const DefaultUserAgent = "httpreq/" + Version

DefaultUserAgent is sent as the User-Agent header when the caller sets none. To suppress it entirely, set an empty User-Agent via WithUserAgent("").

const Version = "0.2.0"

Version is the package version, used only to build DefaultUserAgent. Bump it on release.

Functions

func Curl

func Curl(ctx context.Context, endpoint string, opts ...Option) (string, error)

Curl builds the request from the given options and returns it rendered as a runnable `curl` command string, WITHOUT sending anything. It is the value form of WithCurl: use it to print a request for docs, a dry run, or a debug log, then send the same options through Do when ready.

SECURITY: the returned command contains the full request including the Authorization header and body — see WithCurl for the redaction note.

Example

ExampleCurl renders a request as a runnable curl command without sending it — handy for logging or reproducing a call on the command line.

package main

import (
	"context"
	"fmt"
	"net/http"

	"github.com/ubgo/httpreq"
)

func main() {
	cmd, err := httpreq.Curl(context.Background(), "https://api.example.com/users",
		httpreq.WithMethod(http.MethodPost),
		httpreq.WithHeader("Accept", "application/json"),
		httpreq.WithJSONBody(map[string]string{"name": "alice"}),
	)
	if err != nil {
		fmt.Println("error:", err)
		return
	}
	fmt.Println(cmd)
}

Output:
curl -X POST -H 'Accept: application/json' -H 'Content-Type: application/json' -H 'User-Agent: httpreq/0.2.0' --data-raw '{"name":"alice"}' 'https://api.example.com/users'

func Do

func Do(ctx context.Context, endpoint string, opts ...Option) (*http.Response, error)

Do builds and sends the request. The returned *http.Response has its Body already drained and closed; the body bytes have been routed into the option supplied via WithResponseInto, if any.

Non-2xx responses return an *HTTPError so the caller can inspect the raw body. JSON decode errors are wrapped and returned.

Example

ExampleDo shows the common case: POST a JSON body and decode the JSON response into a struct.

package main

import (
	"context"
	"fmt"
	"net/http"
	"net/http/httptest"

	"github.com/ubgo/httpreq"
)

func main() {
	srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, _ *http.Request) {
		w.Header().Set("Content-Type", "application/json")
		_, _ = w.Write([]byte(`{"name":"alice"}`))
	}))
	defer srv.Close()

	var out struct {
		Name string `json:"name"`
	}
	_, err := httpreq.Do(context.Background(), srv.URL,
		httpreq.WithMethod(http.MethodPost),
		httpreq.WithJSONBody(map[string]int{"id": 1}),
		httpreq.WithResponseInto(&out),
	)
	if err != nil {
		fmt.Println("error:", err)
		return
	}
	fmt.Println(out.Name)
}

Output:
alice

func RequestCurl

func RequestCurl(req *http.Request) (string, error)

RequestCurl renders any *http.Request as a runnable, copy-pasteable `curl` command string. Headers are emitted in sorted order for stable output; values are shell-quoted so the command survives special characters. The method is included as -X for anything other than GET.

The body is read via req.GetBody when available (as it is for requests built by this package), so the request's own body is left intact and the request stays sendable afterward. If GetBody is nil, req.Body is consumed and then restored with an equivalent reader.

SECURITY: the output reproduces the full request, secrets included. Redact before writing to a shared log.

Example

ExampleRequestCurl renders a plain *http.Request you already hold — built with the standard library, not through httpreq — as a curl command.

package main

import (
	"fmt"
	"net/http"

	"github.com/ubgo/httpreq"
)

func main() {
	req, _ := http.NewRequest(http.MethodGet, "https://api.example.com/users", nil)
	req.Header.Set("Accept", "application/json")

	cmd, err := httpreq.RequestCurl(req)
	if err != nil {
		fmt.Println("error:", err)
		return
	}
	fmt.Println(cmd)
}

Output:
curl -H 'Accept: application/json' 'https://api.example.com/users'

func SlogObserver

func SlogObserver(l *slog.Logger, level slog.Level) func(context.Context, Trace)

SlogObserver returns an observer that logs each completed request through l at the given level. Failures (non-nil Trace.Err) are logged at slog.LevelError regardless of level. Only metadata is logged — never bodies or headers. If l is nil, slog.Default is used.

Wire it in with WithObserver:

httpreq.WithObserver(httpreq.SlogObserver(logger, slog.LevelInfo))

Example

ExampleSlogObserver wires the batteries-included log/slog adapter. Failures log at ERROR regardless of the level passed. Output is not asserted here because log lines carry volatile timing fields.

package main

import (
	"context"
	"log/slog"
	"os"

	"github.com/ubgo/httpreq"
)

func main() {
	logger := slog.New(slog.NewTextHandler(os.Stdout, nil))
	_, _ = httpreq.Do(context.Background(), "https://api.example.com/health",
		httpreq.WithObserver(httpreq.SlogObserver(logger, slog.LevelInfo)),
	)
}

Types

type HTTPError

type HTTPError struct {
	StatusCode int
	Status     string
	Body       []byte
}

HTTPError is returned when the response status is outside 2xx. The raw response body is captured in Body so callers can decode their own error shape if needed.

Example

ExampleHTTPError shows how to recover the status code and raw body from a non-2xx response via errors.As.

package main

import (
	"context"
	"errors"
	"fmt"
	"net/http"
	"net/http/httptest"

	"github.com/ubgo/httpreq"
)

func main() {
	srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, _ *http.Request) {
		w.WriteHeader(http.StatusNotFound)
		_, _ = w.Write([]byte("missing"))
	}))
	defer srv.Close()

	_, err := httpreq.Do(context.Background(), srv.URL)

	var herr *httpreq.HTTPError
	if errors.As(err, &herr) {
		fmt.Printf("status=%d body=%s\n", herr.StatusCode, herr.Body)
	}
}

Output:
status=404 body=missing

func (*HTTPError) Error

func (e *HTTPError) Error() string

type Option

type Option func(*request) error

Option configures a single request.

func WithBasicAuth

func WithBasicAuth(user, password string) Option

WithBasicAuth sets `Authorization: Basic <base64(user:password)>`. It overwrites any Authorization set earlier (e.g. via WithBearerToken) — last auth option wins.

Example

ExampleWithBasicAuth sends HTTP Basic credentials. It overrides any bearer token set earlier — the last auth option wins.

package main

import (
	"context"

	"github.com/ubgo/httpreq"
)

func main() {
	_, _ = httpreq.Do(context.Background(), "https://api.example.com/private",
		httpreq.WithBasicAuth("alice", "s3cret"),
	)
}

func WithBearerToken

func WithBearerToken(token string) Option

WithBearerToken sets `Authorization: Bearer <token>`. No-op if the token is empty.

func WithConnTrace

func WithConnTrace() Option

WithConnTrace enables connection-phase timing via net/http/httptrace, populating the DNS/Connect/TLS/TTFB fields of the Trace passed to observers. It has a small per-request overhead and only matters when an observer is also registered. On a reused keep-alive connection the Connect and TLS fields stay zero because no new dial or handshake ran.

Example

ExampleWithConnTrace adds connection-phase timing (DNS/Connect/TLS/TTFB) to the Trace. Connect and TLS stay zero on a reused keep-alive connection. Output is not asserted because timings vary per run.

package main

import (
	"context"
	"fmt"

	"github.com/ubgo/httpreq"
)

func main() {
	_, _ = httpreq.Do(context.Background(), "https://api.example.com/health",
		httpreq.WithConnTrace(),
		httpreq.WithObserver(func(_ context.Context, t httpreq.Trace) {
			fmt.Printf("dns=%s connect=%s tls=%s ttfb=%s total=%s\n",
				t.DNS, t.Connect, t.TLS, t.TTFB, t.Duration)
		}),
	)
}

func WithCurl

func WithCurl(fn func(curl string)) Option

WithCurl registers a callback that receives the request rendered as a runnable `curl` command string, invoked once by Do immediately before the request is sent — so it fires even when the send later fails. Use it to log or print exactly what goes on the wire.

SECURITY: the rendered command reproduces the FULL request, including the Authorization header, cookies, and body. That is the point — it is a faithful dump — but it means secrets appear in whatever you do with the string. Redact before logging to a shared sink. A nil fn is ignored.

To obtain the curl string as a value without sending, use Curl; to render an arbitrary request you already hold, use RequestCurl.

Example

ExampleWithCurl logs the exact request Do sends, right before it goes out.

package main

import (
	"context"

	"github.com/ubgo/httpreq"
)

func main() {
	_, _ = httpreq.Do(context.Background(), "https://api.example.com/health",
		httpreq.WithCurl(func(cmd string) {
			// print it, log it, whatever you want
			_ = cmd
		}),
	)
}

func WithErrorInto

func WithErrorInto(v any) Option

WithErrorInto JSON-decodes the response body into v when the status is non-2xx (i.e. when an HTTPError is returned), letting you read a structured error payload. v must be a pointer. The HTTPError is still returned; if the error body is not valid JSON, v is left unchanged and no extra error is raised (inspect HTTPError.Body for the raw bytes).

Example

ExampleWithErrorInto decodes a structured error payload from a non-2xx response while still returning the *HTTPError.

package main

import (
	"context"
	"errors"
	"fmt"
	"net/http"
	"net/http/httptest"

	"github.com/ubgo/httpreq"
)

func main() {
	srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, _ *http.Request) {
		w.WriteHeader(http.StatusBadRequest)
		_, _ = w.Write([]byte(`{"code":"invalid_field","message":"name is required"}`))
	}))
	defer srv.Close()

	var apiErr struct {
		Code    string `json:"code"`
		Message string `json:"message"`
	}
	_, err := httpreq.Do(context.Background(), srv.URL, httpreq.WithErrorInto(&apiErr))

	var herr *httpreq.HTTPError
	if errors.As(err, &herr) {
		fmt.Printf("%d %s: %s\n", herr.StatusCode, apiErr.Code, apiErr.Message)
	}
}

Output:
400 invalid_field: name is required

func WithExpectStatus

func WithExpectStatus(codes ...int) Option

WithExpectStatus marks additional status codes as success, so they return a nil error instead of an HTTPError and their body is decoded/streamed like any 2xx. The classic case is 304 Not Modified with conditional requests, or a 3xx you handle yourself. Codes already in 2xx need not be listed.

Example

ExampleWithExpectStatus treats 304 Not Modified as success instead of an error, for conditional requests.

package main

import (
	"context"
	"net/http"

	"github.com/ubgo/httpreq"
)

func main() {
	_, err := httpreq.Do(context.Background(), "https://api.example.com/resource",
		httpreq.WithHeader("If-None-Match", `"etag-value"`),
		httpreq.WithExpectStatus(http.StatusNotModified),
	)
	// err is nil on a 304 now, instead of an *HTTPError.
	_ = err
}

func WithFormBody

func WithFormBody(values url.Values) Option

WithFormBody URL-encodes values and sends them as an application/x-www-form-urlencoded body — the shape OAuth token endpoints and classic HTML form posts expect. Pass nil to clear a previously set body.

Example

ExampleWithFormBody posts application/x-www-form-urlencoded values, the shape OAuth token endpoints expect.

package main

import (
	"context"
	"net/http"
	"net/url"

	"github.com/ubgo/httpreq"
)

func main() {
	form := url.Values{}
	form.Set("grant_type", "client_credentials")
	form.Set("scope", "read")

	_, _ = httpreq.Do(context.Background(), "https://auth.example.com/token",
		httpreq.WithMethod(http.MethodPost),
		httpreq.WithBasicAuth("client-id", "client-secret"),
		httpreq.WithFormBody(form),
	)
}

func WithHTTPClient

func WithHTTPClient(c *http.Client) Option

WithHTTPClient overrides the underlying *http.Client. Use this to share a connection pool across requests or to install custom middleware via a transport. WithTimeout still applies — it sets Client.Timeout if the supplied client has none.

func WithHeader

func WithHeader(key, value string) Option

WithHeader sets a single header. Repeat the option to set multiple. To set repeated values for the same header (e.g. multi-value Accept), call it once per value.

func WithHeaders

func WithHeaders(h http.Header) Option

WithHeaders adds every key/value in h to the request, appending to any headers already set (same semantics as calling WithHeader once per value). A nil or empty h is a no-op.

Example

ExampleWithHeaders and ExampleWithQuery add several headers or query params at once, appending to anything already set.

package main

import (
	"context"
	"net/http"

	"github.com/ubgo/httpreq"
)

func main() {
	_, _ = httpreq.Do(context.Background(), "https://api.example.com/x",
		httpreq.WithHeaders(http.Header{
			"Accept":          {"application/json"},
			"X-Request-Id":    {"abc-123"},
			"Accept-Language": {"en-US"},
		}),
	)
}

func WithJSONBody

func WithJSONBody(body any) Option

WithJSONBody marshals body as JSON and sets Content-Type: application/json. Pass nil to clear a previously set body.

func WithMethod

func WithMethod(m string) Option

WithMethod sets the HTTP method. Default: GET. Methods are not validated against [http.Method*] constants — the request goes out as you specify it.

func WithObserver

func WithObserver(fn func(context.Context, Trace)) Option

WithObserver registers a callback invoked exactly once when the request attempt completes — on success and on every failure path. See Trace for what is delivered (metadata only; no bodies or headers). Repeat the option to register multiple observers; all are called in registration order.

The callback runs synchronously on the calling goroutine after the body has been read, so keep it fast: record a metric, emit a log line, annotate a span. Do not block on I/O inside it. A nil fn is ignored.

Example

ExampleWithObserver registers an observer that receives one Trace per request attempt. The Trace carries metadata only — never bodies or headers.

package main

import (
	"context"
	"fmt"
	"net/http"
	"net/http/httptest"

	"github.com/ubgo/httpreq"
)

func main() {
	srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, _ *http.Request) {
		w.WriteHeader(http.StatusNoContent)
	}))
	defer srv.Close()

	_, _ = httpreq.Do(context.Background(), srv.URL,
		httpreq.WithObserver(func(_ context.Context, t httpreq.Trace) {
			fmt.Printf("%s -> %d\n", t.Method, t.StatusCode)
		}),
	)
}

Output:
GET -> 204

func WithQuery

func WithQuery(q url.Values) Option

WithQuery adds every key/value in q to the URL query string, appending to any parameters already set (same semantics as calling WithQueryParam once per value). A nil or empty q is a no-op.

Example

package main

import (
	"context"
	"net/url"

	"github.com/ubgo/httpreq"
)

func main() {
	_, _ = httpreq.Do(context.Background(), "https://api.example.com/search",
		httpreq.WithQuery(url.Values{
			"q":    {"golang"},
			"page": {"2"},
			"tag":  {"http", "client"},
		}),
	)
}

func WithQueryParam

func WithQueryParam(key, value string) Option

WithQueryParam appends a URL query parameter. Repeat for multiple values of the same key.

func WithRawBody

func WithRawBody(body []byte) Option

WithRawBody sends body bytes verbatim. Caller must set Content-Type via WithHeader if it matters. Useful for form posts, protobuf, or pre-encoded payloads.

func WithRequest

func WithRequest(fn func(*http.Request) error) Option

WithRequest registers a hook that receives the fully built *http.Request after all other options are applied and just before it is sent, letting you set anything the options don't model: request signing (an auth header computed over the assembled request), fields net/http keeps off the header map (http.Request.Host, cookies via http.Request.AddCookie, http.Request.Close, http.Request.Trailer), or any one-off tweak.

Returning an error aborts Do (and Curl) before sending, wrapped as "httpreq: request hook". Repeat the option to chain hooks; they run in order. The hook also runs for Curl/WithCurl, so a curl dump reflects the final, mutated request. A nil hook is ignored.

This is an escape hatch — with it comes the ability to build an invalid request. Prefer a dedicated option when one exists.

Example

ExampleWithRequest signs the fully built request just before it is sent — the escape hatch for anything the options don't model.

package main

import (
	"context"
	"net/http"

	"github.com/ubgo/httpreq"
)

func main() {
	_, _ = httpreq.Do(context.Background(), "https://api.example.com/x",
		httpreq.WithRequest(func(req *http.Request) error {
			// Set fields net/http keeps off the header map, or sign the request.
			req.Host = "internal.service"
			req.Header.Set("X-Signature", "computed-over-the-request")
			return nil
		}),
	)
}

func WithResponseBytes

func WithResponseBytes(b *[]byte) Option

WithResponseBytes captures the raw response body into *b, regardless of status and regardless of whether WithResponseInto is also set. Use it for non-JSON responses (HTML, text, XML, binary) or when you need the exact bytes alongside a decoded value. On an empty response *b is set to an empty, non-nil slice.

Example

ExampleWithResponseBytes captures a non-JSON response body verbatim — useful for HTML, text, XML, or binary payloads that WithResponseInto can't decode.

package main

import (
	"context"
	"fmt"
	"net/http"
	"net/http/httptest"

	"github.com/ubgo/httpreq"
)

func main() {
	srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, _ *http.Request) {
		_, _ = w.Write([]byte("<h1>hello</h1>"))
	}))
	defer srv.Close()

	var raw []byte
	if _, err := httpreq.Do(context.Background(), srv.URL, httpreq.WithResponseBytes(&raw)); err != nil {
		fmt.Println("error:", err)
		return
	}
	fmt.Println(string(raw))
}

Output:
<h1>hello</h1>

func WithResponseInto

func WithResponseInto(v any) Option

WithResponseInto JSON-decodes the response body into v. v must be a pointer. Skip this option to discard the body.

func WithResponseWriter

func WithResponseWriter(w io.Writer) Option

WithResponseWriter streams a successful response body straight to w via io.Copy instead of buffering it in memory — the way to handle downloads or responses larger than RAM. Compose sinks with io.MultiWriter (e.g. write to a file and a hash at once).

It applies only to success responses (status < 300, or a code allowed by WithExpectStatus). Error responses are still buffered so the HTTPError keeps the raw body. Because the body is streamed, not held, this option is mutually exclusive with WithResponseInto and WithResponseBytes on the same call — the writer takes precedence and those are not populated. A nil w is ignored (the body is buffered as usual).

Example

ExampleWithResponseWriter streams the response body to a writer instead of buffering it in memory — the way to download large files. Compose sinks with io.MultiWriter to, e.g., write to a file and hash in one pass.

package main

import (
	"context"
	"fmt"
	"net/http"
	"net/http/httptest"
	"os"

	"github.com/ubgo/httpreq"
)

func main() {
	srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, _ *http.Request) {
		_, _ = w.Write([]byte("streamed payload"))
	}))
	defer srv.Close()

	// Stream straight to stdout (use an *os.File for a real download).
	_, err := httpreq.Do(context.Background(), srv.URL, httpreq.WithResponseWriter(os.Stdout))
	if err != nil {
		fmt.Println("error:", err)
		return
	}
}

Output:
streamed payload

func WithTimeout

func WithTimeout(d time.Duration) Option

WithTimeout caps the total request time including dialing, headers, body, and response reading. Default: 30 seconds. Pass 0 to use the caller's context deadline alone.

func WithUserAgent

func WithUserAgent(ua string) Option

WithUserAgent sets the User-Agent header. Pass an empty string to suppress the User-Agent entirely (no header sent). When this option is not used, DefaultUserAgent is sent.

Example

ExampleWithUserAgent sets a custom User-Agent. Without this option, httpreq.DefaultUserAgent is sent; pass "" to suppress the header entirely.

package main

import (
	"context"

	"github.com/ubgo/httpreq"
)

func main() {
	_, _ = httpreq.Do(context.Background(), "https://api.example.com/x",
		httpreq.WithUserAgent("my-service/1.4.2"),
	)
}

type Trace

type Trace struct {
	Method     string
	URL        string        // final URL including query params
	ReqBytes   int           // request body size; 0 for bodyless or streamed bodies
	RespBytes  int           // response body size actually read
	StatusCode int           // 0 if the request never got a response
	Duration   time.Duration // wall time around send + full body read
	Err        error         // nil on success; network err, *HTTPError, or decode err
	Attempt    int           // 1 today; reserved for a future retry feature

	// Connection-phase timings, set only with WithConnTrace(). See the type
	// doc for the keep-alive caveat.
	DNS     time.Duration
	Connect time.Duration
	TLS     time.Duration
	TTFB    time.Duration // start of send to first response byte
}

Trace is metadata about one completed request attempt, delivered to the callbacks registered via WithObserver. It fires once per attempt on every path — success, non-2xx (HTTPError), network failure, and decode failure — so it is a complete lifecycle signal for logs, metrics, and spans.

SHAPE CONTRACT: Trace carries metadata ONLY. It never contains request or response bodies and never contains headers (so no Authorization / cookies / API keys leak into your logs by accident). If you need header or body content in your observability, install a custom http.RoundTripper via WithHTTPClient where you own the redaction policy.

The DNS/Connect/TLS/TTFB fields are populated only when WithConnTrace is set; otherwise they are zero. Connect and TLS are zero on a reused keep-alive connection (no new dial/handshake happened).