Documentation
¶
Overview ¶
Package httpclient provides a configurable outbound HTTP client with built-in trace propagation and structured request/response logging.
Problem ¶
Service-to-service HTTP calls often require the same operational plumbing: timeouts, trace ID forwarding, consistent logs, optional payload dumps in debug mode, and redaction of sensitive data. Repeating this setup in each caller creates inconsistent observability and duplicated boilerplate.
Solution ¶
This package wraps net/http client behavior in a small Client abstraction:
- New builds a client from composable functional options.
- Client.Do executes requests with trace/header enrichment and structured logging around request/response timing.
By default, the client uses a 1-minute timeout, the standard transport, and the default trace ID header from the traceid package.
Features ¶
- Configurable timeout, logger, component tag, log field prefix, and trace ID header name.
- Automatic trace ID propagation: if a trace ID is missing in context, a new UUIDv7-based ID is generated and attached to context and request headers.
- Structured outbound logging with request/response timestamps and duration.
- Debug-level request/response dumps via net/http/httputil, with pluggable redaction before logs are written.
- Transport extensibility through round-tripper wrapping and custom dial context hooks.
Logging Behavior ¶
At debug level, request and response dumps are logged (after redaction). At non-debug levels, summary metadata is logged without payload dumps. Errors are logged with the same trace context and timing fields.
Query strings are redacted before logging at every level, so secrets carried in query parameters (for example api_key or token) are not written to logs. This includes the error field of a failed request, whose embedded URL has its query and userinfo redacted. Two things are not redacted: the request path (so avoid placing secrets in the path, e.g. /reset/{token}), and errors produced by a custom round-tripper that do not wrap a *url.Error (redact those in the round-tripper).
Debug-level dumps are redacted by the configured function (see WithRedactFn), which by default masks Authorization-style headers, cookie and other sensitive key/value pairs, and card-like numbers. Arbitrary secret request headers (for example a custom X-Api-Key) are not covered by the default; supply a stronger redactor via WithRedactFn when such headers are sent.
Debug-level dumps buffer the request and response body in memory, bounded by a configurable maximum (see WithMaxDumpSize, default 1 MiB):
- Request bodies larger than the cap, or of unknown length (streaming/chunked), have their headers dumped but their payload omitted; omitting unknown-length request bodies also avoids a deadlock that would otherwise occur when dumping a streaming request body.
- Response bodies larger than the cap are omitted when the length is known; when the length is unknown (chunked/streaming), the body is truncated to the cap in the dump and marked as truncated, while the caller still receives the complete response body.
Because dumping an unknown-length response reads up to the cap before the log entry is emitted (and before Do returns the body to the caller), debug-level logging is not suitable for genuinely streaming endpoints (server-sent events, long polling): enabling it can add up to WithMaxDumpSize worth of buffering latency. Keep debug logging off for such endpoints, or lower the cap.
Client Reuse ¶
Each New builds a client with its own private transport and connection pool. Create a client once and reuse it for the lifetime of the caller; constructing a new client per request defeats connection reuse. Use Client.CloseIdleConnections to release pooled connections on shutdown or reconfiguration.
The default transport raises MaxIdleConnsPerHost above net/http's default of 2 so that per-host connection reuse is not throttled for the common case of many concurrent calls to a few downstream hosts. Use WithTLSClientConfig for custom CAs or client certificates, or WithTransport to supply a fully tuned transport (pool sizes, proxy, HTTP/2) when the defaults do not fit.
Benefits ¶
httpclient standardizes outbound-call behavior across services, improving traceability and diagnostics while keeping call-site code concise.
Index ¶
- Variables
- type Client
- type DialContextFunc
- type InstrumentRoundTripper
- type Option
- func WithComponent(name string) Option
- func WithDialContext(fn DialContextFunc) Option
- func WithLogPrefix(prefix string) Option
- func WithLogger(logger *slog.Logger) Option
- func WithMaxDumpSize(n int64) Option
- func WithRedactFn(fn RedactFn) Option
- func WithRoundTripper(fn InstrumentRoundTripper) Option
- func WithTLSClientConfig(cfg *tls.Config) Option
- func WithTimeout(timeout time.Duration) Option
- func WithTraceIDHeaderName(name string) Option
- func WithTransport(t *http.Transport) Option
- type RedactFn
Examples ¶
Constants ¶
This section is empty.
Variables ¶
var ( // ErrNilRequest is returned by Do when the request is nil. ErrNilRequest = errors.New("httpclient: nil request") // ErrNilRequestURL is returned by Do when the request has a nil URL. ErrNilRequestURL = errors.New("httpclient: nil Request.URL") )
Sentinel errors returned by Do for a malformed request, mirroring the standard library's behavior of returning an error (rather than panicking) instead of dereferencing a nil field.
Functions ¶
This section is empty.
Types ¶
type Client ¶
type Client struct {
// contains filtered or unexported fields
}
Client wraps http.Client with trace ID propagation, structured request/response logging, and optional debug payload dumps.
func New ¶
New constructs an HTTP client with 1-minute timeout, trace ID propagation, and request/response logging from provided options.
Example ¶
package main
import (
"fmt"
"time"
"github.com/tecnickcom/nurago/pkg/httpclient"
"github.com/tecnickcom/nurago/pkg/traceid"
)
func main() {
// Build a reusable client from composable options. Create it once and reuse
// it: each client owns a private transport and connection pool.
client := httpclient.New(
httpclient.WithTimeout(5*time.Second),
httpclient.WithComponent("my-service"),
httpclient.WithTraceIDHeaderName(traceid.DefaultHeader),
httpclient.WithLogPrefix("http_"),
)
// Release pooled connections when the client is no longer needed.
defer client.CloseIdleConnections()
fmt.Println(client != nil)
}
Output: true
func (*Client) CloseIdleConnections ¶
func (c *Client) CloseIdleConnections()
CloseIdleConnections closes any connections on the client's transport that are currently idle, without interrupting in-flight requests. It is safe for concurrent use and no-ops for transports that do not implement the optional CloseIdleConnections method.
func (*Client) Do ¶
Do executes the request with trace ID attachment, structured logging, and optional debug payload dumps; returns error from underlying client.
Do operates on a private clone, so the caller's request keeps its original headers and context. The request body, however, is consumed as usual for a single-use request (the clone shares the body with the original).
A nil request returns ErrNilRequest and a request with a nil URL returns ErrNilRequestURL, mirroring the standard client's error-on-malformed-request behavior instead of panicking.
The returned response body is wrapped so that the per-request timeout context is canceled when the body is closed. The caller is therefore responsible for closing resp.Body (e.g. defer resp.Body.Close()) on the success path to avoid leaking the timeout timer until the timeout elapses; failing to close the body also leaks the underlying connection as with the standard net/http client.
The log entry records the response status as response_code and, when the response advertises one, its Content-Length as response_content_length. That is the advertised length, not the number of bytes actually read by the caller (which is unknown at log time); it is omitted for chunked or unknown length responses. On failure the error is logged with its URL query and userinfo redacted, so query-parameter secrets are not written to logs.
At debug level the request and response are dumped (after redaction). Dumps buffer the body in memory up to the configured maximum (see WithMaxDumpSize): over-cap request bodies (and unknown-length request bodies) are dumped without their payload, while an unknown-length response body is truncated to the cap in the dump (the caller still receives the full body). Because a debug-level response dump reads the body before the entry is emitted, response_duration includes body transfer time (up to the cap) at debug level but only time-to-response-headers at non-debug levels.
Example ¶
package main
import (
"context"
"fmt"
"io"
"log"
"net/http"
"net/http/httptest"
"github.com/tecnickcom/nurago/pkg/httpclient"
)
func main() {
server := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, _ *http.Request) {
_, _ = w.Write([]byte("pong"))
}))
client := httpclient.New()
req, err := http.NewRequestWithContext(context.Background(), http.MethodGet, server.URL, nil)
if err != nil {
log.Fatal(err)
}
resp, err := client.Do(req)
if err != nil {
log.Fatal(err)
}
body, err := io.ReadAll(resp.Body)
if err != nil {
log.Fatal(err)
}
// Always close the body: it releases the per-request timeout timer and the
// underlying connection (use defer resp.Body.Close() in real code).
_ = resp.Body.Close()
server.Close()
fmt.Println(string(body))
}
Output: pong
type DialContextFunc ¶
DialContextFunc is an alias for a net.Dialer.DialContext function.
type InstrumentRoundTripper ¶
type InstrumentRoundTripper func(next http.RoundTripper) http.RoundTripper
InstrumentRoundTripper is an alias for a RoundTripper function.
type Option ¶
type Option func(c *Client)
Option is the interface that allows to set client options.
func WithComponent ¶
WithComponent customizes component name embedded in log field entries.
An empty name is ignored so the default component tag is preserved.
func WithDialContext ¶
func WithDialContext(fn DialContextFunc) Option
WithDialContext customizes network connection establishment via transport DialContext hook.
A nil fn is ignored. It mutates the client's own *http.Transport (a private clone of http.DefaultTransport), so it never affects http.DefaultTransport or any other client.
Ordering matters relative to WithRoundTripper: this option only takes effect while the client's transport is still an *http.Transport. Once WithRoundTripper has wrapped the transport, this option can no longer reach the underlying *http.Transport and silently does nothing. Apply WithDialContext before WithRoundTripper.
func WithLogPrefix ¶
WithLogPrefix specifies a prefix applied to all log field names in Do (e.g., "http_").
Only field names are prefixed; the log message itself is a stable constant so entries remain filterable regardless of the prefix. An empty prefix (the default) leaves field names unprefixed.
func WithLogger ¶
WithLogger overrides default logger for all request/response logging.
A nil logger is ignored so the default logger is never replaced with one that would panic on the first request.
func WithMaxDumpSize ¶
WithMaxDumpSize caps the request/response body size (in bytes, measured by the advertised Content-Length) buffered into debug-level payload dumps.
Bodies larger than the cap, or of unknown length (streaming/chunked requests), have their headers dumped but their payload omitted. This bounds memory use and avoids a deadlock when dumping a streaming request body. A non-positive value disables the cap. The default is 1 MiB.
func WithRedactFn ¶
WithRedactFn customizes sensitive data redaction applied to debug-level payload dumps and the logged query string.
A nil fn is ignored, keeping the default redaction, so the option can never install a nil function that would panic on the first debug-level request.
fn is called concurrently by simultaneous Do calls (at least once per request for the query string, plus once per dump at debug level), so it must be safe for concurrent use. The default redact.HTTPDataString is.
func WithRoundTripper ¶
func WithRoundTripper(fn InstrumentRoundTripper) Option
WithRoundTripper wraps client transport with custom RoundTripper for middleware instrumentation.
A nil fn, or an fn that returns nil, is ignored and leaves the current transport in place (honoring the no-panics policy rather than deferring a nil-transport failure to the first request).
Ordering matters relative to WithDialContext: WithRoundTripper replaces the transport with the (typically non-*http.Transport) wrapper returned by fn, after which WithDialContext can no longer find the underlying *http.Transport and becomes a silent no-op. Apply WithDialContext before WithRoundTripper.
func WithTLSClientConfig ¶
WithTLSClientConfig sets the TLS configuration on the client's base transport, for custom CAs, client certificates (mTLS), or other TLS tuning, without having to rebuild a transport from scratch (which would drop the default dial, idle, HTTP/2, and proxy settings).
A nil config is ignored, keeping the transport's default TLS behavior. Like WithDialContext it mutates the client's own *http.Transport, so it takes effect only while the transport is still an *http.Transport: apply it after WithTransport (which installs the base) and before WithRoundTripper (which wraps the transport, after which this option silently does nothing). The config is stored by reference; do not mutate it after the client is created.
func WithTimeout ¶
WithTimeout customizes request timeout (default 1 minute).
A zero or negative timeout disables the timeout (the net/http convention), leaving the request bounded only by its own context.
func WithTraceIDHeaderName ¶
WithTraceIDHeaderName specifies custom trace ID header name (default X-Request-ID).
An empty name is ignored (the default is kept) since an empty header name would make every request fail at send time.
func WithTransport ¶
WithTransport replaces the client's base transport (a private clone of http.DefaultTransport) with a clone of t, so callers can tune connection pooling (MaxIdleConnsPerHost, MaxConnsPerHost, timeouts), TLS, proxy, and HTTP/2 settings. The transport is cloned, so later mutations of t do not affect the client and vice versa. A nil transport is ignored.
Apply this before WithDialContext and WithRoundTripper: WithDialContext mutates the base *http.Transport installed here, and WithRoundTripper wraps it. Applied after WithDialContext it would discard the custom dialer; applied after WithRoundTripper it would discard the wrapper.
Source Files