httpc

package module
v1.4.0 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Jul 20, 2026 License: MIT Imports: 13 Imported by: 4

README

httpc

Go

生产级 Go HTTP JSON 客户端，基于 gtkit/json 实现可切换的 JSON 后端。

安装

go get github.com/gtkit/httpc

特性

  • 全 HTTP 方法: GET / POST / PUT / PATCH / DELETE / HEAD / OPTIONS + 通用 RequestJSON
  • 表单请求: PostForm / PostFormWithHeader 发送 application/x-www-form-urlencoded 表单（OAuth2 token 等场景），响应仍按 JSON 解码
  • BaseURL 与默认 Header: WithBaseURL 调用处只传路径；WithDefaultHeaders 公共头设一次，单次调用可覆盖
  • 响应 Header 透出: JSON 与 Raw 方法均提供 *WithHeader 变体，便于读取 X-Request-Id/ETag 等
  • 响应体限流: 默认上限 10 MiB（WithMaxResponseBytes 可调），防止超大/恶意响应打爆内存
  • JSON 编解码: 使用 github.com/gtkit/json/v2，构建时可切换 sonic/go-json/jsoniter
  • 连接池: MaxIdleConns=100, MaxIdleConnsPerHost=10（WithMaxIdleConnsPerHost 可调）, HTTP/2, KeepAlive；WithMaxConnsPerHost 可封顶单 host 连接总数，保护脆弱/限流下游
  • 代理支持: 默认读取 HTTP_PROXY / HTTPS_PROXY / NO_PROXY 环境变量
  • 安全 Body drain: 限量排空（≤4 KiB）以复用连接，避免被恶意 body 拖垮
  • Redirect 控制: 默认跟随 3xx，也可禁用自动跳转或自定义跳转策略
  • 无内置日志: 状态码/Header/错误全部回传，由调用方在业务层记录（错误信息自动屏蔽 URL 中的 userinfo 密码）
  • Context 传播: 所有方法第一个参数都是 context.Context

使用

c := httpc.New(
    httpc.WithTimeout(10 * time.Second),
)

// BaseURL + 全局默认 Header：调用处只传路径，公共头只设一次
c = httpc.New(
    httpc.WithBaseURL("https://api.example.com"),
    httpc.WithDefaultHeaders(map[string]string{
        "Authorization": "Bearer xxx",
        "User-Agent":    "my-service/1.0",
    }),
)
status, err := c.GetJSON(ctx, "/v1/users?id=1", nil, &result) // 实际请求 https://api.example.com/v1/users?id=1
// 单次调用传同名 header 可覆盖默认值；URL 自带 scheme 时绕过 BaseURL 直接使用

// JSON POST
var result MyResponse
status, err := c.PostJSON(ctx, url, reqBody, &result)

// JSON GET with Bearer token
status, err := c.GetJSON(ctx, url, map[string]string{
    "Authorization": "Bearer xxx",
}, &result)

// PUT / PATCH / DELETE
c.PutJSON(ctx, url, body, &result)
c.PatchJSON(ctx, url, body, &result)
c.DeleteJSON(ctx, url, body, &result)

// Raw response (多次 unmarshal)
body, status, err := c.GetRaw(ctx, url, headers)

// 通用方法（自定义 method + headers + body）
c.RequestJSON(ctx, "OPTIONS", url, headers, body, &result)

// 表单编码 POST（OAuth2 token 等）：body 为 application/x-www-form-urlencoded，响应按 JSON 解码
form := url.Values{"grant_type": {"client_credentials"}}
status, err := c.PostForm(ctx, "/oauth/token", nil, form, &tokenResp)
// headers 参数与 JSON 方法同语义：单次调用头覆盖默认头（如单次 Basic auth）
status, err = c.PostForm(ctx, "/oauth/token", map[string]string{
    "Authorization": "Basic xxx",
}, form, &tokenResp)
// 需要响应 Header 时用对称变体
header, status, err := c.PostFormWithHeader(ctx, "/oauth/token", nil, form, &tokenResp)
// 空表单（nil 或零长度）不发送 body、不设置 Content-Type，与 JSON 方法的 nil body 语义一致；
// 表单体可重放（GetBody 已设置），调用方可安全重试

// 需要响应 Header 时，使用 *WithHeader 变体（返回 http.Header, status, err）
header, status, err := c.GetJSONWithHeader(ctx, url, nil, &result)
reqID := header.Get("X-Request-Id")
// 同样提供 Post/Put/Patch/Delete 及通用 RequestJSONWithHeader
header, status, err = c.RequestJSONWithHeader(ctx, "GET", url, headers, nil, &result)
// Raw 路径也有对称变体（返回 http.Header, []byte, status, err）
header, body, status, err = c.GetRawWithHeader(ctx, url, nil)

// 限制响应体大小（默认 10 MiB），超限返回 errors.Is(err, httpc.ErrResponseTooLarge)
c = httpc.New(httpc.WithMaxResponseBytes(5 << 20)) // 5 MiB
// 传 0 关闭限制（仅在完全可信的内网场景使用）
c = httpc.New(httpc.WithMaxResponseBytes(0))

// HEAD
resp, err := c.Head(ctx, url, nil)

// Fire-and-forget（result 传 nil）
c.PostJSON(ctx, url, body, nil)

// 禁止自动跟随 3xx，直接读取 Location / status
c = httpc.New(httpc.WithoutRedirect())
body, status, err := c.GetRaw(ctx, url, nil)

// 主要请求单一上游时，调大每 host 的空闲连接数（默认 10）。
// 该选项会先克隆 transport 再修改，不会改写共享的 transport（代价是独立连接池）
c = httpc.New(httpc.WithMaxIdleConnsPerHost(50))

// 封顶单 host 的连接总数（活跃+空闲，默认 0 不限制）。这是保护参数而非加速参数：
// 防止突发 fan-out 把脆弱或被限流的下游打爆，达到上限后新请求会阻塞等待空闲连接
c = httpc.New(httpc.WithMaxConnsPerHost(100))

// 自定义 http.Client：会浅拷贝，后续选项不会改写你传入的 client；
// 反过来，构造之后你再修改自己的 client 也不会影响 httpc。
// 注意 WithHTTPClient 要放在 WithTimeout 等修改型选项之前。
c = httpc.New(httpc.WithHTTPClient(myClient), httpc.WithTimeout(5*time.Second))

行为说明：body 传 nil（包括存进 any 的 typed-nil 指针/map/slice）时不发送请求体、不设置 Content-Type，而不是发送 JSON 字面量 null。JSON 便捷方法不提供 raw JSON body 入口；如需显式发送原始 null，请使用低层 Do 自行构造 *http.Request。

注意：重试有意不内置 —— 只有调用方知道哪些请求幂等、该用什么退避策略。请求体支持重放（GetBody 已设置），调用方可安全地重发。

复用：Client 是并发安全的，构造后只读。请 New 一次后长期持有、全局或按模块复用，不要每次请求都 New —— 连接池（含 keep-alive、HTTP/2）挂在底层 transport 上，每请求新建会让复用失效，徒增 TLS 握手与 TIME_WAIT。单上游高并发时按需上调 WithMaxIdleConnsPerHost，必要时用 WithMaxConnsPerHost 保护下游。

日志（由调用方负责）

httpc 不内置日志。状态码、Header、错误都通过返回值给到调用方 —— 库内再记一遍只是重复，且缺少业务上下文（trace-id 等）。请在你自己的调用层记录：

status, err := c.GetJSON(ctx, url, headers, &result)
if err != nil {
    log.Errorw("upstream call failed", "url", url, "status", status, "err", err)
    return err
}

错误信息已自动屏蔽 URL 中的 userinfo 密码（user:xxxxx@host）；但 query 中的 token 不会脱敏 —— token 请放 header，勿放 query。

需要连接级追踪（DNS / TLS 握手 / 连接复用）时，给请求 context 挂 net/http/httptrace.ClientTrace 即可，无需库内日志。

空 body 与状态码

JSON 便捷方法遇到空（或纯空白）响应体且传了 result 时，返回 httpc.ErrEmptyBody（用 errors.Is 判断），以区分「无内容」与「坏 JSON」。需要自行掌控状态码语义、错误信封或拿 Header 时，用 Raw 变体（不解码）：

header, body, status, err := c.GetRawWithHeader(ctx, url, nil)
if err != nil { return err }            // 仅传输/读 body 错误
if status >= 400 { /* 自行处理错误信封 */ }

JSON 后端切换

构建时指定 tag 即可切换 JSON 引擎（由 gtkit/json 提供）：

go build -tags=sonic ./...    # 使用 sonic (最快)
go build -tags=go_json ./...  # 使用 go-json
go build -tags=jsoniter ./... # 使用 jsoniter
go build ./...                # 标准库

License

MIT

Documentation

Overview

Package httpc provides a production-grade HTTP client for JSON API communication.

Features:

  • All standard HTTP methods (GET, POST, PUT, PATCH, DELETE, HEAD, OPTIONS)
  • Form-encoded POST (Client.PostForm, Client.PostFormWithHeader) with JSON-decoded responses, for OAuth2 token endpoints and similar APIs
  • Base URL and client-wide default headers (WithBaseURL, WithDefaultHeaders)
  • JSON encode/decode via github.com/gtkit/json/v2 (build-tag swappable backend)
  • Connection pooling, keep-alive, HTTP/2
  • Bounded body draining for connection reuse
  • Response body size cap to guard against memory exhaustion
  • No built-in logging: every outcome (status, headers, error) is returned to the caller, who logs with full business context
  • Context propagation and cancellation

A nil request body — including a typed nil pointer, map, or slice stored in the any parameter — sends no body and no Content-Type, rather than the JSON literal "null".

Retries are intentionally left to the caller: only the caller knows which requests are idempotent and what backoff policy fits. Request bodies are replayable (GetBody is set), so a caller-side retry can resend safely.

Reuse: a Client is concurrency-safe and read-only after construction. Build one with New and keep it for the process lifetime (or per module) — do not call New per request. The connection pool (keep-alive, HTTP/2) lives on the underlying transport; rebuilding the client each request defeats reuse and piles up TLS handshakes and TIME_WAIT. For a single high-concurrency upstream, raise WithMaxIdleConnsPerHost; cap a fragile upstream with WithMaxConnsPerHost.

Quick start: 

c := httpc.New(httpc.WithTimeout(10 * time.Second))
var result MyResponse
status, err := c.PostJSON(ctx, url, reqBody, &result)

Index

Examples

Constants

View Source 
const Version = "v1.3.2"

Variables

View Source 
var ErrEmptyBody = errors.New("httpc: empty response body")

ErrEmptyBody is returned by the JSON convenience methods when the server sends an empty (or whitespace-only) response body but a decode target was provided. It lets callers distinguish "no content" (e.g. 204, an empty 502) from malformed JSON. Detect it with errors.Is. Use the Raw methods if an empty body is expected and should not be treated as an error.

View Source 
var ErrResponseTooLarge = errors.New("httpc: response body exceeds limit")

ErrResponseTooLarge is returned (wrapped) when a response body exceeds the configured maximum. See WithMaxResponseBytes. Detect it with errors.Is.

Functions

This section is empty.

Types

type Client 

type Client struct {
	// contains filtered or unexported fields
}

Client is a concurrency-safe HTTP client optimized for JSON APIs. All fields are read-only after construction — safe to share across goroutines.

By design httpc does not log: every outcome (status, headers, error) is returned to the caller, who should log it from their own layer with full business context. For transport-level tracing (DNS, TLS, connection reuse), attach a net/http/httptrace.ClientTrace to the request context.

func New 

func New(opts ...Option) *Client

New creates a new Client with production-grade defaults.

Example 
package main

import (
	"fmt"
	"time"

	"github.com/gtkit/httpc"
)

func main() {
	client := httpc.New(
		httpc.WithTimeout(5*time.Second),
		httpc.WithBaseURL("https://api.example.com"),
	)

	fmt.Println(client.HTTPClient().Timeout)
}

Output:
5s

func (*Client) DeleteJSON 

func (c *Client) DeleteJSON(ctx context.Context, url string, body any, result any) (int, error)

DeleteJSON sends DELETE with optional JSON body and JSON-decodes the response.

func (*Client) DeleteJSONWithHeader 

func (c *Client) DeleteJSONWithHeader(ctx context.Context, url string, body any, result any) (http.Header, int, error)

DeleteJSONWithHeader sends DELETE with an optional JSON body, JSON-decodes the response, and returns the response headers.

func (*Client) Do 

func (c *Client) Do(req *http.Request) (*http.Response, error)

Do executes an arbitrary http.Request and returns the response. The caller is responsible for closing the response body. Prefer the higher-level methods unless you need full control.

func (*Client) GetJSON 

func (c *Client) GetJSON(ctx context.Context, url string, headers map[string]string, result any) (int, error)

GetJSON sends GET with custom headers and JSON-decodes the response.

func (*Client) GetJSONWithHeader 

func (c *Client) GetJSONWithHeader(ctx context.Context, url string, headers map[string]string, result any) (http.Header, int, error)

GetJSONWithHeader sends GET with custom headers, JSON-decodes the response, and returns the response headers.

func (*Client) GetRaw 

func (c *Client) GetRaw(ctx context.Context, url string, headers map[string]string) ([]byte, int, error)

GetRaw sends GET and returns the raw response body bytes. Useful when the response needs multiple unmarshal passes.

func (*Client) GetRawWithHeader 

func (c *Client) GetRawWithHeader(ctx context.Context, url string, headers map[string]string) (http.Header, []byte, int, error)

GetRawWithHeader sends GET and returns the raw response body together with the response headers. See the *WithHeader JSON methods for header semantics.

func (*Client) HTTPClient 

func (c *Client) HTTPClient() *http.Client

HTTPClient returns the underlying http.Client for advanced usage.

func (*Client) Head 

func (c *Client) Head(ctx context.Context, url string, headers map[string]string) (*http.Response, error)

Head sends a HEAD request and returns the response headers.

The returned response's Body has already been drained and closed — read resp.Header and resp.StatusCode only; reading resp.Body returns an error.

func (*Client) PatchJSON 

func (c *Client) PatchJSON(ctx context.Context, url string, body any, result any) (int, error)

PatchJSON sends PATCH with a JSON body and JSON-decodes the response.

func (*Client) PatchJSONWithHeader 

func (c *Client) PatchJSONWithHeader(ctx context.Context, url string, body any, result any) (http.Header, int, error)

PatchJSONWithHeader sends PATCH with a JSON body, JSON-decodes the response, and returns the response headers.

func (*Client) PostForm 

func (c *Client) PostForm(ctx context.Context, url string, headers map[string]string, form url.Values, result any) (int, error)

PostForm sends a POST request with the form encoded as application/x-www-form-urlencoded and JSON-decodes the response into result. Typical use is OAuth2 token endpoints and other form-based APIs that reply with JSON.

Content-Type and Accept are set automatically; headers follows the same precedence as the JSON methods (standard headers, then client-wide defaults, then per-call headers — later wins), so a per-call Authorization (e.g. Basic auth) overrides a default one. A nil or empty form sends no body and no Content-Type, mirroring the JSON methods' nil-body semantics. The body is replayable (GetBody is set), so caller-side retries can resend safely.

Response semantics match the JSON methods: an empty body with a non-nil result returns ErrEmptyBody; a nil result skips reading the body entirely (fire-and-forget); bodies over the configured cap return an error wrapping ErrResponseTooLarge.

Example 
package main

import (
	"context"
	"fmt"
	"net/http"
	"net/http/httptest"
	"net/url"

	"github.com/gtkit/httpc"
)

func main() {
	server := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
		_ = r.ParseForm()
		w.Header().Set("Content-Type", "application/json")
		_, _ = fmt.Fprintf(w, `{"grant_type":%q}`, r.PostFormValue("grant_type"))
	}))
	defer server.Close()

	client := httpc.New(httpc.WithHTTPClient(server.Client()))
	form := url.Values{"grant_type": {"client_credentials"}}
	var result struct {
		GrantType string `json:"grant_type"`
	}
	status, err := client.PostForm(context.Background(), server.URL, nil, form, &result)

	fmt.Println(status, result.GrantType, err == nil)
}

Output:
200 client_credentials true

func (*Client) PostFormWithHeader 

func (c *Client) PostFormWithHeader(ctx context.Context, url string, headers map[string]string, form url.Values, result any) (http.Header, int, error)

PostFormWithHeader behaves like Client.PostForm and additionally returns the response headers. The header is non-nil whenever a response was received (including on decode failure); it is nil only on transport-level errors.

func (*Client) PostJSON 

func (c *Client) PostJSON(ctx context.Context, url string, body any, result any) (int, error)

PostJSON sends POST with a JSON body and JSON-decodes the response.

Example 
package main

import (
	"context"
	"fmt"
	"net/http"
	"net/http/httptest"

	"github.com/gtkit/httpc"
)

func main() {
	server := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
		w.Header().Set("Content-Type", "application/json")
		_, _ = w.Write([]byte(`{"ok":true}`))
	}))
	defer server.Close()

	client := httpc.New(httpc.WithHTTPClient(server.Client()))
	var result struct {
		OK bool `json:"ok"`
	}
	status, err := client.PostJSON(context.Background(), server.URL, map[string]string{"name": "gtkit"}, &result)

	fmt.Println(status, result.OK, err == nil)
}

Output:
200 true true

func (*Client) PostJSONWithHeader 

func (c *Client) PostJSONWithHeader(ctx context.Context, url string, body any, result any) (http.Header, int, error)

PostJSONWithHeader sends POST with a JSON body, JSON-decodes the response, and returns the response headers.

func (*Client) PostRaw 

func (c *Client) PostRaw(ctx context.Context, url string, body any) ([]byte, int, error)

PostRaw sends POST with a JSON body and returns raw response bytes.

func (*Client) PostRawWithHeader 

func (c *Client) PostRawWithHeader(ctx context.Context, url string, body any) (http.Header, []byte, int, error)

PostRawWithHeader sends POST with a JSON body and returns the raw response body together with the response headers.

func (*Client) PutJSON 

func (c *Client) PutJSON(ctx context.Context, url string, body any, result any) (int, error)

PutJSON sends PUT with a JSON body and JSON-decodes the response.

func (*Client) PutJSONWithHeader 

func (c *Client) PutJSONWithHeader(ctx context.Context, url string, body any, result any) (http.Header, int, error)

PutJSONWithHeader sends PUT with a JSON body, JSON-decodes the response, and returns the response headers.

func (*Client) RequestJSON 

func (c *Client) RequestJSON(ctx context.Context, method, url string, headers map[string]string, body any, result any) (int, error)

RequestJSON sends any HTTP method with custom headers, optional JSON body, and JSON-decodes the response.

func (*Client) RequestJSONWithHeader 

func (c *Client) RequestJSONWithHeader(ctx context.Context, method, url string, headers map[string]string, body any, result any) (http.Header, int, error)

RequestJSONWithHeader sends any HTTP method with custom headers, optional JSON body, JSON-decodes the response, and returns the response headers.

This is the most general header-returning method; the *WithHeader convenience methods delegate to the same code path. See the *WithHeader documentation for the header return semantics.

func (*Client) RequestRaw 

func (c *Client) RequestRaw(ctx context.Context, method, url string, headers map[string]string, body any) ([]byte, int, error)

RequestRaw sends any HTTP method with custom headers and optional JSON body, returns raw response bytes.

func (*Client) RequestRawWithHeader 

func (c *Client) RequestRawWithHeader(ctx context.Context, method, url string, headers map[string]string, body any) (http.Header, []byte, int, error)

RequestRawWithHeader sends any HTTP method with custom headers and optional JSON body, returning the raw response body together with the response headers.

type Option 

type Option func(*Client)

Option configures a Client. Options are applied in the order given; WithHTTPClient replaces the underlying client wholesale, so list it before mutating options such as WithTimeout or WithCheckRedirect.

func WithBaseURL 

func WithBaseURL(u string) Option

WithBaseURL sets a base URL that is prefixed to relative request URLs, so call sites pass only the path ("/v1/users") instead of rebuilding the full URL. A trailing slash on the base is trimmed. Request URLs that already contain a scheme ("https://...") bypass the base and are used as-is.

func WithCheckRedirect 

func WithCheckRedirect(fn func(req *http.Request, via []*http.Request) error) Option

WithCheckRedirect sets the redirect policy used by the underlying http.Client. Pass http.ErrUseLastResponse from the function to return the latest 3xx response.

func WithDefaultHeaders 

func WithDefaultHeaders(h map[string]string) Option

WithDefaultHeaders sets headers applied to every request — typically Authorization, User-Agent, or a service-identity header. A per-call header with the same name overrides the default. The map is copied; changes the caller makes to it after New are not observed. Repeated use merges into the existing defaults.

func WithHTTPClient 

func WithHTTPClient(hc *http.Client) Option

WithHTTPClient overrides the underlying http.Client.

The client struct is shallow-copied so that later mutating options (WithTimeout, WithCheckRedirect) modify the copy, never the caller's client. The Transport is shared by reference — its connection pool is reused. Isolation cuts both ways: changes the caller makes to hc after New (Timeout, Jar, CheckRedirect) are not observed by this Client either.

Place this option before mutating options: a WithTimeout listed earlier would be overwritten by the replacement client.

func WithMaxConnsPerHost 

func WithMaxConnsPerHost(n int) Option

WithMaxConnsPerHost caps the total connections (active plus idle) the transport opens to a single host. This is a protection knob, not a speed knob: it shields a fragile or rate-limited upstream from being overwhelmed by a burst of fan-out requests. Once the cap is reached, further requests to that host block until a connection frees. The default is 0 (unlimited).

Like WithMaxIdleConnsPerHost, the transport is cloned before the change, so a transport shared via WithHTTPClient is never mutated — at the cost that this client gets its own connection pool. A nil transport falls back to a clone of http.DefaultTransport. A custom non-*http.Transport RoundTripper is left untouched — configure its pool directly instead.

func WithMaxIdleConnsPerHost 

func WithMaxIdleConnsPerHost(n int) Option

WithMaxIdleConnsPerHost sets the maximum idle (keep-alive) connections kept per host (default 10). Raise this when most traffic targets a single upstream, so high concurrency does not churn through new connections.

The transport is cloned before the change, so a transport shared via WithHTTPClient is never mutated — at the cost that this client gets its own connection pool. A nil transport falls back to a clone of http.DefaultTransport. A custom non-*http.Transport RoundTripper is left untouched — configure its pool directly instead.

func WithMaxResponseBytes 

func WithMaxResponseBytes(n int64) Option

WithMaxResponseBytes caps how many response body bytes httpc reads, guarding against memory exhaustion from oversized or malicious responses. The default is 10 MiB. Pass 0 (or a negative value) to disable the limit. When a body exceeds the cap, JSON/Raw methods return an error wrapping ErrResponseTooLarge.

The limit counts decompressed bytes (Go transparently decompresses gzip responses), which is what actually occupies memory — do not size it from Content-Length, which reflects the compressed payload.

func WithTimeout 

func WithTimeout(d time.Duration) Option

WithTimeout sets the HTTP client timeout (default 10s).

func WithTransport 

func WithTransport(t http.RoundTripper) Option

WithTransport overrides the HTTP transport.

func WithoutRedirect 

func WithoutRedirect() Option

WithoutRedirect disables automatic redirect following.

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.