requests

README

requests

An elegant and simple HTTP client package, which learned a lot from the well-known Python package Requests: HTTP for Humans™.

Why not just use the standard library HTTP client?

Brad Fitzpatrick, long time maintainer of the net/http package, wrote Problems with the net/http Client API. The four main points are:

• Too easy to not call Response.Body.Close.
• Too easy to not check return status codes
• Context support is oddly bolted on
• Proper usage is too many lines of boilerplate

requests solves these issues by:

• always closing the response body,
• checking status codes by default,
• optional context.Context parameter,
• and simplifying the boilerplate.

Features

• Keep-Alive & Connection Pooling
• International Domains and URLs
• Sessions with Cookie Persistence
• Browser-style SSL Verification
• Automatic Content Decoding
• Basic/Digest Authentication
• Elegant Key/Value Cookies
• Automatic Decompression
• Unicode Response Bodies
• HTTP(S) Proxy Support
• Multipart File Uploads
• Streaming Downloads
• Connection Timeouts
• Chunked Requests
• .netrc Support
• context.Context Support
• Interceptor Support

Examples

Simple GET a text

code with net/http	code with requests
req, err := http.NewRequestWithContext( ctx, http.MethodGet, "http://example.com", nil) if err != nil { // ... } res, err := http.DefaultClient.Do(req) if err != nil { // ... } defer res.Body.Close() b, err := io.ReadAll(res.Body) if err != nil { // ... } s := string(b)	var txt string r, err := requests.Get("http://example.com", requests.ToText(&txt)) if err != nil { // ... }
14+ lines	4+ lines

POST a raw body

code with net/http	code with requests
body := bytes.NewReader(([]byte(`hello, world`)) req, err := http.NewRequestWithContext( ctx, http.MethodPost, "http://example.com", body) if err != nil { // ... } req.Header.Set("Content-Type", "text/plain") res, err := http.DefaultClient.Do(req) if err != nil { // ... } defer res.Body.Close() _, err := io.ReadAll(res.Body) if err != nil { // ... }	r, err := requests.Post("http://example.com", requests.Data(`hello, world`)) if err != nil { // ... }
15+ lines	4+ lines

GET a JSON object with context

code with net/http	code with requests
u, err := url.Parse("http://example.com") if err != nil { // ... } req, err := http.NewRequestWithContext( ctx, http.MethodGet, u.String(), nil) if err != nil { // ... } resp, err := http.DefaultClient.Do(req) if err != nil { // ... } defer resp.Body.Close() b, err := io.ReadAll(res.Body) if err != nil { // ... } var res JSONResponse err := json.Unmarshal(b, &res) if err != nil { // ... }	var res JSONResponse r, err := requests.Post("http://example.com", requests.Context(ctx), requests.ToJSON(&res)) if err != nil { // ... }
22+ lines	5+ lines

POST a JSON object and parse the response

req := JSONRequest{
    Title:  "foo",
    Body:   "baz",
    UserID: 1,
}
var res JSONResponse
r, err := requests.Post("http://example.com",
            requests.JSON(&req),
            requests.ToJSON(&res))

Set custom header and form for a request

// Set headers and forms
r, err := requests.Post("http://example.com", 
            requests.HeaderPairs("martini", "shaken"),
            requests.FormPairs("name", "Jacky"))

Easily manipulate URLs and query parameters

// Set parameters
r, err := requests.Get("http://example.com?a=1&b=2", 
            requests.ParamPairs("c", "3"))
// URL: http://example.com?a=1&b=2&c=3

Dump request and response

var reqDump, respDump string
r, err := requests.Get("http://example.com", 
            requests.Dump(&reqDump, &respDump))

Documentation

Overview

Package requests is an elegant and simple HTTP library for golang, built for human beings.

This package mimics the implementation of the classic Python package Requests(https://requests.readthedocs.io/)

Index

• func SetEnvTimeout(timeout time.Duration)
• func WithInterceptor(interceptors ...Interceptor)
• type Client
  • func (c *Client) Do(ctx context.Context, r *Request) (*Response, error)
• type Do
• type Interceptor
• type Option
  • func BasicAuth(username, password string) Option
  • func Body(body io.Reader) Option
  • func Context(ctx context.Context) Option
  • func Data(data any) Option
  • func DisableKeepAlives() Option
  • func Dump(req, resp *string) Option
  • func Files(files map[string]*os.File) Option
  • func Form[T map[string]string | url.Values](params T) Option
  • func FormPairs(kv ...string) Option
  • func HeaderPairs(kv ...string) Option
  • func Headers[T map[string]string | http.Header](headers T) Option
  • func JSON(v any) Option
  • func ParamPairs(kv ...string) Option
  • func Params[T map[string]string | url.Values](params T) Option
  • func Timeout(timeout time.Duration) Option
  • func ToJSON(v any) Option
  • func ToText(v *string) Option
• type Options
• type Request
  • func (r *Request) WithContext(ctx context.Context) *Request
• type Response
  • func Delete(url string, options ...Option) (*Response, error)
  • func Get(url string, options ...Option) (*Response, error)
  • func Patch(url string, options ...Option) (*Response, error)
  • func Post(url string, options ...Option) (*Response, error)
  • func Put(url string, options ...Option) (*Response, error)
  • func (r *Response) Bytes() []byte
  • func (r *Response) Cookies() map[string]*http.Cookie
  • func (r *Response) Headers() http.Header
  • func (r *Response) JSON(v any) error
  • func (r *Response) Method() string
  • func (r *Response) StatusCode() int
  • func (r *Response) StatusText() string
  • func (r *Response) Text() string
  • func (r *Response) URL() string
• type Stats

Functions

func SetEnvTimeout

func SetEnvTimeout(timeout time.Duration)

SetEnvTimeout sets the default timeout for each HTTP request at the environment level.

func WithInterceptor

func WithInterceptor(interceptors ...Interceptor)

WithInterceptor specifies the interceptor for each HTTP request.

Types

type Client

type Client struct {
	*http.Client
}

func (*Client) Do

func (c *Client) Do(ctx context.Context, r *Request) (*Response, error)

Do sends the HTTP request and returns after response is received.

type Do

type Do func(ctx context.Context, r *Request) (*Response, error)

Do is called by Interceptor to complete HTTP requests.

type Interceptor

type Interceptor func(ctx context.Context, r *Request, do Do) (*Response, error)

Interceptor provides a hook to intercept the execution of an HTTP request invocation. When an interceptor(s) is set, requests delegates all HTTP client invocations to the interceptor, and it is the responsibility of the interceptor to call do to complete the processing of the HTTP request.

type Option

type Option func(*Options)

Option is the functional option type.

func BasicAuth

func BasicAuth(username, password string) Option

BasicAuth is the option to implement HTTP Basic Auth.

func Body

func Body(body io.Reader) Option

Body sets io.Reader to hold request body.

func Context

func Context(ctx context.Context) Option

Context sets the HTTP request context.

For outgoing client request, the context controls the entire lifetime of a request and its response: obtaining a connection, sending the request, and reading the response headers and body.

func Data

func Data(data any) Option

Data sets raw string into the request body.

func DisableKeepAlives

func DisableKeepAlives() Option

DisableKeepAlives, if true, disables HTTP keep-alives and will only use the connection to the server for a single HTTP request.

This is unrelated to the similarly named TCP keep-alives.

func Dump

func Dump(req, resp *string) Option

Dump dumps outgoing client request and response to the corresponding input param (req or resp) if not nil.

Refer: - https://pkg.go.dev/net/http/httputil#DumpRequestOut - https://pkg.go.dev/net/http/httputil#DumpResponse

func Files

func Files(files map[string]*os.File) Option

Files sets files to a map of (field, fileHandler). It also sets the Content-Type as "multipart/form-data".

func Form

func Form[T map[string]string | url.Values](params T) Option

Form sets the given form values into the request body. It also sets the Content-Type as "application/x-www-form-urlencoded". Two types are supported:

Type 1: map[string]string

map[string]string(
	"key1", "val1",
	"key2", "val2",
)

Type 2: url.Values

url.Values(
	"key1", []string{"val1", "val1-2"},
	"key2", "val2",
)

func FormPairs

func FormPairs(kv ...string) Option

FormPairs sets form values by the mapping of key-value pairs. It panics if len(kv) is odd.

Values with the same key will be merged into a list:

FormPairs(
	"key1", "val1",
	"key1", "val1-2", // "key1" will have map value []string{"val1", "val1-2"}
	"key2", "val2",
)

func HeaderPairs

func HeaderPairs(kv ...string) Option

HeaderPairs sets HTTP headers formed by the mapping of key-value pairs. The keys should be in canonical form, as returned by http.CanonicalHeaderKey. It panics if len(kv) is odd.

Values with the same key will be merged into a list:

HeaderPairs(
	"Key1", "val1",
	"Key1", "val1-2", // "Key1" will have map value []string{"val1", "val1-2"}
	"Key2", "val2",
)

func Headers

func Headers[T map[string]string | http.Header](headers T) Option

Headers sets the HTTP headers. The keys should be in canonical form, as returned by http.CanonicalHeaderKey. Two types are supported:

Type 1: map[string]string

map[string]string(
	"Header-Key1", "val1",
	"Header-Key2", "val2",
)

Type 2: http.Header

http.Header(
	"Header-Key1", []string{"val1", "val1-2"},
	"Header-Key2", "val2",
)

func JSON

func JSON(v any) Option

JSON marshals the given struct as JSON into the request body. It also sets the Content-Type as "application/json".

func ParamPairs

func ParamPairs(kv ...string) Option

ParamPairs sets the query parameters formed by the mapping of key-value pairs. It panics if len(kv) is odd.

Values with the same key will be merged into a list:

ParamPairs(
	"key1", "val1",
	"key1", "val1-2", // "key1" will have map value []string{"val1", "val1-2"}
	"key2", "val2",
)

func Params

func Params[T map[string]string | url.Values](params T) Option

Params sets the given query parameters into the URL query string. Two types are supported:

Type 1: map[string]string

map[string]string(
	"key1", "val1",
	"key2", "val2",
)

Type 2: url.Values

url.Values(
	"key1", []string{"val1", "val1-2"},
	"key2", "val2",
)

func Timeout

func Timeout(timeout time.Duration) Option

Timeout specifies a time limit for requests made by this Client. The timeout includes connection time, any redirects, and reading the response body. The timer remains running after Get, Head, Post, or Do return and will interrupt reading of the Response.Body.

A Timeout of zero means no timeout. Default is 60s.

func ToJSON

func ToJSON(v any) Option

ToJSON unmarshals HTTP response body to given struct as JSON.

func ToText

func ToText(v *string) Option

ToText unmarshals HTTP response body to string.

type Options

type Options struct {
	Headers http.Header
	Params  url.Values

	Body io.Reader
	// different request body types
	Data  any
	Form  url.Values
	JSON  any
	Files map[string]*os.File

	// different response body types
	ToText *string
	ToJSON any

	// auth
	AuthInfo *auth.AuthInfo
	// request timeout
	Timeout time.Duration

	DisableKeepAlives bool
	// dump
	DumpRequestOut *string
	DumpResponse   *string
	// contains filtered or unexported fields
}

Options defines all optional parameters for HTTP request.

type Request

type Request struct {
	*http.Request

	Stats *Stats
	// contains filtered or unexported fields
}

Request is a wrapper of http.Request.

func (*Request) WithContext

func (r *Request) WithContext(ctx context.Context) *Request

WithContext returns a shallow copy of r.Request with its context changed to ctx. The provided ctx must be non-nil.

type Response

type Response struct {
	*http.Response
	// contains filtered or unexported fields
}

Response is a wrapper of http.Response.

func Delete

func Delete(url string, options ...Option) (*Response, error)

Delete sends an HTTP request with DELETE method.

On error, any Response can be ignored. A non-nil Response with a non-nil error only occurs when Response.StatusCode() is not 2xx.

func Get

func Get(url string, options ...Option) (*Response, error)

Get sends an HTTP request with GET method.

On error, any Response can be ignored. A non-nil Response with a non-nil error only occurs when Response.StatusCode() is not 2xx.

func Patch

func Patch(url string, options ...Option) (*Response, error)

Patch sends an HTTP request with PATCH method.

On error, any Response can be ignored. A non-nil Response with a non-nil error only occurs when Response.StatusCode() is not 2xx.

func Post

func Post(url string, options ...Option) (*Response, error)

Post sends an HTTP POST request.

func Put

func Put(url string, options ...Option) (*Response, error)

Put sends an HTTP request with PUT method.

On error, any Response can be ignored. A non-nil Response with a non-nil error only occurs when Response.StatusCode() is not 2xx.

func (*Response) Bytes

func (r *Response) Bytes() []byte

Bytes returns the HTTP response body as []byte.

func (*Response) Cookies

func (r *Response) Cookies() map[string]*http.Cookie

Cookies parses and returns the cookies set in the Set-Cookie headers.

func (*Response) Headers

func (r *Response) Headers() http.Header

Headers maps header keys to values. If the response had multiple headers with the same key, they may be concatenated, with comma delimiters.

func (*Response) JSON

func (r *Response) JSON(v any) error

JSON decodes the HTTP response body as JSON format.

func (*Response) Method

func (r *Response) Method() string

Method returns the HTTP request method.

func (*Response) StatusCode

func (r *Response) StatusCode() int

StatusCode returns status code of HTTP response.

NOTE: It returns -1 if response is nil.

func (*Response) StatusText

func (r *Response) StatusText() string

StatusText returns a text for the HTTP status code.

NOTE:

• It returns "Response is nil" if response is nil.
• It returns the empty string if the code is unknown.

e.g. "OK"

func (*Response) Text

func (r *Response) Text() string

Text parses the HTTP response body as string.

func (*Response) URL

func (r *Response) URL() string

URL returns the HTTP request URL string.

type Stats

type Stats struct {
	BodySize int
}

Stats contains common metrics for an HTTP request.