httpsimp

README

httpsimplified

Package httpsimplified sends outgoing HTTP requests via a simple straightforward API distilled from many internal Golang projects at USA Today Network. It embraces Go stdlib types like url.Values and http.Header, provides composable building blocks for more complex use cases and doesn't try to be clever.

See godoc.org/github.com/andreyvit/httpsimplified/v2 for a full reference.

Usage

Call Do with MakeGet, MakeForm, MakeJSON or Make to send a request and parse the response:

var resp responseType
err := httpsimp.Do(httpsimp.MakeGet(baseURL, path, params, headers), client, httpsimp.JSON(&resp))

where:

• httpsimp.MakeGet is a request builder function returning *http.Request, we also provide MakeForm, MakeJSON and Make (and, of course, you're free to build http.Request any other way);

• httpsimp.JSON is a body parser, we also provide PlainText, Bytes, Raw and None parsers; you can use multiple parsers and/or adjust their parameters; and you can define your own parsers.

See GoDoc for more details.

Features

Provides simple, composable building blocks and embraces Go stdlib types:

• request builder functions (MakeGet, MakeForm, MakeJSON, Make) just return an *http.Request that you can further customize if you want (e.g. you can call .WithContext(ctx) to make the request cancelable);

• Parse parses any *http.Response using one or more body parsers;

• Do accepts any *http.Request, executes it using http.Client and handles the response via Parse;

• when building a custom request, you can use lower-level helper functions: URL, EncodeForm, EncodeJSONBody, SetBody;

• request parameters are specified via url.Values — you can pass an inline map like url.Values{"key": []string{"value"}}, or build url.Values some other way, or just pass nil;

• request headers are specified via http.Header — you can pass an inline map like http.Header{"X-Something": []string{"value"}}, or build http.Header some other way, or just pass nil.

This library strives to be as straight-forward and non-magical as possible.

We used to define one-shot helpers like Get and Post, but then the number of combination exploded (Get, GetContext, PostForm, PostFormContext, PostJSON, PostJSONContext, plus same for PUT), and we instead opted for a 2-call combination (Do(MakeXxx(http.MethodPost, ...), ...)), which we believe to be superior in every way.

Change Log

See CHANGELOG for a history of changes.

Documentation

Overview

Package httpsimp sends outgoing HTTP requests via a simple straightforward API distilled from many internal Golang projects at USA Today Network. It embraces Go stdlib types like url.Values and http.Header, provides composable building blocks for more complex use cases and doesn't try to be clever.

Call Perform with MakeGet, MakeForm, MakeJSON or Make to send a request and parse the response:

var resp responseType
err := httpsimp.Do(httpsimp.MakeGet(baseURL, path, params, headers), client, httpsimp.JSON(&resp))

where httpsimp.JSON is a body parser function (we also provide PlainText, Bytes, Raw and None parsers, and you can define your own). See the example for more details.

You need to pass an instance of *http.Client. You can use http.DefaultClient, but note that the default client has no timeouts and might potentially hang forever, causing goroutine leaks. A custom client is strongly recommended:

client := &http.Client{
    Timeout: time.Second * 10,
}

You can adjust body parser parameters by passing additional options to body parser functions, like this:

httpsimp.JSON(nil, httpsimp.ContentType("application/something"))

Available options:

- httpsimp.StatusAny, httpsimp.Status4xx, httpsimp.Status4xx5xx, or a specific status like httpsimp.StatusOK or httpsimp.StatusSpec(http.StatusTeapot) will match only responses with the given status.

- httpsimp.ContentType("application/something") will match only response with the given content type.

- httpsimp.ContentType("") will match any content type (can be used to cancel default application/json filter used by JSON).

- httpsimp.ReturnError() results in a non-nil error returned.

Pass multiple parsers to handle alternative response types or non-2xx status codes:

var resp responseStruct
var bytes []byte
var e errorStruct
err := httpsimp.Perform(...,
    httpsimp.JSON(&resp),
    httpsimp.Bytes(&bytes, httpsimp.ContentType("image/png")),
    httpsimp.JSON(&e, httpsimp.Status4xx5xx))

If you need a cancelable request, use http.Request.WithContext:

var resp responseType
req := httpsimp.MakeGet(baseURL, path, params, headers).WithContext(myCtx)
err := httpsimp.Perform(req, client, httpsimp.JSON(&resp))

You can build the entire http.Request yourself and just call Perform:

var resp responseType
err := httpsimp.Perform(&http.Request{
    Method: http.MethodPut,
    URL:    httpsimp.URL(baseURL, path, params),
    Header: http.Header{...},
    Body:   myReader,
}, httpsimp.JSON(&resp))

When building custom requests, use our helpers:

URL concatenates a URL and adds query params.

EncodeForm, EncodeJSONBody and SetBody add a body to a request.

Finally, you're free to obtain an http.Response through other means and then call Parse to handle the response:

httpResp, err := executeSomehow(obtainHTTPRequestSomehow())
if err != nil { ... }

var resp responseType
err = httpsimp.Parse(httpResp, httpsimp.JSON(&resp))

To handle HTTP basic authentication, use BasicAuthValue helper:

err := httpsimp.Get("...", "...", url.Values{...}, http.Header{
    httpsimp.AuthorizationHeader: []string{httpsimp.BasicAuthValue("user", "pw")},
}, httpsimp.JSON, &resp)

Example

var resp exampleResponse
// url.Values is just a map[string][]string
err := httpsimp.Do(httpsimp.MakeGet(endpointURL, "examples/foo.json", url.Values{
	"param1": []string{"value1"},
	"param2": []string{"value2"},
}, nil), http.DefaultClient, httpsimp.JSON(&resp))

if err != nil {
	log.Fatal(err)
}
log.Printf("foo = %#v", resp)

Example (CustomHeaders)

var resp exampleResponse
// url.Values and http.Header are both just map[string][]string
err := httpsimp.Do(httpsimp.MakeGet(endpointURL, "examples/foo.json", url.Values{
	"param1": []string{"value1"},
	"param2": []string{"value2"},
}, http.Header{
	"X-Powered-By":               []string{"Golang"},
	httpsimp.AuthorizationHeader: []string{httpsimp.BasicAuthValue("user", "secret")},
}), http.DefaultClient, httpsimp.JSON(&resp))

if err != nil {
	log.Fatal(err)
}
log.Printf("foo = %#v", resp)

Index

• Constants
• func BasicAuthValue(username, password string) string
• func Do(r *http.Request, client HTTPClient, parsers ...Parser) error
• func EncodeForm(r *http.Request, params url.Values) *http.Request
• func EncodeJSONBody(r *http.Request, obj interface{}) *http.Request
• func Is4xx(err error) bool
• func Is5xx(err error) bool
• func Make(method string, base, path string, params url.Values, body []byte, ...) *http.Request
• func MakeForm(method string, base, path string, params url.Values, headers http.Header) *http.Request
• func MakeGet(base, path string, params url.Values, headers http.Header) *http.Request
• func MakeJSON(method string, base, path string, params url.Values, obj interface{}, ...) *http.Request
• func Parse(resp *http.Response, parsers ...Parser) error
• func SetBody(r *http.Request, data []byte) *http.Request
• func StatusCode(err error) int
• func URL(base, path string, params url.Values) *url.URL
• type HTTPClient
• type ParseOption
  • func ContentType(ctype string) ParseOption
  • func ReturnError() ParseOption
• type Parser
  • func Bytes(result *[]byte, mopt ...ParseOption) Parser
  • func JSON(result interface{}, mopt ...ParseOption) Parser
  • func MakeParser(defaultCtype string, mopt []ParseOption, ...) Parser
  • func None(mopt ...ParseOption) Parser
  • func PlainText(result *string, mopt ...ParseOption) Parser
  • func Raw(ptr **http.Response, mopt ...ParseOption) Parser
• type StatusSpec
  • func (desired StatusSpec) Matches(actual int) bool

Examples

• Package
• Package (CustomHeaders)

Constants

const (
	// ContentTypeJSON is "application/json"
	ContentTypeJSON = "application/json"

	// ContentTypeTextPlain is "text/plain"
	ContentTypeTextPlain = "text/plain"

	// ContentTypeFormURLEncoded is "application/x-www-form-urlencoded"
	ContentTypeFormURLEncoded = "application/x-www-form-urlencoded"
)

const (
	// StatusNone matches no status code and is a zero value of StatusSpec.
	StatusNone StatusSpec = 0

	// StatusAny matches all status codes.
	StatusAny StatusSpec = -1500

	// Status1xx matches all 1xx status codes.
	Status1xx StatusSpec = -100

	// Status2xx matches all 2xx status codes.
	Status2xx StatusSpec = -200

	// Status3xx matches all 3xx status codes.
	Status3xx StatusSpec = -300

	// Status4xx matches all 4xx status codes.
	Status4xx StatusSpec = -400

	// Status4xx matches all 4xx status codes.
	Status5xx StatusSpec = -500

	// Status4xx5xx matches all 4xx and 5xx status codes.
	Status4xx5xx StatusSpec = -900

	StatusOK             = StatusSpec(http.StatusOK)
	StatusCreated        = StatusSpec(http.StatusCreated)
	StatusAccepted       = StatusSpec(http.StatusAccepted)
	StatusNoContent      = StatusSpec(http.StatusNoContent)
	StatusPartialContent = StatusSpec(http.StatusPartialContent)

	StatusUnauthorized = StatusSpec(http.StatusUnauthorized)
	StatusForbidden    = StatusSpec(http.StatusForbidden)
	StatusNotFound     = StatusSpec(http.StatusNotFound)
)

const (
	// AuthorizationHeader is the "Authorization" HTTP header
	AuthorizationHeader = "Authorization"
)

Functions

func BasicAuthValue

func BasicAuthValue(username, password string) string

BasicAuthValue returns an Authorization header value for HTTP Basic authentication method with the given username and password, i.e. it returns:

"Basic " + base64(username + ":" + password)

Use AuthorizationHeader constant for the header name.

func Do

func Do(r *http.Request, client HTTPClient, parsers ...Parser) error

Do executes the given request via the given http.Client and handles the body using the specified parsers.

Pass an instance of *http.Client as client. You can use http.DefaultClient, but note that the default client has no timeouts and might potentially hang forever, causing goroutine leaks. A custom client is strongly recommended.

For the parsers, use JSON, Bytes, PlainText, Raw or None from this package, or define your own custom one using MakeParser.

func EncodeForm

func EncodeForm(r *http.Request, params url.Values) *http.Request

EncodeForm encodes the given params into application/x-www-form-urlencoded format and sets the body and Content-Type on the given request.

To properly handle HTTP redirects, both Body and GetBody are set.

func EncodeJSONBody

func EncodeJSONBody(r *http.Request, obj interface{}) *http.Request

EncodeJSONBody encodes the given object into JSON (application/json) format and sets the body and Content-Type on the given request.

If JSON encoding fails, the method panics.

To properly handle HTTP redirects, both Body and GetBody are set.

func Is4xx

func Is4xx(err error) bool

func Is5xx

func Is5xx(err error) bool

func Make

func Make(method string, base, path string, params url.Values, body []byte, headers http.Header) *http.Request

Make builds a POST/PUT/etc request with the given URL, headers and body.

base and path are concatenated to form a URL; at least one of them must be provided, but the other one can be an empty string. The resulting URL must be valid and parsable via net/url, otherwise panic ensues.

url.Values and http.Header are just maps that can be provided in place, no need to use their fancy Set or Add methods.

func MakeForm

func MakeForm(method string, base, path string, params url.Values, headers http.Header) *http.Request

MakeForm builds a POST/PUT/etc request with the given URL, headers and body (which contains the given params in application/x-www-form-urlencoded format).

base and path are concatenated to form a URL; at least one of them must be provided, but the other one can be an empty string. The resulting URL must be valid and parsable via net/url, otherwise panic ensues.

url.Values and http.Header are just maps that can be provided in place, no need to use their fancy Set or Add methods.

func MakeGet

func MakeGet(base, path string, params url.Values, headers http.Header) *http.Request

MakeGet builds a GET request with the given URL, headers and params (encoded into a query string).

base and path are concatenated to form a URL; at least one of them must be provided, but the other one can be an empty string. The resulting URL must be valid and parsable via net/url, otherwise panic ensues.

url.Values and http.Header are just maps that can be provided in place, no need to use their fancy Set or Add methods.

func MakeJSON

func MakeJSON(method string, base, path string, params url.Values, obj interface{}, headers http.Header) *http.Request

MakeJSON builds a POST/PUT/etc request with the given URL, headers and body (which contains the given object encoded in JSON format).

base and path are concatenated to form a URL; at least one of them must be provided, but the other one can be an empty string. The resulting URL must be valid and parsable via net/url, otherwise panic ensues.

url.Values and http.Header are just maps that can be provided in place, no need to use their fancy Set or Add methods.

If JSON encoding fails, the method panics.

func Parse

func Parse(resp *http.Response, parsers ...Parser) error

Parse handles the HTTP response using of the provided parsers. The first matching parser wins.

If no parsers match, some predefined fallback parsers are tried; all of them cause a non-nil error to be returned.

func SetBody

func SetBody(r *http.Request, data []byte) *http.Request

SetBody sets the given request's body to the given data.

To properly handle HTTP redirects, both Body and GetBody are set.

func StatusCode

func StatusCode(err error) int

StatusCode returns the HTTP status code carried by the given error. Returns 0 if the error is not produced by a body parser function.

func URL

func URL(base, path string, params url.Values) *url.URL

URL returns a *url.URL (conveniently suitable for http.Request's URL field) concatenating the two given URL strings and optionally appending a query string with the given params.

base and path are concatenated to form a URL; at least one of them must be provided, but the other one can be an empty string. The resulting URL must be valid and parsable via net/url, otherwise panic ensues.

url.Values and http.Header are just maps that can be provided in place, no need to use their fancy Set or Add methods.

Types

type HTTPClient

type HTTPClient interface {
	Do(req *http.Request) (*http.Response, error)
}

HTTPClient is an interface implemented by *http.Client, requiring only the Do method. Instead of accepting *http.Client, the methods in this package accept HTTPClients for extra flexibility.

type ParseOption

type ParseOption interface {
	// contains filtered or unexported methods
}

ParseOption is passed into MakeParser and built-in parser functions to adjust which responses the parser matches and whether it matches an error response.

You cannot define custom parser options.

func ContentType

func ContentType(ctype string) ParseOption

ContentType causes the parser to only match responses with the given content type. If an empty string is passed in, the parser will match any content type.

func ReturnError

func ReturnError() ParseOption

ReturnError causes Do or Parse to return a non-nil error if this parser matches. (The body is still parsed and handled.)

type Parser

type Parser struct {
	// contains filtered or unexported fields
}

Parser matches and handles an http.Response.

To create a parser, use of the built-in parser functions like JSON, PlainText, etc, or build a custom one using MakeParser.

func Bytes

func Bytes(result *[]byte, mopt ...ParseOption) Parser

Bytes is a Parser function that verifies the response status code and reads the entire body into a byte array.

Pass the result of this function into Do or Parse to handle a response.

func JSON

func JSON(result interface{}, mopt ...ParseOption) Parser

JSON is a Parser function that verifies the response status code and content type (which must be ContentTypeJSON) and unmarshals the body into the result variable (which can be anything that you'd pass to json.Unmarshal).

Pass the result of this function into Do or Parse to handle a response.

func MakeParser

func MakeParser(defaultCtype string, mopt []ParseOption, bodyParser func(resp *http.Response) (interface{}, error)) Parser

MakeParser builds a parser wrapping the given parse function.

The parser starts out matching responses with the given content type (which can be empty to match any response).

The provided options change the behavior of the parser and may override the content type that it matches.

func None

func None(mopt ...ParseOption) Parser

None is a Parser function that verifies the response status code and discards the response body.

Pass the result of this function into Do or Parse to handle a response.

func PlainText

func PlainText(result *string, mopt ...ParseOption) Parser

PlainText is a Parser function that verifies the response status code and reads the entire body into a string.

Pass the result of this function into Do or Parse to handle a response.

func Raw

func Raw(ptr **http.Response, mopt ...ParseOption) Parser

Raw is a Parser function that verifies the response status code and returns the raw *http.Response without reading or closing the body (which you MUST do when you're done with the response).

Pass the result of this function into Do or Parse to handle a response.

type StatusSpec

type StatusSpec int

func (StatusSpec) Matches

func (desired StatusSpec) Matches(actual int) bool

Matches returns whether the given actual HTTP status code matches the desired status code spec, which may be a specific status code or one of special constants: StatusNone (won't match anything), Status1xx, Status2xx, Status3xx, Status4xx, Status5xx.