requests

package module
v0.23.5 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Oct 6, 2023 License: MIT Imports: 27 Imported by: 135

README

Requests GoDoc Go Report Card Coverage Status Mentioned in Awesome Go

Requests logo

HTTP requests for Gophers.

The problem: Go's net/http is powerful and versatile, but using it correctly for client requests can be extremely verbose.

The solution: The requests.Builder type is a convenient way to build, send, and handle HTTP requests. Builder has a fluent API with methods returning a pointer to the same struct, which allows for declaratively describing a request by method chaining.

Requests also comes with tools for building custom http transports, include a request recorder and replayer for testing.

Features

  • Simplifies HTTP client usage compared to net/http
  • Can't forget to close response body
  • Checks status codes by default
  • Supports context.Context
  • JSON serialization and deserialization helpers
  • Easily manipulate URLs and query parameters
  • Request recording and replaying for tests
  • Customizable transports and validators that are compatible with the standard library and third party libraries
  • No third party dependencies
  • Good test coverage

Examples

Simple GET into a string
code with net/http code with requests 
req, err := http.NewRequestWithContext(ctx, 
	http.MethodGet, "http://example.com", nil)
if err != nil {
	// ...
}
res, err := http.DefaultClient.Do(req)
if err != nil {
	// ...
}
defer res.Body.Close()
b, err := io.ReadAll(res.Body)
if err != nil {
	// ...
}
s := string(b)

var s string
err := requests.
	URL("http://example.com").
	ToString(&s).
	Fetch(ctx)
11+ lines5 lines
POST a raw body
code with net/http code with requests 
body := bytes.NewReader(([]byte(`hello, world`))
req, err := http.NewRequestWithContext(ctx, http.MethodPost, 
	"https://postman-echo.com/post", body)
if err != nil {
	// ...
}
req.Header.Set("Content-Type", "text/plain")
res, err := http.DefaultClient.Do(req)
if err != nil {
	// ...
}
defer res.Body.Close()
_, err := io.ReadAll(res.Body)
if err != nil {
	// ...
}

err := requests.
	URL("https://postman-echo.com/post").
	BodyBytes([]byte(`hello, world`)).
	ContentType("text/plain").
	Fetch(ctx)
12+ lines5 lines
GET a JSON object
code with net/http code with requests 
var post placeholder
u, err := url.Parse("https://jsonplaceholder.typicode.com")
if err != nil {
	// ...
}
u.Path = fmt.Sprintf("/posts/%d", 1)
req, err := http.NewRequestWithContext(ctx, 
	http.MethodGet, u.String(), nil)
if err != nil {
	// ...
}
res, err := http.DefaultClient.Do(req)
if err != nil {
	// ...
}
defer res.Body.Close()
b, err := io.ReadAll(res.Body)
if err != nil {
	// ...
}
err := json.Unmarshal(b, &post)
if err != nil {
	// ...
}

var post placeholder
err := requests.
	URL("https://jsonplaceholder.typicode.com").
	Pathf("/posts/%d", 1).
	ToJSON(&post).
	Fetch(ctx)
18+ lines7 lines
POST a JSON object and parse the response
var res placeholder
req := placeholder{
	Title:  "foo",
	Body:   "baz",
	UserID: 1,
}
err := requests.
	URL("/posts").
	Host("jsonplaceholder.typicode.com").
	BodyJSON(&req).
	ToJSON(&res).
	Fetch(ctx)
// net/http equivalent left as an exercise for the reader
Set custom headers for a request
// Set headers
var headers postman
err := requests.
	URL("https://postman-echo.com/get").
	UserAgent("bond/james-bond").
	ContentType("secret").
	Header("martini", "shaken").
	Fetch(ctx)
Easily manipulate URLs and query parameters
u, err := requests.
	URL("https://prod.example.com/get?a=1&b=2").
	Hostf("%s.example.com", "dev1").
	Param("b", "3").
	ParamInt("c", 4).
	URL()
if err != nil { /* ... */ }
fmt.Println(u.String()) // https://dev1.example.com/get?a=1&b=3&c=4
Record and replay responses
// record a request to the file system
var s1, s2 string
err := requests.URL("http://example.com").
	Transport(requests.Record(nil, "somedir")).
	ToString(&s1).
	Fetch(ctx)
check(err)

// now replay the request in tests
err = requests.URL("http://example.com").
	Transport(requests.Replay("somedir")).
	ToString(&s2).
	Fetch(ctx)
check(err)
assert(s1 == s2) // true

FAQs

See wiki for more details.

Why not just use the standard library HTTP client?

Brad Fitzpatrick, long time maintainer of the net/http package, wrote an extensive list of problems with the standard library HTTP client. His four main points (ignoring issues that can't be resolved by a wrapper around the standard library) are:

  • Too easy to not call Response.Body.Close.
  • Too easy to not check return status codes
  • Context support is oddly bolted on
  • Proper usage is too many lines of boilerplate

Requests solves these issues by always closing the response body, checking status codes by default, always requiring a context.Context, and simplifying the boilerplate with a descriptive UI based on fluent method chaining.

Why requests and not some other helper library?

There are two major flaws in other libraries as I see it. One is that in other libraries support for context.Context tends to be bolted on if it exists at all. Two, many hide the underlying http.Client in such a way that it is difficult or impossible to replace or mock out. Beyond that, I believe that none have acheived the same core simplicity that the requests library has.

How do I just get some JSON?
var data SomeDataType
err := requests.
	URL("https://example.com/my-json").
	ToJSON(&data).
	Fetch(ctx)
How do I post JSON and read the response JSON?
body := MyRequestType{}
var resp MyResponseType
err := requests.
	URL("https://example.com/my-json").
	BodyJSON(&body).
	ToJSON(&resp).
	Fetch(ctx)
How do I just save a file to disk?

It depends on exactly what you need in terms of file atomicity and buffering, but this will work for most cases:

err := requests.
	URL("http://example.com").
	ToFile("myfile.txt").
	Fetch(ctx)

For more advanced use case, use ToWriter.

How do I save a response to a string?
var s string
err := requests.
	URL("http://example.com").
	ToString(&s).
	Fetch(ctx)
How do I validate the response status?

By default, if no other validators are added to a builder, requests will check that the response is in the 2XX range. If you add another validator, you can add builder.CheckStatus(200) or builder.AddValidator(requests.DefaultValidator) to the validation stack.

To disable all response validation, run builder.AddValidator(nil).

Contributing

Please create a discussion before submitting a pull request for a new feature.

Documentation

Overview

Package requests is a convenience wrapper around net/http to make it faster and easier to build requests and custom transports.

Example 
package main

import (
	"context"
	"fmt"
	"strings"

	"github.com/carlmjohnson/requests"
)

func main() {
	// Simple GET into a string
	var s string
	err := requests.
		URL("http://example.com").
		ToString(&s).
		Fetch(context.Background())
	if err != nil {
		fmt.Println("could not connect to example.com:", err)
	}
	fmt.Println(strings.Contains(s, "Example Domain"))
}

Output:

true
Example (GetJSON) 
package main

import (
	"context"
	"fmt"

	"github.com/carlmjohnson/requests"
)

func main() {
	// GET a JSON object
	id := 1
	var post placeholder
	err := requests.
		URL("https://jsonplaceholder.typicode.com").
		Pathf("/posts/%d", id).
		ToJSON(&post).
		Fetch(context.Background())
	if err != nil {
		fmt.Println("could not connect to jsonplaceholder.typicode.com:", err)
	}
	fmt.Println(post.Title)
}

type placeholder struct {
	ID     int    `json:"id,omitempty"`
	Title  string `json:"title"`
	Body   string `json:"body"`
	UserID int    `json:"userId"`
}

Output:

sunt aut facere repellat provident occaecati excepturi optio reprehenderit
Example (PostJSON) 
package main

import (
	"context"
	"fmt"

	"github.com/carlmjohnson/requests"
)

func main() {
	// POST a JSON object and parse the response
	var res placeholder
	req := placeholder{
		Title:  "foo",
		Body:   "baz",
		UserID: 1,
	}
	err := requests.
		URL("/posts").
		Host("jsonplaceholder.typicode.com").
		BodyJSON(&req).
		ToJSON(&res).
		Fetch(context.Background())
	if err != nil {
		fmt.Println("could not connect to jsonplaceholder.typicode.com:", err)
	}
	fmt.Println(res)
}

type placeholder struct {
	ID     int    `json:"id,omitempty"`
	Title  string `json:"title"`
	Body   string `json:"body"`
	UserID int    `json:"userId"`
}

Output:

{101 foo baz 1}
Example (QueryParam) 
package main

import (
	"fmt"

	"github.com/carlmjohnson/requests"
)

func main() {
	subdomain := "dev1"
	c := 4

	u, err := requests.
		URL("https://prod.example.com/get?a=1&b=2").
		Hostf("%s.example.com", subdomain).
		Param("b", "3").
		ParamInt("c", c).
		URL()
	if err != nil {
		fmt.Println("Error!", err)
	}
	fmt.Println(u.String())

}

Output:

https://dev1.example.com/get?a=1&b=3&c=4

Index

Examples

Constants

This section is empty.

Variables

View Source 
var (
	// JSONSerializer is used by BodyJSON and Builder.BodyJSON.
	// The default serializer may be changed in a future version of requests.
	JSONSerializer Serializer = json.Marshal
	// JSONDeserializer is used by ToJSON and Builder.ToJSON.
	// The default deserializer may be changed in a future version of requests.
	JSONDeserializer Deserializer = json.Unmarshal
)
View Source 
var ErrInvalidHandled = errors.New("handled recovery from invalid response")
View Source 
var ToHeaders = CopyHeaders

ToHeaders is an alias for backwards compatibility.

Functions

func HasStatusErr 

func HasStatusErr(err error, codes ...int) bool

HasStatusErr returns true if err is a ResponseError caused by any of the codes given.

Example 
package main

import (
	"context"
	"fmt"

	"github.com/carlmjohnson/requests"
)

func main() {
	err := requests.
		URL("http://example.com/404").
		CheckStatus(200).
		Fetch(context.Background())
	if requests.HasStatusErr(err, 404) {
		fmt.Println("got a 404")
	}
}

Output:

got a 404

func NewCookieJar added in v0.21.10 

func NewCookieJar() http.CookieJar

NewCookieJar returns a cookie jar using the standard public suffix list.

Example 
package main

import (
	"context"
	"fmt"
	"net/http"
	"net/url"

	"github.com/carlmjohnson/requests"
)

func main() {
	// Create a client that preserve cookies between requests
	myClient := *http.DefaultClient
	myClient.Jar = requests.NewCookieJar()
	// Use the client to make a request
	err := requests.
		URL("http://httpbin.org/cookies/set/chocolate/chip").
		Client(&myClient).
		Fetch(context.Background())
	if err != nil {
		fmt.Println("could not connect to httpbin.org:", err)
	}
	// Now check that cookies we got
	for _, cookie := range myClient.Jar.Cookies(&url.URL{
		Scheme: "http",
		Host:   "httpbin.org",
	}) {
		fmt.Println(cookie)
	}
	// And we'll see that they're reused on subsequent requests
	var cookies struct {
		Cookies map[string]string
	}
	err = requests.
		URL("http://httpbin.org/cookies").
		Client(&myClient).
		ToJSON(&cookies).
		Fetch(context.Background())
	if err != nil {
		fmt.Println("could not connect to httpbin.org:", err)
	}
	fmt.Println(cookies)

	// And we can manually add our own cookie values
	// without overriding existing ones
	err = requests.
		URL("http://httpbin.org/cookies").
		Client(&myClient).
		Cookie("oatmeal", "raisin").
		ToJSON(&cookies).
		Fetch(context.Background())
	if err != nil {
		fmt.Println("could not connect to httpbin.org:", err)
	}
	fmt.Println(cookies)

}

Output:

chocolate=chip
{map[chocolate:chip]}
{map[chocolate:chip oatmeal:raisin]}

Types

type BodyGetter 

type BodyGetter = func() (io.ReadCloser, error)

BodyGetter provides a Builder with a source for a request body.

func BodyBytes 

func BodyBytes(b []byte) BodyGetter

BodyBytes is a BodyGetter that returns the provided raw bytes.

func BodyFile added in v0.22.0 

func BodyFile(name string) BodyGetter

BodyFile is a BodyGetter that reads the provided file path.

func BodyForm 

func BodyForm(data url.Values) BodyGetter

BodyForm is a BodyGetter that builds an encoded form body.

func BodyJSON 

func BodyJSON(v any) BodyGetter

BodyJSON is a BodySerializer that uses JSONSerializer to marshal the object.

func BodyReader 

func BodyReader(r io.Reader) BodyGetter

BodyReader is a BodyGetter that returns an io.Reader.

func BodySerializer added in v0.23.5 

func BodySerializer(s Serializer, v any) BodyGetter

BodySerializer is a BodyGetter that uses the provided Serializer to build the body of a request from v.

func BodyWriter added in v0.21.11 

func BodyWriter(f func(w io.Writer) error) BodyGetter

BodyWriter is a BodyGetter that pipes writes into a request body.

type Builder 

type Builder struct {
	// contains filtered or unexported fields
}

Builder is a convenient way to build, send, and handle HTTP requests. Builder has a fluent API with methods returning a pointer to the same struct, which allows for declaratively describing a request by method chaining.

Builder can build a url.URL, build an http.Request, or handle a full http.Client request and response with validation.

Build a url.URL with Builder.URL

Set the base URL by creating a new Builder with requests.URL or by calling Builder.BaseURL then customize it with Builder.Scheme, Builder.Host, Builder.Hostf, Builder.Path, Builder.Pathf, Builder.Param, and Builder.ParamInt.

Build an http.Request with Builder.Request

Set the method for a request with Builder.Method or use the Builder.Delete, Builder.Head, Builder.Patch, Builder.Post, and Builder.Put methods. By default, requests without a body are GET, and those with a body are POST.

Set headers with Builder.Header or set conventional header keys with Builder.Accept, Builder.BasicAuth, Builder.Bearer, Builder.CacheControl, Builder.ContentType, Builder.Cookie, and Builder.UserAgent.

Set the body of the request, if any, with Builder.Body or use built in Builder.BodyBytes, Builder.BodyFile, Builder.BodyForm, Builder.BodyJSON, Builder.BodyReader, or Builder.BodyWriter.

Handle a request and response with Builder.Do or Builder.Fetch

Set the http.Client to use for a request with Builder.Client and/or set an http.RoundTripper with Builder.Transport.

Add a response validator to the Builder with Builder.AddValidator or use the built in Builder.CheckStatus, Builder.CheckContentType, Builder.CheckPeek, Builder.CopyHeaders, and Builder.ErrorJSON. If no validator has been added, Builder will use DefaultValidator.

Set a handler for a response with Builder.Handle or use the built in Builder.ToHeaders, Builder.ToJSON, Builder.ToString, Builder.ToBytesBuffer, or Builder.ToWriter.

Builder.Fetch creates an http.Request with Builder.Request and validates and handles it with Builder.Do.

Other methods

Builder.Config can be used to set several options on a Builder at once. New creates a new Builder and applies Config options to it.

In many cases, it will be possible to set most options for an API endpoint in a Builder at the package or struct level and then call Builder.Clone in a function to add request specific details for the URL, parameters, headers, body, or handler.

Errors returned by Builder methods will have an ErrorKind indicating their origin.

The zero value of Builder is usable.

func New added in v0.23.3 

func New(cfgs ...Config) *Builder

New creates a new Builder suitable for method chaining by applying the specified Configs. It is equivalent to calling Config on an empty Builder. The zero value of Builder is usable, so it is not necessary to call New when you do not have any Configs to apply.

Example 
package main

import (
	"context"
	"fmt"
	"strings"

	"github.com/carlmjohnson/requests"
)

func main() {
	// Suppose all requests in your project need some common options set.
	// First, define a Config function in your project...
	myProjectConfig := func(rb *requests.Builder) {
		rb.
			BaseURL("http://example.com").
			UserAgent("myproj/1.0").
			Accept("application/vnd.myproj+json;charset=utf-8")
	}

	// Then build your requests using that Config as the base Builder.
	var s string
	err := requests.
		New(myProjectConfig).
		Path("/").
		Param("some_param", "some-value").
		ToString(&s).
		Fetch(context.Background())
	if err != nil {
		fmt.Println("my project fetch failed", err)
	}
	fmt.Println(strings.Contains(s, "Example Domain"))
}

Output:

true

func URL 

func URL(baseurl string) *Builder

URL creates a new Builder suitable for method chaining. It is equivalent to calling BaseURL on an empty Builder.

func (*Builder) Accept added in v0.21.4 

func (rb *Builder) Accept(contentTypes string) *Builder

Accept sets the Accept header for a request.

func (*Builder) AddValidator 

func (rb *Builder) AddValidator(h ResponseHandler) *Builder

AddValidator adds a response validator to the Builder. Adding a validator disables DefaultValidator. To disable all validation, just add nil.

func (*Builder) BaseURL added in v0.23.3 

func (rb *Builder) BaseURL(baseurl string) *Builder

BaseURL sets the base URL that other URL methods modify. It is usually more convenient to use URL instead.

func (*Builder) BasicAuth 

func (rb *Builder) BasicAuth(username, password string) *Builder

BasicAuth sets the Authorization header to a basic auth credential.

func (*Builder) Bearer added in v0.21.4 

func (rb *Builder) Bearer(token string) *Builder

Bearer sets the Authorization header to a bearer token.

Example 
package main

import (
	"context"
	"fmt"
	"net/http"

	"github.com/carlmjohnson/requests"
)

func main() {
	// We get a 401 response if no bearer token is provided
	err := requests.
		URL("http://httpbin.org/bearer").
		CheckStatus(http.StatusUnauthorized).
		Fetch(context.Background())
	if err != nil {
		fmt.Println("problem with httpbin:", err)
	}
	// But our response is accepted when we provide a bearer token
	var res struct {
		Authenticated bool
		Token         string
	}
	err = requests.
		URL("http://httpbin.org/bearer").
		Bearer("whatever").
		ToJSON(&res).
		Fetch(context.Background())
	if err != nil {
		fmt.Println("problem with httpbin:", err)
	}
	fmt.Println(res.Authenticated)
	fmt.Println(res.Token)
}

Output:

true
whatever

func (*Builder) Body added in v0.21.11 

func (rb *Builder) Body(src BodyGetter) *Builder

Body sets the BodyGetter to use to build the body of a request. The provided BodyGetter is used as an http.Request.GetBody func. It implicitly sets method to POST.

func (*Builder) BodyBytes 

func (rb *Builder) BodyBytes(b []byte) *Builder

BodyBytes sets the Builder's request body to b.

Example 
package main

import (
	"context"
	"fmt"

	"github.com/carlmjohnson/requests"
)

// Examples with the Postman echo server
type postman struct {
	Args    map[string]string `json:"args"`
	Data    string            `json:"data"`
	Headers map[string]string `json:"headers"`
	JSON    map[string]string `json:"json"`
}

func main() {
	// Post a raw body
	var data postman
	err := requests.
		URL("https://postman-echo.com/post").
		BodyBytes([]byte(`hello, world`)).
		ContentType("text/plain").
		ToJSON(&data).
		Fetch(context.Background())
	if err != nil {
		fmt.Println("problem with postman:", err)
	}
	fmt.Println(data.Data)
}

Output:

hello, world

func (*Builder) BodyFile added in v0.22.0 

func (rb *Builder) BodyFile(name string) *Builder

BodyFile sets the Builder's request body to read from the given file path.

Example 
package main

import (
	"context"
	"fmt"
	"log"
	"os"
	"path/filepath"

	"github.com/carlmjohnson/requests"
)

// Examples with the Postman echo server
type postman struct {
	Args    map[string]string `json:"args"`
	Data    string            `json:"data"`
	Headers map[string]string `json:"headers"`
	JSON    map[string]string `json:"json"`
}

func main() {
	// Make a file to read from
	dir, err := os.MkdirTemp("", "body_file_*")
	if err != nil {
		log.Fatal(err)
	}
	defer os.RemoveAll(dir) // clean up

	exampleFilename := filepath.Join(dir, "example.txt")
	exampleContent := `hello, world`
	if err = os.WriteFile(exampleFilename, []byte(exampleContent), 0644); err != nil {
		log.Fatal(err)
	}

	// Post a raw file
	var data postman
	err = requests.
		URL("https://postman-echo.com/post").
		BodyFile(exampleFilename).
		ContentType("text/plain").
		ToJSON(&data).
		Fetch(context.Background())
	if err != nil {
		fmt.Println("problem with postman:", err)
	}
	fmt.Println(data.Data)
}

Output:

hello, world

func (*Builder) BodyForm 

func (rb *Builder) BodyForm(data url.Values) *Builder

BodyForm sets the Builder's request body to the encoded form. It also sets the ContentType to "application/x-www-form-urlencoded".

Example 
package main

import (
	"context"
	"fmt"
	"net/url"

	"github.com/carlmjohnson/requests"
)

// Examples with the Postman echo server
type postman struct {
	Args    map[string]string `json:"args"`
	Data    string            `json:"data"`
	Headers map[string]string `json:"headers"`
	JSON    map[string]string `json:"json"`
}

func main() {
	// Submit form values
	var echo postman
	err := requests.
		URL("https://postman-echo.com/put").
		Put().
		BodyForm(url.Values{
			"hello": []string{"world"},
		}).
		ToJSON(&echo).
		Fetch(context.Background())
	if err != nil {
		fmt.Println("problem with postman:", err)
	}
	fmt.Println(echo.JSON)
}

Output:

map[hello:world]

func (*Builder) BodyJSON 

func (rb *Builder) BodyJSON(v any) *Builder

BodyJSON sets the Builder's request body to the marshaled JSON. It uses JSONSerializer to marshal the object. It also sets ContentType to "application/json".

Example 
package main

import (
	"context"
	"encoding/json"
	"fmt"
	"io"
	"os"

	"github.com/carlmjohnson/requests"
)

func main() {
	// Restore defaults after this test
	defaultSerializer := requests.JSONSerializer
	defer func() {
		requests.JSONSerializer = defaultSerializer
	}()

	data := struct {
		A string `json:"a"`
		B int    `json:"b"`
		C []bool `json:"c"`
	}{
		"Hello", 42, []bool{true, false},
	}

	// Build a request using the default JSON serializer
	req, err := requests.
		New().
		BodyJSON(&data).
		Request(context.Background())
	if err != nil {
		panic(err)
	}

	// JSON is packed in with no whitespace
	io.Copy(os.Stdout, req.Body)
	fmt.Println()

	// Change the default JSON serializer to indent with two spaces
	requests.JSONSerializer = func(v any) ([]byte, error) {
		return json.MarshalIndent(v, "", "  ")
	}

	// Build a new request using the new indenting serializer
	req, err = requests.
		New().
		BodyJSON(&data).
		Request(context.Background())
	if err != nil {
		panic(err)
	}

	// Now the request body is indented
	io.Copy(os.Stdout, req.Body)
	fmt.Println()

}

Output:

{"a":"Hello","b":42,"c":[true,false]}
{
  "a": "Hello",
  "b": 42,
  "c": [
    true,
    false
  ]
}

func (*Builder) BodyReader 

func (rb *Builder) BodyReader(r io.Reader) *Builder

BodyReader sets the Builder's request body to r.

Example 
package main

import (
	"context"
	"fmt"
	"log"
	"os"
	"path/filepath"

	"github.com/carlmjohnson/requests"
)

// Examples with the Postman echo server
type postman struct {
	Args    map[string]string `json:"args"`
	Data    string            `json:"data"`
	Headers map[string]string `json:"headers"`
	JSON    map[string]string `json:"json"`
}

func main() {
	// temp file creation boilerplate
	dir, err := os.MkdirTemp("", "body_reader_*")
	if err != nil {
		log.Fatal(err)
	}
	defer os.RemoveAll(dir) // clean up

	exampleFilename := filepath.Join(dir, "example.txt")
	exampleContent := `hello, world`
	if err := os.WriteFile(exampleFilename, []byte(exampleContent), 0644); err != nil {
		log.Fatal(err)
	}

	// suppose there is some io.Reader you want to stream from
	f, err := os.Open(exampleFilename)
	if err != nil {
		log.Fatal(err)
	}
	defer f.Close()

	// send the raw file to server
	var echo postman
	err = requests.
		URL("https://postman-echo.com/post").
		ContentType("text/plain").
		BodyReader(f).
		ToJSON(&echo).
		Fetch(context.Background())
	if err != nil {
		fmt.Println("problem with postman:", err)
	}
	fmt.Println(echo.Data)
}

Output:

hello, world

func (*Builder) BodySerializer added in v0.23.5 

func (rb *Builder) BodySerializer(s Serializer, v any) *Builder

BodySerializer sets the Builder's request body to the serialized object.

Example 
package main

import (
	"bytes"
	"context"
	"encoding/binary"
	"fmt"
	"io"

	"github.com/carlmjohnson/requests"
)

func main() {
	// Have some binary data
	data := struct {
		Header  [3]byte
		Payload uint32
	}{
		Header:  [3]byte([]byte("ABC")),
		Payload: 0xbadc0fee,
	}
	// Serialize it by just shoving data onto the wire
	serializer := func(v any) ([]byte, error) {
		var buf bytes.Buffer
		err := binary.Write(&buf, binary.BigEndian, v)
		return buf.Bytes(), err
	}
	// Make a request using the serializer
	req, err := requests.
		New().
		BodySerializer(serializer, data).
		Request(context.Background())
	if err != nil {
		panic(err)
	}
	b, err := io.ReadAll(req.Body)
	if err != nil {
		panic(err)
	}
	// Request body is just serialized bytes
	fmt.Printf("%q", b)
}

Output:

"ABC\xba\xdc\x0f\xee"

func (*Builder) BodyWriter added in v0.21.11 

func (rb *Builder) BodyWriter(f func(w io.Writer) error) *Builder

BodyWriter pipes writes from w to the Builder's request body.

Example 
package main

import (
	"context"
	"encoding/csv"
	"fmt"
	"io"

	"github.com/carlmjohnson/requests"
)

// Examples with the Postman echo server
type postman struct {
	Args    map[string]string `json:"args"`
	Data    string            `json:"data"`
	Headers map[string]string `json:"headers"`
	JSON    map[string]string `json:"json"`
}

func main() {
	var echo postman
	err := requests.
		URL("https://postman-echo.com/post").
		ContentType("text/plain").
		BodyWriter(func(w io.Writer) error {
			cw := csv.NewWriter(w)
			cw.Write([]string{"col1", "col2"})
			cw.Write([]string{"val1", "val2"})
			cw.Flush()
			return cw.Error()
		}).
		ToJSON(&echo).
		Fetch(context.Background())
	if err != nil {
		fmt.Println("problem with postman:", err)
	}
	fmt.Printf("%q\n", echo.Data)
}

Output:

"col1,col2\nval1,val2\n"

func (*Builder) CacheControl added in v0.21.4 

func (rb *Builder) CacheControl(directive string) *Builder

CacheControl sets the client-side Cache-Control directive for a request.

func (*Builder) CheckContentType added in v0.21.2 

func (rb *Builder) CheckContentType(cts ...string) *Builder

CheckContentType adds a validator for the content type header of a response.

Example 
package main

import (
	"context"
	"errors"
	"fmt"

	"github.com/carlmjohnson/requests"
)

func main() {
	// Expect a specific status code
	err := requests.
		URL("https://jsonplaceholder.typicode.com").
		Pathf("/posts/%d", 1).
		CheckContentType("application/bison").
		Fetch(context.Background())
	if err != nil {
		if re := new(requests.ResponseError); errors.As(err, &re) {
			fmt.Println("content-type was", re.Header.Get("Content-Type"))
		}
	}
}

Output:

content-type was application/json; charset=utf-8

func (*Builder) CheckPeek added in v0.21.11 

func (rb *Builder) CheckPeek(n int, f func([]byte) error) *Builder

CheckPeek adds a validator that peeks at the first n bytes of a response body.

Example 
package main

import (
	"context"
	"fmt"
	"strings"

	"github.com/carlmjohnson/requests"
)

func main() {
	// Check that a response has a doctype
	const doctype = "<!doctype html>"
	var s string
	err := requests.
		URL("http://example.com").
		CheckPeek(len(doctype), func(b []byte) error {
			if string(b) != doctype {
				return fmt.Errorf("missing doctype: %q", b)
			}
			return nil
		}).
		ToString(&s).
		Fetch(context.Background())
	if err != nil {
		fmt.Println("could not connect to example.com:", err)
	}
	fmt.Println(
		// Final result still has the prefix
		strings.HasPrefix(s, doctype),
		// And the full body
		strings.HasSuffix(s, "</html>\n"),
	)
}

Output:

true true

func (*Builder) CheckStatus 

func (rb *Builder) CheckStatus(acceptStatuses ...int) *Builder

CheckStatus adds a validator for status code of a response.

Example 
package main

import (
	"context"
	"fmt"

	"github.com/carlmjohnson/requests"
)

func main() {
	// Expect a specific status code
	err := requests.
		URL("https://jsonplaceholder.typicode.com").
		Pathf("/posts/%d", 9001).
		CheckStatus(404).
		CheckContentType("application/json").
		Fetch(context.Background())
	if err != nil {
		fmt.Println("should be a 404:", err)
	} else {
		fmt.Println("OK")
	}
}

Output:

OK

func (*Builder) Client 

func (rb *Builder) Client(cl *http.Client) *Builder

Client sets the http.Client to use for requests. If nil, it uses http.DefaultClient.

func (*Builder) Clone 

func (rb *Builder) Clone() *Builder

Clone creates a new Builder suitable for independent mutation.

func (*Builder) Config added in v0.21.11 

func (rb *Builder) Config(cfgs ...Config) *Builder

Config allows Builder to be extended by functions that set several options at once.

func (*Builder) ContentType 

func (rb *Builder) ContentType(ct string) *Builder

ContentType sets the Content-Type header on a request.

func (*Builder) Cookie added in v0.22.0 

func (rb *Builder) Cookie(name, value string) *Builder

Cookie adds a cookie to a request. Unlike other headers, adding a cookie does not overwrite existing values.

func (*Builder) CopyHeaders added in v0.22.3 

func (rb *Builder) CopyHeaders(h map[string][]string) *Builder

CopyHeaders adds a validator which copies the response headers to h. Note that because CopyHeaders adds a validator, the DefaultValidator is disabled and must be added back manually if status code validation is desired.

Example 
package main

import (
	"context"
	"fmt"
	"net/http"
	"strings"

	"github.com/carlmjohnson/requests"
)

func main() {
	// Get headers while also getting body
	var s string
	headers := http.Header{}
	err := requests.
		URL("http://example.com").
		CopyHeaders(headers).
		// CopyHeaders disables status validation, so add it back
		CheckStatus(http.StatusOK).
		ToString(&s).
		Fetch(context.Background())
	if err != nil {
		fmt.Println("problem with example.com:", err)
	}
	fmt.Println(headers.Get("Etag"))
	fmt.Println(strings.Contains(s, "Example Domain"))
}

Output:

"3147526947+gzip"
true

func (*Builder) Delete added in v0.22.0 

func (rb *Builder) Delete() *Builder

Delete sets HTTP method to DELETE.

func (*Builder) Do 

func (rb *Builder) Do(req *http.Request) (err error)

Do calls the underlying http.Client and validates and handles any resulting response. The response body is closed after all validators and the handler run.

func (*Builder) ErrorJSON added in v0.23.1 

func (rb *Builder) ErrorJSON(v any) *Builder

ErrorJSON adds a validator that applies DefaultValidator and decodes the response as a JSON object if the DefaultValidator check fails.

Example 
package main

import (
	"context"
	"errors"
	"fmt"

	"github.com/carlmjohnson/requests"
)

func main() {
	{
		trans := requests.ReplayString(`HTTP/1.1 200 OK

	{"x": 1}`)

		var goodJSON struct{ X int }
		var errJSON struct{ Error string }
		err := requests.
			URL("http://example.com/").
			Transport(trans).
			ToJSON(&goodJSON).
			ErrorJSON(&errJSON).
			Fetch(context.Background())
		if err != nil {
			fmt.Println("Error!", err)
		} else {
			fmt.Println("X", goodJSON.X)
		}
	}
	{
		trans := requests.ReplayString(`HTTP/1.1 418 I'm a teapot

	{"error": "brewing"}`)

		var goodJSON struct{ X int }
		var errJSON struct{ Error string }
		err := requests.
			URL("http://example.com/").
			Transport(trans).
			ToJSON(&goodJSON).
			ErrorJSON(&errJSON).
			Fetch(context.Background())
		switch {
		case errors.Is(err, requests.ErrInvalidHandled):
			fmt.Println(errJSON.Error)
		case err != nil:
			fmt.Println("Error!", err)
		case err == nil:
			fmt.Println("unexpected success")
		}
	}
}

Output:

X 1
brewing

func (*Builder) Fetch 

func (rb *Builder) Fetch(ctx context.Context) (err error)

Fetch builds a request, sends it, and handles the response.

func (*Builder) Handle 

func (rb *Builder) Handle(h ResponseHandler) *Builder

Handle sets the response handler for a Builder. To use multiple handlers, use ChainHandlers.

func (*Builder) Head added in v0.21.7 

func (rb *Builder) Head() *Builder

Head sets HTTP method to HEAD.

func (*Builder) Header 

func (rb *Builder) Header(key string, values ...string) *Builder

Header sets a header on a request. It overwrites the existing values of a key.

Example 
package main

import (
	"context"
	"fmt"

	"github.com/carlmjohnson/requests"
)

// Examples with the Postman echo server
type postman struct {
	Args    map[string]string `json:"args"`
	Data    string            `json:"data"`
	Headers map[string]string `json:"headers"`
	JSON    map[string]string `json:"json"`
}

func main() {
	// Set headers
	var headers postman
	err := requests.
		URL("https://postman-echo.com/get").
		UserAgent("bond/james-bond").
		BasicAuth("bondj", "007!").
		ContentType("secret").
		Header("martini", "shaken").
		ToJSON(&headers).
		Fetch(context.Background())
	if err != nil {
		fmt.Println("problem with postman:", err)
	}
	fmt.Println(headers.Headers["user-agent"])
	fmt.Println(headers.Headers["authorization"])
	fmt.Println(headers.Headers["content-type"])
	fmt.Println(headers.Headers["martini"])
}

Output:

bond/james-bond
Basic Ym9uZGo6MDA3IQ==
secret
shaken

func (*Builder) Headers added in v0.23.4 

func (rb *Builder) Headers(m map[string][]string) *Builder

Headers calls Header with all the members of m.

Example 
package main

import (
	"context"
	"fmt"
	"net/http"

	"github.com/carlmjohnson/requests"
)

func main() {
	// Set headers conditionally
	h := make(http.Header)
	if "x-forwarded-for" != "true" {
		h.Add("x-forwarded-for", "127.0.0.1")
	}
	if "has-trace-id" != "true" {
		h.Add("x-trace-id", "abc123")
	}
	// Then add them to a request
	req, err := requests.
		URL("https://example.com").
		Headers(h).
		Request(context.Background())
	if err != nil {
		fmt.Println("Error!", err)
	}
	fmt.Println(req.Header)
}

Output:

map[X-Forwarded-For:[127.0.0.1] X-Trace-Id:[abc123]]

func (*Builder) Host 

func (rb *Builder) Host(host string) *Builder

Host sets the host for a Builder's URL. It overrides the host set by BaseURL.

func (*Builder) Hostf added in v0.21.2 

func (rb *Builder) Hostf(format string, a ...any) *Builder

Hostf calls Host with fmt.Sprintf.

func (*Builder) Method 

func (rb *Builder) Method(method string) *Builder

Method sets the HTTP method for a request. By default, requests without a body are GET, and those with a body are POST.

func (*Builder) Param 

func (rb *Builder) Param(key string, values ...string) *Builder

Param sets a query parameter on a Builder's URL. It overwrites the existing values of a key.

func (*Builder) ParamInt added in v0.22.2 

func (rb *Builder) ParamInt(key string, value int) *Builder

ParamInt converts value to a string and calls Param.

func (*Builder) Params added in v0.23.4 

func (rb *Builder) Params(m map[string][]string) *Builder

Params calls Param with all the members of m.

Example 
package main

import (
	"fmt"
	"net/url"

	"github.com/carlmjohnson/requests"
)

func main() {
	// Conditionally add parameters
	values := url.Values{"a": {"1"}}
	values.Set("b", "3")
	if "cond" != "example" {
		values.Add("b", "4")
		values.Set("c", "5")
	}

	// Then add them to the URL
	u, err := requests.
		URL("https://www.example.com/get?a=0&z=6").
		Params(values).
		URL()
	if err != nil {
		fmt.Println("Error!", err)
	}
	fmt.Println(u.String())

}

Output:

https://www.example.com/get?a=1&b=3&b=4&c=5&z=6

func (*Builder) Patch added in v0.22.0 

func (rb *Builder) Patch() *Builder

Patch sets HTTP method to PATCH.

func (*Builder) Path 

func (rb *Builder) Path(path string) *Builder

Path joins a path to a Builder's URL per the path joining rules of RFC 3986. If the path begins with /, it overrides any existing path. If the path begins with ./ or ../, the final path will be rewritten in its absolute form when creating a request.

Example 
package main

import (
	"fmt"

	"github.com/carlmjohnson/requests"
)

func main() {
	// Add an ID to a base URL path
	id := 1
	u, err := requests.
		URL("https://api.example.com/posts/").
		// inherits path /posts from base URL
		Pathf("%d", id).
		URL()
	if err != nil {
		fmt.Println("Error!", err)
	}
	fmt.Println(u.String())
}

Output:

https://api.example.com/posts/1

func (*Builder) Pathf added in v0.21.2 

func (rb *Builder) Pathf(format string, a ...any) *Builder

Pathf calls Path with fmt.Sprintf.

Note that for security reasons, you must not use %s with a user provided string!

func (*Builder) Post 

func (rb *Builder) Post() *Builder

Post sets HTTP method to POST.

Note that setting a Body causes a request to be POST by default.

func (*Builder) Put 

func (rb *Builder) Put() *Builder

Put sets HTTP method to PUT.

func (*Builder) Request 

func (rb *Builder) Request(ctx context.Context) (req *http.Request, err error)

Request builds a new http.Request with its context set.

func (*Builder) Scheme added in v0.21.7 

func (rb *Builder) Scheme(scheme string) *Builder

Scheme sets the scheme for a Builder's URL. It overrides the scheme set by BaseURL.

func (*Builder) ToBytesBuffer 

func (rb *Builder) ToBytesBuffer(buf *bytes.Buffer) *Builder

ToBytesBuffer sets the Builder to write the response body to the provided bytes.Buffer.

Example 
package main

import (
	"bytes"
	"context"
	"fmt"
	"strings"

	"github.com/carlmjohnson/requests"
)

func main() {
	// Simple GET into a buffer
	var buf bytes.Buffer
	err := requests.
		URL("http://example.com").
		ToBytesBuffer(&buf).
		Fetch(context.Background())
	if err != nil {
		fmt.Println("could not connect to example.com:", err)
	}
	fmt.Println(strings.Contains(buf.String(), "Example Domain"))
}

Output:

true

func (*Builder) ToDeserializer added in v0.23.5 

func (rb *Builder) ToDeserializer(d Deserializer, v any) *Builder

ToDeserializer sets the Builder to decode a response into v using a Deserializer.

Example 
package main

import (
	"bytes"
	"context"
	"encoding/binary"
	"fmt"

	"github.com/carlmjohnson/requests"
)

func main() {
	trans := requests.ReplayString(
		"HTTP/1.1 200 OK\r\n\r\nXYZ\x00\xde\xca\xff",
	)
	// Have some binary structure
	var data struct {
		Header  [3]byte
		Payload uint32
	}
	// Deserialize it by just pulling data off the wire
	deserializer := func(data []byte, v any) error {
		buf := bytes.NewReader(data)
		return binary.Read(buf, binary.BigEndian, v)
	}
	// Make a request using the deserializer
	err := requests.
		New().
		Transport(trans).
		ToDeserializer(deserializer, &data).
		Fetch(context.Background())
	if err != nil {
		panic(err)
	}

	// We read the data out of the response body
	fmt.Printf("%q, %X", data.Header, data.Payload)
}

Output:

"XYZ", DECAFF

func (*Builder) ToFile added in v0.21.13 

func (rb *Builder) ToFile(name string) *Builder

ToFile sets the Builder to write the response body to the given file name. The file and its parent directories are created automatically. For more advanced use cases, use ToWriter.

Example 
package main

import (
	"context"
	"fmt"
	"log"
	"os"
	"path/filepath"

	"github.com/carlmjohnson/requests"
)

func main() {
	dir, err := os.MkdirTemp("", "to_file_*")
	if err != nil {
		log.Fatal(err)
	}
	defer os.RemoveAll(dir) // clean up

	exampleFilename := filepath.Join(dir, "example.txt")

	err = requests.
		URL("http://example.com").
		ToFile(exampleFilename).
		Fetch(context.Background())

	if err != nil {
		log.Fatal(err)
	}
	stat, err := os.Stat(exampleFilename)
	if err != nil {
		log.Fatal(err)
	}
	fmt.Printf("file is %d bytes\n", stat.Size())

}

Output:

file is 1256 bytes

func (*Builder) ToHeaders added in v0.22.0 

func (rb *Builder) ToHeaders(h map[string][]string) *Builder

ToHeaders sets the method to HEAD and adds a handler which copies the response headers to h. To just copy headers, see Builder.CopyHeaders.

Example 
package main

import (
	"context"
	"fmt"
	"net/http"

	"github.com/carlmjohnson/requests"
)

func main() {
	// Send a HEAD request and look at headers
	headers := http.Header{}
	err := requests.
		URL("http://example.com").
		ToHeaders(headers).
		Fetch(context.Background())
	if err != nil {
		fmt.Println("problem with example.com:", err)
	}
	fmt.Println(headers.Get("Etag"))
}

Output:

"3147526947"

func (*Builder) ToJSON 

func (rb *Builder) ToJSON(v any) *Builder

ToJSON sets the Builder to decode a response as a JSON object.

It uses JSONDeserializer to unmarshal the object.

func (*Builder) ToString 

func (rb *Builder) ToString(sp *string) *Builder

ToString sets the Builder to write the response body to the provided string pointer.

func (*Builder) ToWriter added in v0.21.7 

func (rb *Builder) ToWriter(w io.Writer) *Builder

ToWriter sets the Builder to copy the response body into w.

Example 
package main

import (
	"context"
	"fmt"
	"log"
	"os"

	"github.com/carlmjohnson/requests"
)

func main() {
	f, err := os.CreateTemp("", "*.to_writer.html")
	if err != nil {
		log.Fatal(err)
	}
	defer os.Remove(f.Name()) // clean up

	// suppose there is some io.Writer you want to stream to
	err = requests.
		URL("http://example.com").
		ToWriter(f).
		Fetch(context.Background())
	if err != nil {
		log.Fatal(err)
	}
	if err = f.Close(); err != nil {
		log.Fatal(err)
	}
	stat, err := os.Stat(f.Name())
	if err != nil {
		log.Fatal(err)
	}
	fmt.Printf("file is %d bytes\n", stat.Size())

}

Output:

file is 1256 bytes

func (*Builder) Transport added in v0.22.0 

func (rb *Builder) Transport(rt http.RoundTripper) *Builder

Transport sets the http.RoundTripper to use for requests. If set, it makes a shallow copy of the http.Client before modifying it.

Example 
package main

import (
	"context"
	"fmt"
	"io"
	"net/http"
	"strings"

	"github.com/carlmjohnson/requests"
)

func main() {
	const text = "Hello, from transport!"
	var myCustomTransport requests.RoundTripFunc = func(req *http.Request) (res *http.Response, err error) {
		res = &http.Response{
			StatusCode: http.StatusOK,
			Body:       io.NopCloser(strings.NewReader(text)),
		}
		return
	}
	var s string
	err := requests.
		URL("x://transport.example").
		Transport(myCustomTransport).
		ToString(&s).
		Fetch(context.Background())
	if err != nil {
		fmt.Println("transport failed:", err)
	}
	fmt.Println(s == text) // true
}

Output:

true

func (*Builder) URL added in v0.23.1 

func (rb *Builder) URL() (u *url.URL, err error)

URL builds a *url.URL from the base URL and options set on the Builder. If a valid url.URL cannot be built, URL() nevertheless returns a new url.URL, so it is always safe to call u.String().

func (*Builder) UserAgent 

func (rb *Builder) UserAgent(s string) *Builder

UserAgent sets the User-Agent header.

type CheckRedirectPolicy added in v0.21.10 

type CheckRedirectPolicy = func(req *http.Request, via []*http.Request) error

CheckRedirectPolicy is a function suitable for use as CheckRedirect on an http.Client.

Example 
package main

import (
	"context"
	"fmt"
	"net/http"

	"github.com/carlmjohnson/requests"
)

func main() {
	cl := *http.DefaultClient
	cl.CheckRedirect = requests.NoFollow

	if err := requests.
		URL("https://httpbingo.org/redirect/1").
		Client(&cl).
		CheckStatus(http.StatusFound).
		Handle(func(res *http.Response) error {
			fmt.Println("Status", res.Status)
			fmt.Println("From", res.Request.URL)
			fmt.Println("To", res.Header.Get("Location"))
			return nil
		}).
		Fetch(context.Background()); err != nil {
		panic(err)
	}
}

Output:

Status 302 Found
From https://httpbingo.org/redirect/1
To /get

var NoFollow CheckRedirectPolicy = MaxFollow(0)

NoFollow is a CheckRedirectPolicy that does not follow redirects.

func MaxFollow added in v0.21.10 

func MaxFollow(n int) CheckRedirectPolicy

MaxFollow returns a CheckRedirectPolicy that follows a maximum of n redirects.

Example 
package main

import (
	"context"
	"fmt"
	"net/http"

	"github.com/carlmjohnson/requests"
)

func main() {
	cl := *http.DefaultClient
	cl.CheckRedirect = requests.MaxFollow(1)

	if err := requests.
		URL("https://httpbingo.org/redirect/2").
		Client(&cl).
		CheckStatus(http.StatusFound).
		Handle(func(res *http.Response) error {
			fmt.Println("Status", res.Status)
			fmt.Println("From", res.Request.URL)
			fmt.Println("To", res.Header.Get("Location"))
			return nil
		}).
		Fetch(context.Background()); err != nil {
		panic(err)
	}
}

Output:

Status 302 Found
From https://httpbingo.org/relative-redirect/1
To /get

type Config added in v0.21.11 

type Config = func(rb *Builder)

Config allows Builder to be extended by setting several options at once. For example, a Config might set a Body and its ContentType.

func GzipConfig added in v0.21.11 

func GzipConfig(level int, h func(gw *gzip.Writer) error) Config

GzipConfig writes a gzip stream to its request body using a callback. It also sets the appropriate Content-Encoding header and automatically closes and the stream when the callback returns.

Example 
var echo postman
err := requests.
	URL("https://postman-echo.com/post").
	ContentType("text/plain").
	Config(requests.GzipConfig(
		gzip.DefaultCompression,
		func(gw *gzip.Writer) error {
			_, err := gw.Write([]byte(`hello, world`))
			return err
		})).
	ToJSON(&echo).
	Fetch(context.Background())
if err != nil {
	fmt.Println("problem with postman:", err)
}
fmt.Println(echo.Data)

Output:

hello, world

func TestServerConfig added in v0.23.3 

func TestServerConfig(s *httptest.Server) Config

TestServerConfig returns a Config which sets the Builder's BaseURL to s.URL and the Builder's Client to s.Client().

Example 
package main

import (
	"context"
	"fmt"
	"net/http"
	"net/http/httptest"

	"github.com/carlmjohnson/requests"
)

func main() {
	// Create an httptest.Server for your project's router
	mux := http.NewServeMux()
	mux.HandleFunc("/greeting", func(w http.ResponseWriter, r *http.Request) {
		fmt.Fprintf(w, "Hello, world!")
	})
	mux.HandleFunc("/salutation", func(w http.ResponseWriter, r *http.Request) {
		fmt.Fprintf(w, "Howdy, planet!")
	})

	srv := httptest.NewServer(mux)
	defer srv.Close()

	// Now test that the handler has the expected return values
	{
		var s string
		err := requests.
			New(requests.TestServerConfig(srv)).
			Path("/greeting").
			ToString(&s).
			Fetch(context.Background())
		if err != nil {
			fmt.Println("Error!", err)
		}
		fmt.Println(s) // Hello, world!
	}
	{
		var s string
		err := requests.
			New(requests.TestServerConfig(srv)).
			Path("/salutation").
			ToString(&s).
			Fetch(context.Background())
		if err != nil {
			fmt.Println("Error!", err)
		}
		fmt.Println(s) // Howdy, planet!
	}
}

Output:

Hello, world!
Howdy, planet!

type Deserializer added in v0.23.5 

type Deserializer = func(data []byte, v any) error

Deserializer is a function that can read data in some format and store the result in v.

type ErrorKind added in v0.23.1 

type ErrorKind int8

ErrorKind indicates where an error was returned in the process of building, validating, and handling a request. Errors returned by Builder can be tested for their ErrorKind using errors.Is or errors.As. 

const (
	ErrURL       ErrorKind = iota // error building URL
	ErrRequest                    // error building the request
	ErrTransport                  // error connecting
	ErrValidator                  // validator error
	ErrHandler                    // handler error
)

Enum values for type ErrorKind

func (ErrorKind) Error added in v0.23.1 

func (ek ErrorKind) Error() string

func (ErrorKind) String added in v0.23.1 

func (i ErrorKind) String() string

type ResponseError added in v0.21.11 

type ResponseError http.Response

ResponseError is the error type produced by CheckStatus and CheckContentType.

func (*ResponseError) Error added in v0.21.11 

func (se *ResponseError) Error() string

Error fulfills the error interface.

type ResponseHandler 

type ResponseHandler = func(*http.Response) error

ResponseHandler is used to validate or handle the response to a request. 

var DefaultValidator ResponseHandler = CheckStatus(
	http.StatusOK,
	http.StatusCreated,
	http.StatusAccepted,
	http.StatusNonAuthoritativeInfo,
	http.StatusNoContent,
)

DefaultValidator is the validator applied by Builder unless otherwise specified.

func ChainHandlers 

func ChainHandlers(handlers ...ResponseHandler) ResponseHandler

ChainHandlers allows for the composing of validators or response handlers.

func CheckContentType added in v0.21.2 

func CheckContentType(cts ...string) ResponseHandler

CheckContentType validates that a response has one of the given content type headers.

func CheckPeek added in v0.21.11 

func CheckPeek(n int, f func([]byte) error) ResponseHandler

CheckPeek wraps the body of a response in a bufio.Reader and gives f a peek at the first n bytes for validation.

func CheckStatus 

func CheckStatus(acceptStatuses ...int) ResponseHandler

CheckStatus validates the response has an acceptable status code.

func CopyHeaders added in v0.22.3 

func CopyHeaders(h map[string][]string) ResponseHandler

CopyHeaders copies the response headers to h.

func ErrorJSON added in v0.23.1 

func ErrorJSON(v any) ResponseHandler

ErrorJSON is a ValidatorHandler that applies DefaultValidator and decodes the response as a JSON object if the DefaultValidator check fails.

func ToBufioReader 

func ToBufioReader(f func(r *bufio.Reader) error) ResponseHandler

ToBufioReader takes a callback which wraps the response body in a bufio.Reader.

Example 
package main

import (
	"bufio"
	"context"
	"fmt"
	"io"
	"strings"

	"github.com/carlmjohnson/requests"
)

func main() {
	// read a response line by line for a sentinel
	found := false
	err := requests.
		URL("http://example.com").
		Handle(requests.ToBufioReader(func(r *bufio.Reader) error {
			var err error
			for s := ""; err == nil; {
				if strings.Contains(s, "Example Domain") {
					found = true
					return nil
				}
				// read one line from response
				s, err = r.ReadString('\n')
			}
			if err == io.EOF {
				return nil
			}
			return err
		})).
		Fetch(context.Background())
	if err != nil {
		fmt.Println("could not connect to example.com:", err)
	}
	fmt.Println(found)
}

Output:

true

func ToBufioScanner added in v0.21.9 

func ToBufioScanner(f func(r *bufio.Scanner) error) ResponseHandler

ToBufioScanner takes a callback which wraps the response body in a bufio.Scanner.

Example 
package main

import (
	"bufio"
	"bytes"
	"context"
	"fmt"

	"github.com/carlmjohnson/requests"
)

func main() {
	// read a response line by line for a sentinel
	found := false
	needle := []byte("Example Domain")
	err := requests.
		URL("http://example.com").
		Handle(requests.ToBufioScanner(func(s *bufio.Scanner) error {
			// read one line at time from response
			for s.Scan() {
				if bytes.Contains(s.Bytes(), needle) {
					found = true
					return nil
				}
			}
			return s.Err()
		})).
		Fetch(context.Background())
	if err != nil {
		fmt.Println("could not connect to example.com:", err)
	}
	fmt.Println(found)
}

Output:

true

func ToBytesBuffer 

func ToBytesBuffer(buf *bytes.Buffer) ResponseHandler

ToBytesBuffer writes the response body to the provided bytes.Buffer.

func ToDeserializer added in v0.23.5 

func ToDeserializer(d Deserializer, v any) ResponseHandler

ToDeserializer decodes a response into v using a Deserializer.

func ToFile added in v0.21.13 

func ToFile(name string) ResponseHandler

ToFile writes the response body at the provided file path. The file and its parent directories are created automatically.

func ToHTML added in v0.21.7 

func ToHTML(n *html.Node) ResponseHandler

ToHTML parses the page with x/net/html.Parse.

Example 
package main

import (
	"context"
	"fmt"

	"github.com/carlmjohnson/requests"
	"golang.org/x/net/html"
	"golang.org/x/net/html/atom"
)

func main() {
	var doc html.Node
	err := requests.
		URL("http://example.com").
		Handle(requests.ToHTML(&doc)).
		Fetch(context.Background())
	if err != nil {
		fmt.Println("could not connect to example.com:", err)
	}
	var f func(*html.Node)
	f = func(n *html.Node) {
		if n.DataAtom == atom.A {
			for _, attr := range n.Attr {
				if attr.Key == "href" {
					fmt.Println("link:", attr.Val)
				}
			}
		}
		for c := n.FirstChild; c != nil; c = c.NextSibling {
			f(c)
		}
	}
	f(&doc)
}

Output:

link: https://www.iana.org/domains/example

func ToJSON 

func ToJSON(v any) ResponseHandler

ToJSON decodes a response as a JSON object.

It uses JSONDeserializer to unmarshal the object.

func ToString 

func ToString(sp *string) ResponseHandler

ToString writes the response body to the provided string pointer.

func ToWriter added in v0.21.7 

func ToWriter(w io.Writer) ResponseHandler

ToWriter copies the response body to w.

func ValidatorHandler added in v0.23.1 

func ValidatorHandler(v, h ResponseHandler) ResponseHandler

ValidatorHandler composes a Validator and a Handler. If the validation check fails, it triggers the handler. Any errors from validator or handler will be joined to the error returned. If the handler succeeds, the error will matching ErrInvalidHandled.

Example 
package main

import (
	"context"
	"errors"
	"fmt"
	"strings"

	"github.com/carlmjohnson/requests"
)

func main() {
	var (
		regularBody string
		errBody     string
	)

	// If we fail validation because the response is a 404,
	// we handle the body with errBody instead of regularBody
	// for separate processing.
	err := requests.
		URL("http://example.com/404").
		ToString(&regularBody).
		AddValidator(
			requests.ValidatorHandler(
				requests.DefaultValidator,
				requests.ToString(&errBody),
			)).
		Fetch(context.Background())
	switch {
	case errors.Is(err, requests.ErrInvalidHandled):
		fmt.Println("got errBody:",
			strings.Contains(errBody, "Example Domain"))
	case err != nil:
		fmt.Println("unexpected error", err)
	case err == nil:
		fmt.Println("unexpected success")
	}

	fmt.Println("got regularBody:", strings.Contains(regularBody, "Example Domain"))
}

Output:

got errBody: true
got regularBody: false

type RoundTripFunc added in v0.21.3 

type RoundTripFunc func(req *http.Request) (res *http.Response, err error)

RoundTripFunc is an adaptor to use a function as an http.RoundTripper.

Example 
// Wrap an underlying transport in order to add request middleware
baseTrans := http.DefaultClient.Transport
var checksumTransport requests.RoundTripFunc = func(req *http.Request) (res *http.Response, err error) {
	// Read and checksum the body
	b, err := io.ReadAll(req.Body)
	if err != nil {
		return nil, err
	}
	h := md5.New()
	h.Write(b)
	checksum := fmt.Sprintf("%X", h.Sum(nil))
	// Must clone requests before modifying them
	req2 := *req
	req2.Header = req.Header.Clone()
	// Add header and body to the clone
	req2.Header.Add("Checksum", checksum)
	req2.Body = io.NopCloser(bytes.NewBuffer(b))
	return baseTrans.RoundTrip(&req2)
}
var data postman
err := requests.
	URL("https://postman-echo.com/post").
	BodyBytes([]byte(`Hello, World!`)).
	ContentType("text/plain").
	Transport(checksumTransport).
	ToJSON(&data).
	Fetch(context.Background())
if err != nil {
	fmt.Println("Error!", err)
}
fmt.Println(data.Headers["checksum"])

Output:

65A8E27D8879283831B664BD8B7F0AD4

func (RoundTripFunc) RoundTrip added in v0.21.3 

func (rtf RoundTripFunc) RoundTrip(r *http.Request) (*http.Response, error)

RoundTrip implements http.RoundTripper.

type Serializer added in v0.23.5 

type Serializer = func(v any) ([]byte, error)

Serializer is a function that can convert arbitrary data to bytes in some format.

type Transport added in v0.22.0 

type Transport = http.RoundTripper

Transport is an alias of http.RoundTripper for documentation purposes.

func Caching added in v0.22.0 

func Caching(rt http.RoundTripper, basepath string) Transport

Caching returns an http.RoundTripper that attempts to read its responses from text files in basepath. If the response is absent, it caches the result of issuing the request with rt in basepath. Requests are named according to a hash of their contents. Responses are named according to the request that made them.

func DoerTransport added in v0.23.3 

func DoerTransport(cl interface {
	Do(req *http.Request) (*http.Response, error)
}) Transport

DoerTransport converts a Doer into a Transport. It exists for compatibility with other libraries. A Doer is an interface with a Do method. Users should prefer Transport, because Do is the interface of http.Client which has higher level concerns.

func LogTransport added in v0.23.2 

func LogTransport(rt http.RoundTripper, fn func(req *http.Request, res *http.Response, err error, duration time.Duration)) Transport

LogTransport returns a wrapped http.RoundTripper that calls fn with details when a response has finished. A response is considered finished when the wrapper http.RoundTripper returns an error or the Response.Body is closed, whichever comes first. To simplify logging code, a nil *http.Response is replaced with a new http.Response.

Example 
package main

import (
	"context"
	"fmt"
	"net/http"
	"time"

	"github.com/carlmjohnson/requests"
)

func main() {
	logger := func(req *http.Request, res *http.Response, err error, d time.Duration) {
		fmt.Printf("method=%q url=%q err=%v status=%q duration=%v\n",
			req.Method, req.URL, err, res.Status, d.Round(1*time.Second))
	}
	// Wrap an existing transport or use nil for http.DefaultTransport
	baseTrans := http.DefaultClient.Transport
	trans := requests.LogTransport(baseTrans, logger)
	var s string
	if err := requests.
		URL("http://example.com/").
		Transport(trans).
		ToString(&s).
		Fetch(context.Background()); err != nil {
		fmt.Println("Error!", err)
	}
	// Works for bad responses too
	baseTrans = requests.RoundTripFunc(func(req *http.Request) (*http.Response, error) {
		return nil, fmt.Errorf("can't connect")
	})
	trans = requests.LogTransport(baseTrans, logger)

	if err := requests.
		URL("http://example.com/").
		Transport(trans).
		ToString(&s).
		Fetch(context.Background()); err != nil {
		fmt.Println("Error!", err)
	}
}

Output:

method="GET" url="http://example.com/" err=<nil> status="200 OK" duration=0s
method="GET" url="http://example.com/" err=can't connect status="" duration=0s
Error! ErrTransport: Get "http://example.com/": can't connect

func PermitURLTransport added in v0.21.11 

func PermitURLTransport(rt http.RoundTripper, regex string) Transport

PermitURLTransport returns a wrapped http.RoundTripper that rejects any requests whose URL doesn't match the provided regular expression string.

PermitURLTransport will panic if the regexp does not compile.

Example 
package main

import (
	"context"
	"fmt"
	"net/http"
	"strings"

	"github.com/carlmjohnson/requests"
)

func main() {
	// Wrap an existing transport or use nil for http.DefaultTransport
	baseTrans := http.DefaultClient.Transport
	trans := requests.PermitURLTransport(baseTrans, `^http://example\.com/`)
	var s string
	if err := requests.
		URL("http://example.com/").
		Transport(trans).
		ToString(&s).
		Fetch(context.Background()); err != nil {
		panic(err)
	}
	fmt.Println(strings.Contains(s, "Example Domain"))

	if err := requests.
		URL("http://unauthorized.example.com/").
		Transport(trans).
		ToString(&s).
		Fetch(context.Background()); err != nil {
		fmt.Println(err) // unauthorized subdomain not allowed
	}
}

Output:

true
ErrTransport: Get "http://unauthorized.example.com/": requested URL not permitted by regexp: ^http://example\.com/

func Record added in v0.21.3 

func Record(rt http.RoundTripper, basepath string) Transport

Record returns an http.RoundTripper that writes out its requests and their responses to text files in basepath. Requests are named according to a hash of their contents. Responses are named according to the request that made them.

func Replay added in v0.21.3 

func Replay(basepath string) Transport

Replay returns an http.RoundTripper that reads its responses from text files in basepath. Responses are looked up according to a hash of the request.

func ReplayFS added in v0.21.5 

func ReplayFS(fsys fs.FS) Transport

ReplayFS returns an http.RoundTripper that reads its responses from text files in the fs.FS. Responses are looked up according to a hash of the request. Response file names may optionally be prefixed with comments for better human organization.

Example 
package main

import (
	"context"
	"fmt"
	"testing/fstest"

	"github.com/carlmjohnson/requests"
)

func main() {
	fsys := fstest.MapFS{
		"fsys.example - MKIYDwjs.res.txt": &fstest.MapFile{
			Data: []byte(`HTTP/1.1 200 OK
Content-Type: text/plain; charset=UTF-8
Date: Mon, 24 May 2021 18:48:50 GMT

An example response.`),
		},
	}
	var s string
	const expected = `An example response.`
	if err := requests.
		URL("http://fsys.example").
		Transport(requests.ReplayFS(fsys)).
		ToString(&s).
		Fetch(context.Background()); err != nil {
		panic(err)
	}
	fmt.Println(s == expected)
}

Output:

true

func ReplayString added in v0.21.8 

func ReplayString(rawResponse string) Transport

ReplayString returns an http.RoundTripper that always responds with a request built from rawResponse. It is intended for use in one-off tests.

Example 
package main

import (
	"context"
	"fmt"

	"github.com/carlmjohnson/requests"
)

func main() {
	const res = `HTTP/1.1 200 OK

An example response.`

	var s string
	const expected = `An example response.`
	if err := requests.
		URL("http://response.example").
		Transport(requests.ReplayString(res)).
		ToString(&s).
		Fetch(context.Background()); err != nil {
		panic(err)
	}
	fmt.Println(s == expected)
}

Output:

true

func UserAgentTransport added in v0.21.10 

func UserAgentTransport(rt http.RoundTripper, s string) Transport

UserAgentTransport returns a wrapped http.RoundTripper that sets the User-Agent header on requests to s.

Source Files

View all Source files

Directories

Path Synopsis
internal
be
Vendored copy of https://github.com/carlmjohnson/be/tree/d3d9b39d71dd594af2ce96ba7fb599233e82377c
 Vendored copy of https://github.com/carlmjohnson/be/tree/d3d9b39d71dd594af2ce96ba7fb599233e82377c
minitrue
Package minitrue - Whatever the Package holds to be the truth, is truth.
 Package minitrue - Whatever the Package holds to be the truth, is truth.
slicex
Package reqxml contains utilities for sending and receiving XML.
 Package reqxml contains utilities for sending and receiving XML.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.