README

go-httpclient

Travis godoc License Go Report Card cover.run

Advanced HTTP client for golang.

Features

  • Chainable API
  • Direct file upload
  • Timeout
  • HTTP Proxy
  • Cookie
  • GZIP
  • Redirect Policy
  • Cancel(with context)

Installation

go get github.com/ddliu/go-httpclient

Quick Start

package main

import (
    "github.com/ddliu/go-httpclient"
)

func main() {
    httpclient.Defaults(httpclient.Map {
        httpclient.OPT_USERAGENT: "my awsome httpclient",
        "Accept-Language": "en-us",
    })

    res, err := httpclient.Get("http://google.com/search", map[string]string{
        "q": "news",
    })

    println(res.StatusCode, err)
}

Usage

Setup

Use httpclient.Defaults to setup default behaviors of the HTTP client.

httpclient.Defaults(httpclient.Map {
    httpclient.OPT_USERAGENT: "my awsome httpclient",
    "Accept-Language": "en-us",
})

The OPT_XXX options define basic behaviours of this client, other values are default request headers of this request. They are shared between different HTTP requests.

Sending Request
// get
httpclient.Get("http://httpbin.org/get", map[string]string{
    "q": "news",
})

// get with url.Values
httpclient.Get("http://httpbin.org/get", url.Values{
    "q": []string{"news", "today"}
})

// post
httpclient.Post("http://httpbin.org/post", map[string]string {
    "name": "value"
})

// post file(multipart)
httpclient.Post("http://httpbin.org/multipart", map[string]string {
    "@file": "/tmp/hello.pdf",
})

// put json
httpclient.PutJson("http://httpbin.org/put", 
`{
    "name": "hello",
}`)

// delete
httpclient.Delete("http://httpbin.org/delete")

// options
httpclient.Options("http://httpbin.org")

// head
httpclient.Head("http://httpbin.org/get")
Customize Request

Before you start a new HTTP request with Get or Post method, you can specify temporary options, headers or cookies for current request.

httpclient.
    WithHeader("User-Agent", "Super Robot").
    WithHeader("custom-header", "value").
    WithHeaders(map[string]string {
        "another-header": "another-value",
        "and-another-header": "another-value",
    }).
    WithOption(httpclient.OPT_TIMEOUT, 60).
    WithCookie(&http.Cookie{
        Name: "uid",
        Value: "123",
    }).
    Get("http://github.com")
Response

The httpclient.Response is a thin wrap of http.Response.

// traditional
res, err := httpclient.Get("http://google.com")
bodyBytes, err := ioutil.ReadAll(res.Body)
res.Body.Close()

// ToString
res, err = httpclient.Get("http://google.com")
bodyString, err := res.ToString()

// ReadAll
res, err = httpclient.Get("http://google.com")
bodyBytes, err := res.ReadAll()
Handle Cookies
url := "http://github.com"
httpclient.
    WithCookie(&http.Cookie{
        Name: "uid",
        Value: "123",
    }).
    Get(url)

for _, cookie := range httpclient.Cookies() {
    fmt.Println(cookie.Name, cookie.Value)
}

for k, v := range httpclient.CookieValues() {
    fmt.Println(k, v)
}

fmt.Println(httpclient.CookieValue("uid"))
Concurrent Safe

If you want to start many requests concurrently, remember to call the Begin method when you begin:

go func() {
    httpclient.
        Begin().
        WithHeader("Req-A", "a").
        Get("http://google.com")
}()
go func() {
    httpclient.
        Begin().
        WithHeader("Req-B", "b").
        Get("http://google.com")
}()

Error Checking

You can use httpclient.IsTimeoutError to check for timeout error:

res, err := httpclient.Get("http://google.com")
if httpclient.IsTimeoutError(err) {
    // do something
}
Full Example

See examples/main.go

Options

Available options as below:

  • OPT_FOLLOWLOCATION: TRUE to follow any "Location: " header that the server sends as part of the HTTP header. Default to true.
  • OPT_CONNECTTIMEOUT: The number of seconds or interval (with time.Duration) to wait while trying to connect. Use 0 to wait indefinitely.
  • OPT_CONNECTTIMEOUT_MS: The number of milliseconds to wait while trying to connect. Use 0 to wait indefinitely.
  • OPT_MAXREDIRS: The maximum amount of HTTP redirections to follow. Use this option alongside OPT_FOLLOWLOCATION.
  • OPT_PROXYTYPE: Specify the proxy type. Valid options are PROXY_HTTP, PROXY_SOCKS4, PROXY_SOCKS5, PROXY_SOCKS4A. Only PROXY_HTTP is supported currently.
  • OPT_TIMEOUT: The maximum number of seconds or interval (with time.Duration) to allow httpclient functions to execute.
  • OPT_TIMEOUT_MS: The maximum number of milliseconds to allow httpclient functions to execute.
  • OPT_COOKIEJAR: Set to true to enable the default cookiejar, or you can set to a http.CookieJar instance to use a customized jar. Default to true.
  • OPT_INTERFACE: TODO
  • OPT_PROXY: Proxy host and port(127.0.0.1:1080).
  • OPT_REFERER: The Referer header of the request.
  • OPT_USERAGENT: The User-Agent header of the request. Default to "go-httpclient v{{VERSION}}".
  • OPT_REDIRECT_POLICY: Function to check redirect.
  • OPT_PROXY_FUNC: Function to specify proxy.
  • OPT_UNSAFE_TLS: Set to true to disable TLS certificate checking.
  • OPT_DEBUG: Print request info.
  • OPT_CONTEXT: Set context.context (can be used to cancel request).
  • OPT_BEFORE_REQUEST_FUNC: Function to call before request is sent, option should be type func(*http.Client, *http.Request).

Seperate Clients

By using the httpclient.Get, httpclient.Post methods etc, you are using a default shared HTTP client.

If you need more than one client in a single programme. Just create and use them seperately.

c1 := httpclient.NewHttpClient().Defaults(httpclient.Map {
    httpclient.OPT_USERAGENT: "browser1",
})

c1.Get("http://google.com/")

c2 := httpclient.NewHttpClient().Defaults(httpclient.Map {
    httpclient.OPT_USERAGENT: "browser2",
})

c2.Get("http://google.com/")

Expand ▾ Collapse ▴

Documentation

Overview

    Powerful and easy to use http client

    Powerful and easy to use http client

    Index

    Constants

    View Source
    const (
    	ERR_DEFAULT
    	ERR_TIMEOUT
    	ERR_REDIRECT_POLICY
    )

      Package errors

      View Source
      const (
      	VERSION   = "0.6.9"
      	USERAGENT = "go-httpclient v" + VERSION
      )

        Constants definations CURL options, see https://github.com/bagder/curl/blob/169fedbdce93ecf14befb6e0e1ce6a2d480252a3/packages/OS400/curl.inc.in

        View Source
        const (
        	PROXY_HTTP int = iota
        	PROXY_SOCKS4
        	PROXY_SOCKS5
        	PROXY_SOCKS4A
        
        	// CURL like OPT
        	OPT_AUTOREFERER
        	OPT_FOLLOWLOCATION
        	OPT_CONNECTTIMEOUT
        	OPT_CONNECTTIMEOUT_MS
        	OPT_MAXREDIRS
        	OPT_PROXYTYPE
        	OPT_TIMEOUT
        	OPT_TIMEOUT_MS
        	OPT_COOKIEJAR
        	OPT_INTERFACE
        	OPT_PROXY
        	OPT_REFERER
        	OPT_USERAGENT
        
        	// Other OPT
        	OPT_REDIRECT_POLICY
        	OPT_PROXY_FUNC
        	OPT_DEBUG
        	OPT_UNSAFE_TLS
        
        	OPT_CONTEXT
        
        	OPT_BEFORE_REQUEST_FUNC
        )

        Variables

        View Source
        var Begin = defaultClient.Begin
        View Source
        var CONST = map[string]int{
        	"OPT_AUTOREFERER":         OPT_AUTOREFERER,
        	"OPT_FOLLOWLOCATION":      OPT_FOLLOWLOCATION,
        	"OPT_CONNECTTIMEOUT":      OPT_CONNECTTIMEOUT,
        	"OPT_CONNECTTIMEOUT_MS":   OPT_CONNECTTIMEOUT_MS,
        	"OPT_MAXREDIRS":           OPT_MAXREDIRS,
        	"OPT_PROXYTYPE":           OPT_PROXYTYPE,
        	"OPT_TIMEOUT":             OPT_TIMEOUT,
        	"OPT_TIMEOUT_MS":          OPT_TIMEOUT_MS,
        	"OPT_COOKIEJAR":           OPT_COOKIEJAR,
        	"OPT_INTERFACE":           OPT_INTERFACE,
        	"OPT_PROXY":               OPT_PROXY,
        	"OPT_REFERER":             OPT_REFERER,
        	"OPT_USERAGENT":           OPT_USERAGENT,
        	"OPT_REDIRECT_POLICY":     OPT_REDIRECT_POLICY,
        	"OPT_PROXY_FUNC":          OPT_PROXY_FUNC,
        	"OPT_DEBUG":               OPT_DEBUG,
        	"OPT_UNSAFE_TLS":          OPT_UNSAFE_TLS,
        	"OPT_CONTEXT":             OPT_CONTEXT,
        	"OPT_BEFORE_REQUEST_FUNC": OPT_BEFORE_REQUEST_FUNC,
        }

          String map of options

          View Source
          var Connect = defaultClient.Connect
          View Source
          var CookieValue = defaultClient.CookieValue
          View Source
          var CookieValues = defaultClient.CookieValues
          View Source
          var Cookies = defaultClient.Cookies
          View Source
          var Defaults = defaultClient.Defaults
          View Source
          var Delete = defaultClient.Delete
          View Source
          var Do = defaultClient.Do
          View Source
          var Get = defaultClient.Get
          View Source
          var Head = defaultClient.Head
          View Source
          var Options = defaultClient.Options
          View Source
          var Patch = defaultClient.Patch
          View Source
          var PatchJson = defaultClient.PatchJson
          View Source
          var Post = defaultClient.Post
          View Source
          var PostJson = defaultClient.PostJson
          View Source
          var PostMultipart = defaultClient.PostMultipart
          View Source
          var Put = defaultClient.Put
          View Source
          var PutJson = defaultClient.PutJson
          View Source
          var Trace = defaultClient.Trace
          View Source
          var WithCookie = defaultClient.WithCookie
          View Source
          var WithHeader = defaultClient.WithHeader
          View Source
          var WithHeaders = defaultClient.WithHeaders
          View Source
          var WithOption = defaultClient.WithOption
          View Source
          var WithOptions = defaultClient.WithOptions

          Functions

          func IsRedirectError

          func IsRedirectError(err error) bool

            Check a redirect error

            func IsTimeoutError

            func IsTimeoutError(err error) bool

              Check a timeout error.

              func Option

              func Option(o map[string]interface{}) map[int]interface{}

                Convert options with string keys to desired format.

                Types

                type Error

                type Error struct {
                	Code    int
                	Message string
                }

                  Custom error

                  func (Error) Error

                  func (this Error) Error() string

                    Implement the error interface

                    type HttpClient

                    type HttpClient struct {
                    
                    	// Default headers of this client.
                    	Headers map[string]string
                    	// contains filtered or unexported fields
                    }

                      Powerful and easy to use HTTP client.

                      func NewHttpClient

                      func NewHttpClient() *HttpClient

                        Create an HTTP client.

                        func (*HttpClient) Begin

                        func (this *HttpClient) Begin() *HttpClient

                          Begin marks the begining of a request, it's necessary for concurrent requests.

                          func (*HttpClient) Connect

                          func (this *HttpClient) Connect(url string, params ...map[string]string) (*Response, error)

                            The CONNECT request

                            func (*HttpClient) CookieValue

                            func (this *HttpClient) CookieValue(url_ string, key string) string

                              Get cookie value of a specified cookie name.

                              func (*HttpClient) CookieValues

                              func (this *HttpClient) CookieValues(url_ string) map[string]string

                                Get cookie values(k-v map) of the client jar.

                                func (*HttpClient) Cookies

                                func (this *HttpClient) Cookies(url_ string) []*http.Cookie

                                  Get cookies of the client jar.

                                  func (*HttpClient) Defaults

                                  func (this *HttpClient) Defaults(defaults Map) *HttpClient

                                    Set default options and headers.

                                    func (*HttpClient) Delete

                                    func (this *HttpClient) Delete(url string, params ...interface{}) (*Response, error)

                                      The DELETE request

                                      func (*HttpClient) Do

                                      func (this *HttpClient) Do(method string, url string, headers map[string]string,
                                      	body io.Reader) (*Response, error)

                                        Start a request, and get the response.

                                        Usually we just need the Get and Post method.

                                        func (*HttpClient) Get

                                        func (this *HttpClient) Get(url string, params ...interface{}) (*Response, error)

                                          The GET request

                                          func (*HttpClient) Head

                                          func (this *HttpClient) Head(url string) (*Response, error)

                                            The HEAD request

                                            func (*HttpClient) Options

                                            func (this *HttpClient) Options(url string, params ...map[string]string) (*Response, error)

                                              The OPTIONS request

                                              func (*HttpClient) Patch

                                              func (this *HttpClient) Patch(url string, params ...map[string]string) (*Response, error)

                                                The PATCH request

                                                func (*HttpClient) PatchJson

                                                func (this *HttpClient) PatchJson(url string, data interface{}) (*Response, error)

                                                  Patch json data

                                                  func (*HttpClient) Post

                                                  func (this *HttpClient) Post(url string, params interface{}) (*Response,
                                                  	error)

                                                    The POST request

                                                    With multipart set to true, the request will be encoded as "multipart/form-data".

                                                    If any of the params key starts with "@", it is considered as a form file (similar to CURL but different).

                                                    func (*HttpClient) PostJson

                                                    func (this *HttpClient) PostJson(url string, data interface{}) (*Response, error)

                                                    func (*HttpClient) PostMultipart

                                                    func (this *HttpClient) PostMultipart(url string, params interface{}) (
                                                    	*Response, error)

                                                      Post with the request encoded as "multipart/form-data".

                                                      func (*HttpClient) Put

                                                      func (this *HttpClient) Put(url string, body io.Reader) (*Response, error)

                                                        The PUT request

                                                        func (*HttpClient) PutJson

                                                        func (this *HttpClient) PutJson(url string, data interface{}) (*Response, error)

                                                          Put json data

                                                          func (*HttpClient) Trace

                                                          func (this *HttpClient) Trace(url string, params ...map[string]string) (*Response, error)

                                                            The TRACE request

                                                            func (*HttpClient) WithCookie

                                                            func (this *HttpClient) WithCookie(cookies ...*http.Cookie) *HttpClient

                                                              Specify cookies of the current request.

                                                              func (*HttpClient) WithHeader

                                                              func (this *HttpClient) WithHeader(k string, v string) *HttpClient

                                                                Temporarily specify a header of the current request.

                                                                func (*HttpClient) WithHeaders

                                                                func (this *HttpClient) WithHeaders(m map[string]string) *HttpClient

                                                                  Temporarily specify multiple headers of the current request.

                                                                  func (*HttpClient) WithOption

                                                                  func (this *HttpClient) WithOption(k int, v interface{}) *HttpClient

                                                                    Temporarily specify an option of the current request.

                                                                    func (*HttpClient) WithOptions

                                                                    func (this *HttpClient) WithOptions(m Map) *HttpClient

                                                                      Temporarily specify multiple options of the current request.

                                                                      type Map

                                                                      type Map map[interface{}]interface{}

                                                                        Map is a mixed structure with options and headers

                                                                        type Response

                                                                        type Response struct {
                                                                        	*http.Response
                                                                        }

                                                                          Thin wrapper of http.Response(can also be used as http.Response).

                                                                          func (*Response) ReadAll

                                                                          func (this *Response) ReadAll() ([]byte, error)

                                                                            Read response body into a byte slice.

                                                                            func (*Response) ToString

                                                                            func (this *Response) ToString() (string, error)

                                                                              Read response body into string.

                                                                              Directories

                                                                              Path Synopsis