httpclient

README

httpclient

Advanced HTTP client for golang.

Features

• Chainable API
• Direct file upload
• Timeout
• SOCKS5 Proxy
• Cookie
• GZIP
• Redirect Policy
• Cancel(with context)

Installation

go get code.afis.me/httpclient

Original code from this, i'm add some features, like SOCKS5 proxy
https://github.com/ddliu/go-httpclient

Quick Start

package main

import (
    "code.afis.me/httpclient"
)

func main() {
    httpclient.Defaults(httpclient.Map {
        httpclient.OPT_USERAGENT: "my awsome httpclient",
        "Accept-Language": "en-us",
    })

    res, err := httpclient.Get("http://google.com/search", map[string]string{
        "q": "news",
    })

    println(res.StatusCode, err)
}

Usage

Setup

Use httpclient.Defaults to setup default behaviors of the HTTP client.

httpclient.Defaults(httpclient.Map {
    httpclient.OPT_USERAGENT: "my awsome httpclient",
    "Accept-Language": "en-us",
})

The OPT_XXX options define basic behaviours of this client, other values are default request headers of this request. They are shared between different HTTP requests.

Sending Request

// get
httpclient.Get("http://httpbin.org/get", map[string]string{
    "q": "news",
})

// get with url.Values
httpclient.Get("http://httpbin.org/get", url.Values{
    "q": []string{"news", "today"}
})

// post
httpclient.Post("http://httpbin.org/post", map[string]string {
    "name": "value"
})

// post file(multipart)
httpclient.Post("http://httpbin.org/multipart", map[string]string {
    "@file": "/tmp/hello.pdf",
})

// put json
httpclient.PutJson("http://httpbin.org/put", 
`{
    "name": "hello",
}`)

// delete
httpclient.Delete("http://httpbin.org/delete")

// options
httpclient.Options("http://httpbin.org")

// head
httpclient.Head("http://httpbin.org/get")

Customize Request

Before you start a new HTTP request with Get or Post method, you can specify temporary options, headers or cookies for current request.

httpclient.
    WithHeader("User-Agent", "Super Robot").
    WithHeader("custom-header", "value").
    WithHeaders(map[string]string {
        "another-header": "another-value",
        "and-another-header": "another-value",
    }).
    WithOption(httpclient.OPT_TIMEOUT, 60).
    WithCookie(&http.Cookie{
        Name: "uid",
        Value: "123",
    }).
    Get("http://github.com")

Response

The httpclient.Response is a thin wrap of http.Response.

// traditional
res, err := httpclient.Get("http://google.com")
bodyBytes, err := ioutil.ReadAll(res.Body)
res.Body.Close()

// ToString
res, err = httpclient.Get("http://google.com")
bodyString, err := res.ToString()

// ReadAll
res, err = httpclient.Get("http://google.com")
bodyBytes, err := res.ReadAll()

Handle Cookies

url := "http://github.com"
httpclient.
    WithCookie(&http.Cookie{
        Name: "uid",
        Value: "123",
    }).
    Get(url)

for _, cookie := range httpclient.Cookies() {
    fmt.Println(cookie.Name, cookie.Value)
}

for k, v := range httpclient.CookieValues() {
    fmt.Println(k, v)
}

fmt.Println(httpclient.CookieValue("uid"))

Concurrent Safe

If you want to start many requests concurrently, remember to call the Begin method when you begin:

go func() {
    httpclient.
        Begin().
        WithHeader("Req-A", "a").
        Get("http://google.com")
}()
go func() {
    httpclient.
        Begin().
        WithHeader("Req-B", "b").
        Get("http://google.com")
}()

Error Checking

You can use httpclient.IsTimeoutError to check for timeout error:

res, err := httpclient.Get("http://google.com")
if httpclient.IsTimeoutError(err) {
    // do something
}

Full Example

See examples/main.go

Options

Available options as below:

• OPT_FOLLOWLOCATION: TRUE to follow any "Location: " header that the server sends as part of the HTTP header. Default to true.
• OPT_CONNECTTIMEOUT: The number of seconds to wait while trying to connect. Use 0 to wait indefinitely.
• OPT_CONNECTTIMEOUT_MS: The number of milliseconds to wait while trying to connect. Use 0 to wait indefinitely.
• OPT_MAXREDIRS: The maximum amount of HTTP redirections to follow. Use this option alongside OPT_FOLLOWLOCATION.
• OPT_PROXYTYPE: Specify the proxy type. Valid options are PROXY_HTTP, PROXY_SOCKS4, PROXY_SOCKS5, PROXY_SOCKS4A. Only PROXY_SOCKS5 is supported currently.
• OPT_TIMEOUT: The maximum number of seconds to allow httpclient functions to execute.
• OPT_TIMEOUT_MS: The maximum number of milliseconds to allow httpclient functions to execute.
• OPT_COOKIEJAR: Set to true to enable the default cookiejar, or you can set to a http.CookieJar instance to use a customized jar. Default to true.
• OPT_INTERFACE: TODO
• OPT_PROXY: ProxyURL.
• OPT_REFERER: The Referer header of the request.
• OPT_USERAGENT: The User-Agent header of the request. Default to "go-httpclient v{{VERSION}}".
• OPT_REDIRECT_POLICY: Function to check redirect.
• OPT_PROXY_FUNC: Function to specify proxy. Valid options are PROXY_HTTP, PROXY_SOCKS4, PROXY_SOCKS5, PROXY_SOCKS4A. currently not supported.
• OPT_UNSAFE_TLS: Set to true to disable TLS certificate checking.
• OPT_DEBUG: Print request info.
• OPT_CONTEXT: Set context.context (can be used to cancel request).

Seperate Clients

By using the httpclient.Get, httpclient.Post methods etc, you are using a default shared HTTP client.

If you need more than one client in a single programme. Just create and use them seperately.

c1 := httpclient.NewHttpClient().Defaults(httpclient.Map {
    httpclient.OPT_USERAGENT: "browser1",
})

c1.Get("http://google.com/")

c2 := httpclient.NewHttpClient().Defaults(httpclient.Map {
    httpclient.OPT_USERAGENT: "browser2",
})

c2.Get("http://google.com/")

Documentation

Index

• Constants
• Variables
• func IsRedirectError(err error) bool
• func IsTimeoutError(err error) bool
• func Option(o map[string]interface{}) map[int]interface{}
• type Error
  • func (this Error) Error() string
• type HttpClient
  • func NewHttpClient() *HttpClient
  • func (cc *HttpClient) Begin() *HttpClient
  • func (cc *HttpClient) Connect(url string, params ...map[string]string) (*Response, error)
  • func (cc *HttpClient) CookieValue(url_ string, key string) string
  • func (cc *HttpClient) CookieValues(url string) map[string]string
  • func (cc *HttpClient) Cookies(url_ string) []*http.Cookie
  • func (cc *HttpClient) Defaults(defaults Map) *HttpClient
  • func (cc *HttpClient) Delete(url string, params ...interface{}) (*Response, error)
  • func (cc *HttpClient) Do(method string, url string, headers map[string]string, body io.Reader) (*Response, error)
  • func (cc *HttpClient) Get(url string, params ...interface{}) (*Response, error)
  • func (cc *HttpClient) Head(url string) (*Response, error)
  • func (cc *HttpClient) Options(url string, params ...map[string]string) (*Response, error)
  • func (cc *HttpClient) Patch(url string, params ...map[string]string) (*Response, error)
  • func (cc *HttpClient) PatchJson(url string, data interface{}) (*Response, error)
  • func (cc *HttpClient) Post(url string, params interface{}) (*Response, error)
  • func (cc *HttpClient) PostJson(url string, data interface{}) (*Response, error)
  • func (cc *HttpClient) PostMultipart(url string, params interface{}) (*Response, error)
  • func (cc *HttpClient) Put(url string, body io.Reader) (*Response, error)
  • func (cc *HttpClient) PutJson(url string, data interface{}) (*Response, error)
  • func (cc *HttpClient) Trace(url string, params ...map[string]string) (*Response, error)
  • func (cc *HttpClient) WithCookie(cookies ...*http.Cookie) *HttpClient
  • func (cc *HttpClient) WithHeader(k string, v string) *HttpClient
  • func (cc *HttpClient) WithHeaders(m map[string]string) *HttpClient
  • func (cc *HttpClient) WithOption(k int, v interface{}) *HttpClient
  • func (cc *HttpClient) WithOptions(m Map) *HttpClient
• type Map
• type ProxyURL
• type Response
  • func (cc *Response) ReadAll() ([]byte, error)
  • func (cc *Response) ToString() (string, error)

Constants

const (
	ERR_DEFAULT
	ERR_TIMEOUT
	ERR_REDIRECT_POLICY
)

Package errors

const (
	VERSION   = "0.6.6"
	USERAGENT = "go-httpclient v" + VERSION

	PROXY_HTTP    = 0
	PROXY_SOCKS4  = 4
	PROXY_SOCKS5  = 5
	PROXY_SOCKS4A = 6

	// CURL like OPT
	OPT_AUTOREFERER       = 58
	OPT_FOLLOWLOCATION    = 52
	OPT_CONNECTTIMEOUT    = 78
	OPT_CONNECTTIMEOUT_MS = 156
	OPT_MAXREDIRS         = 68
	OPT_PROXYTYPE         = 101
	OPT_TIMEOUT           = 13
	OPT_TIMEOUT_MS        = 155
	OPT_COOKIEJAR         = 10082
	OPT_INTERFACE         = 10062
	OPT_PROXY             = 10004
	OPT_REFERER           = 10016
	OPT_USERAGENT         = 10018

	// Other OPT
	OPT_REDIRECT_POLICY = 100000
	OPT_PROXY_FUNC      = 100001
	OPT_DEBUG           = 100002
	OPT_UNSAFE_TLS      = 100004

	OPT_CONTEXT = 100005
)

Constants definations CURL options, see https://github.com/bagder/curl/blob/169fedbdce93ecf14befb6e0e1ce6a2d480252a3/packages/OS400/curl.inc.in

Variables

var Begin = defaultClient.Begin

Begin -

var CONST = map[string]int{
	"OPT_AUTOREFERER":       58,
	"OPT_FOLLOWLOCATION":    52,
	"OPT_CONNECTTIMEOUT":    78,
	"OPT_CONNECTTIMEOUT_MS": 156,
	"OPT_MAXREDIRS":         68,
	"OPT_PROXYTYPE":         101,
	"OPT_TIMEOUT":           13,
	"OPT_TIMEOUT_MS":        155,
	"OPT_COOKIEJAR":         10082,
	"OPT_INTERFACE":         10062,
	"OPT_PROXY":             10004,
	"OPT_REFERER":           10016,
	"OPT_USERAGENT":         10018,

	"OPT_REDIRECT_POLICY": 100000,
	"OPT_PROXY_FUNC":      100001,
	"OPT_DEBUG":           100002,
	"OPT_UNSAFE_TLS":      100004,
	"OPT_CONTEXT":         100005,
}

CONST - String map of options

var Connect = defaultClient.Connect

Connect -

var CookieValue = defaultClient.CookieValue

CookieValue -

var CookieValues = defaultClient.CookieValues

CookieValues -

var Cookies = defaultClient.Cookies

Cookies -

var Defaults = defaultClient.Defaults

Defaults -

var Delete = defaultClient.Delete

Delete -

var Do = defaultClient.Do

Do -

var Get = defaultClient.Get

Get -

var Head = defaultClient.Head

Head -

var Options = defaultClient.Options

Options -

var Patch = defaultClient.Patch

Patch -

var PatchJson = defaultClient.PatchJson

PatchJson -

var Post = defaultClient.Post

Post -

var PostJson = defaultClient.PostJson

PostJson -

var PostMultipart = defaultClient.PostMultipart

PostMultipart -

var Put = defaultClient.Put

Put -

var PutJson = defaultClient.PutJson

PutJson -

var Trace = defaultClient.Trace

Trace -

var WithCookie = defaultClient.WithCookie

WithCookie -

var WithHeader = defaultClient.WithHeader

WithHeader -

var WithHeaders = defaultClient.WithHeaders

WithHeaders -

var WithOption = defaultClient.WithOption

WithOption -

var WithOptions = defaultClient.WithOptions

WithOptions -

Functions

func IsRedirectError

func IsRedirectError(err error) bool

Check a redirect error

func IsTimeoutError

func IsTimeoutError(err error) bool

Check a timeout error.

func Option

func Option(o map[string]interface{}) map[int]interface{}

Convert options with string keys to desired format.

Types

type Error

type Error struct {
	Code    int
	Message string
}

Custom error

func (Error) Error

func (this Error) Error() string

Implement the error interface

type HttpClient

type HttpClient struct {

	// Default headers of cc client.
	Headers map[string]string
	// contains filtered or unexported fields
}

HttpClient - Powerful and easy to use HTTP client.

func NewHttpClient

func NewHttpClient() *HttpClient

Create an HTTP client.

func (*HttpClient) Begin

func (cc *HttpClient) Begin() *HttpClient

Begin marks the begining of a request, it's necessary for concurrent requests.

func (*HttpClient) Connect

func (cc *HttpClient) Connect(url string, params ...map[string]string) (*Response, error)

The CONNECT request

func (*HttpClient) CookieValue

func (cc *HttpClient) CookieValue(url_ string, key string) string

Get cookie value of a specified cookie name.

func (*HttpClient) CookieValues

func (cc *HttpClient) CookieValues(url string) map[string]string

CookieValues - Get cookie values(k-v map) of the client jar.

func (*HttpClient) Cookies

func (cc *HttpClient) Cookies(url_ string) []*http.Cookie

Get cookies of the client jar.

func (*HttpClient) Defaults

func (cc *HttpClient) Defaults(defaults Map) *HttpClient

Defaults - Set default options and headers.

func (*HttpClient) Delete

func (cc *HttpClient) Delete(url string, params ...interface{}) (*Response, error)

The DELETE request

func (*HttpClient) Do

func (cc *HttpClient) Do(method string, url string, headers map[string]string,
	body io.Reader) (*Response, error)

Do -Start a request, and get the response. Usually we just need the Get and Post method.

func (*HttpClient) Get

func (cc *HttpClient) Get(url string, params ...interface{}) (*Response, error)

The GET request

func (*HttpClient) Head

func (cc *HttpClient) Head(url string) (*Response, error)

The HEAD request

func (*HttpClient) Options

func (cc *HttpClient) Options(url string, params ...map[string]string) (*Response, error)

The OPTIONS request

func (*HttpClient) Patch

func (cc *HttpClient) Patch(url string, params ...map[string]string) (*Response, error)

The PATCH request

func (*HttpClient) PatchJson

func (cc *HttpClient) PatchJson(url string, data interface{}) (*Response, error)

Patch json data

func (*HttpClient) Post

func (cc *HttpClient) Post(url string, params interface{}) (*Response,
	error)

The POST request

With multipart set to true, the request will be encoded as "multipart/form-data".

If any of the params key starts with "@", it is considered as a form file (similar to CURL but different).

func (*HttpClient) PostJson

func (cc *HttpClient) PostJson(url string, data interface{}) (*Response, error)

func (*HttpClient) PostMultipart

func (cc *HttpClient) PostMultipart(url string, params interface{}) (
	*Response, error)

Post with the request encoded as "multipart/form-data".

func (*HttpClient) Put

func (cc *HttpClient) Put(url string, body io.Reader) (*Response, error)

The PUT request

func (*HttpClient) PutJson

func (cc *HttpClient) PutJson(url string, data interface{}) (*Response, error)

Put json data

func (*HttpClient) Trace

func (cc *HttpClient) Trace(url string, params ...map[string]string) (*Response, error)

The TRACE request

func (*HttpClient) WithCookie

func (cc *HttpClient) WithCookie(cookies ...*http.Cookie) *HttpClient

WithCookie - Specify cookies of the current request.

func (*HttpClient) WithHeader

func (cc *HttpClient) WithHeader(k string, v string) *HttpClient

WithHeader - Temporarily specify a header of the current request.

func (*HttpClient) WithHeaders

func (cc *HttpClient) WithHeaders(m map[string]string) *HttpClient

WithHeaders - Temporarily specify multiple headers of the current request.

func (*HttpClient) WithOption

func (cc *HttpClient) WithOption(k int, v interface{}) *HttpClient

WithOption - Temporarily specify an option of the current request.

func (*HttpClient) WithOptions

func (cc *HttpClient) WithOptions(m Map) *HttpClient

WithOptions - Temporarily specify multiple options of the current request.

type Map

type Map map[interface{}]interface{}

Map is a mixed structure with options and headers

type ProxyURL

type ProxyURL struct {
	Host     string
	Scheme   string
	Path     string
	Port     int
	Origin   string
	Username string
	Password string
	WithAuth bool
}

ProxyURL -

type Response

type Response struct {
	*http.Response
}

Response - Thin wrapper of http.Response(can also be used as http.Response).

func (*Response) ReadAll

func (cc *Response) ReadAll() ([]byte, error)

ReadAll - Read response body into a byte slice.

func (*Response) ToString

func (cc *Response) ToString() (string, error)

Read response body into string.