wildcat

README

wildcat

A high performance golang HTTP parser.

Baseline benchmarking results:

zero :: evanphx/wildcat> go test -bench . -benchmem
PASS
BenchmarkParseSimple	50000000	        44.4 ns/op	       0 B/op	       0 allocs/op
BenchmarkNetHTTP	  500000	      4608 ns/op	    4627 B/op	       7 allocs/op
BenchmarkParseSimpleHeaders	20000000	       106 ns/op	       0 B/op	       0 allocs/op
BenchmarkParseSimple3Headers	10000000	       213 ns/op	       0 B/op	       0 allocs/op
BenchmarkNetHTTP3	  500000	      6733 ns/op	    5064 B/op	      11 allocs/op
ok  	github.com/evanphx/wildcat	12.665s

NOTE: these are a bit of lie because wildcat doesn't yet do everything that net/http.ReadRequest does. The numbers are included here only to provide a baseline comparison for future work.

Documentation

Overview

Wildcat provides a HTTP parser that performs zero allocations, in scans a buffer and tracks slices into the buffer to provide information about the request.

As a result, it is also quite fast. `go test -bench .` to verify for yourself.

A couple of cases where it wil perform an allocation:

1. When the parser is created, it makes enough space to track 10 headers. If there end up being more than 10, this is resized up to 20, then 30, etc.

This can be overriden by using NewSizedHTTPParser to pass in the initial size to use, eliminated allocations for your use case (you could set this to 1000 for instance).

2. If a mime-style multiline header is encountered, wildcat will make a new buffer to contain the concatination of values.

NOTE: FindHeader only returns the first header that matches the requested name. If a request contains multiple values for the same header, use FindAllHeaders.

Index

• Constants
• Variables
• func BodyReader(size int64, rest []byte, c io.ReadCloser) io.ReadCloser
• func ListenAndServe(addr string, handler Handler) error
• func ListenAndServeTLS(addr string, certFile string, keyFile string, handler Handler) error
• func StatusText(code int) string
• type HTTPParser
  • func NewHTTPParser() *HTTPParser
  • func NewSizedHTTPParser(size int) *HTTPParser
  • func (hp *HTTPParser) BodyReader(rest []byte, in io.ReadCloser) io.ReadCloser
  • func (hp *HTTPParser) ContentLength() int64
  • func (hp *HTTPParser) FindAllHeaders(name []byte) [][]byte
  • func (hp *HTTPParser) FindHeader(name []byte) []byte
  • func (hp *HTTPParser) Get() bool
  • func (hp *HTTPParser) Host() []byte
  • func (hp *HTTPParser) Parse(input []byte) (int, error)
  • func (hp *HTTPParser) Post() bool
• type Handler
  • func AdaptServeHTTP(h http.Handler) Handler
• type Redirector
• type Response
  • func NewResponse(c net.Conn) *Response
  • func (r *Response) AddHeader(key, val []byte)
  • func (r *Response) AddStringHeader(key, val string)
  • func (r *Response) WriteBodyBytes(body []byte)
  • func (r *Response) WriteBodySizedStream(size int, reader io.Reader)
  • func (r *Response) WriteBodyStream(size int, reader io.Reader)
  • func (r *Response) WriteBodyString(body string)
  • func (r *Response) WriteHeaders()
  • func (r *Response) WriteStatus(code int)
• type ReverseProxy
  • func NewReverseProxy(dir Redirector) *ReverseProxy
  • func (r *ReverseProxy) HandleConnection(hp *HTTPParser, rest []byte, c net.Conn)
• type Server
  • func (s *Server) ListenAndServe(addr string) error
  • func (srv *Server) ListenAndServeTLS(addr, certFile, keyFile string) error
  • func (s *Server) Serve(l net.Listener) error

Constants

const (
	StatusContinue           = 100
	StatusSwitchingProtocols = 101

	StatusOK                   = 200
	StatusCreated              = 201
	StatusAccepted             = 202
	StatusNonAuthoritativeInfo = 203
	StatusNoContent            = 204
	StatusResetContent         = 205
	StatusPartialContent       = 206

	StatusMultipleChoices   = 300
	StatusMovedPermanently  = 301
	StatusFound             = 302
	StatusSeeOther          = 303
	StatusNotModified       = 304
	StatusUseProxy          = 305
	StatusTemporaryRedirect = 307

	StatusBadRequest                   = 400
	StatusUnauthorized                 = 401
	StatusPaymentRequired              = 402
	StatusForbidden                    = 403
	StatusNotFound                     = 404
	StatusMethodNotAllowed             = 405
	StatusNotAcceptable                = 406
	StatusProxyAuthRequired            = 407
	StatusRequestTimeout               = 408
	StatusConflict                     = 409
	StatusGone                         = 410
	StatusLengthRequired               = 411
	StatusPreconditionFailed           = 412
	StatusRequestEntityTooLarge        = 413
	StatusRequestURITooLong            = 414
	StatusUnsupportedMediaType         = 415
	StatusRequestedRangeNotSatisfiable = 416
	StatusExpectationFailed            = 417
	StatusTeapot                       = 418

	StatusInternalServerError     = 500
	StatusNotImplemented          = 501
	StatusBadGateway              = 502
	StatusServiceUnavailable      = 503
	StatusGatewayTimeout          = 504
	StatusHTTPVersionNotSupported = 505
)

HTTP status codes, defined in RFC 2616.

const DefaultHeaderSlice = 10

const OptimalBufferSize = 1500

Variables

var (
	ErrBadProto    = errors.New("bad protocol")
	ErrMissingData = errors.New("missing data")
	ErrUnsupported = errors.New("unsupported http feature")
)

Functions

func BodyReader

func BodyReader(size int64, rest []byte, c io.ReadCloser) io.ReadCloser

func ListenAndServe

func ListenAndServe(addr string, handler Handler) error

func ListenAndServeTLS

func ListenAndServeTLS(addr string, certFile string, keyFile string, handler Handler) error

func StatusText

func StatusText(code int) string

StatusText returns a text for the HTTP status code. It returns the empty string if the code is unknown.

Types

type HTTPParser

type HTTPParser struct {
	Method, Path, Version []byte

	Headers      []header
	TotalHeaders int
	// contains filtered or unexported fields
}

func NewHTTPParser

func NewHTTPParser() *HTTPParser

Create a new parser

func NewSizedHTTPParser

func NewSizedHTTPParser(size int) *HTTPParser

Create a new parser allocating size for size headers

func (*HTTPParser) BodyReader

func (hp *HTTPParser) BodyReader(rest []byte, in io.ReadCloser) io.ReadCloser

func (*HTTPParser) ContentLength

func (hp *HTTPParser) ContentLength() int64

Return the value of the Content-Length header. A value of -1 indicates the header was not set.

func (*HTTPParser) FindAllHeaders

func (hp *HTTPParser) FindAllHeaders(name []byte) [][]byte

Return all values of a header matching name.

func (*HTTPParser) FindHeader

func (hp *HTTPParser) FindHeader(name []byte) []byte

Return a value of a header matching name.

func (*HTTPParser) Get

func (hp *HTTPParser) Get() bool

func (*HTTPParser) Host

func (hp *HTTPParser) Host() []byte

Return the value of the Host header

func (*HTTPParser) Parse

func (hp *HTTPParser) Parse(input []byte) (int, error)

Parse the buffer as an HTTP Request. The buffer must contain the entire request or Parse will return ErrMissingData for the caller to get more data. (this thusly favors getting a completed request in a single Read() call).

Returns the number of bytes used by the header (thus where the body begins). Also can return ErrUnsupported if an HTTP feature is detected but not supported.

func (*HTTPParser) Post

func (hp *HTTPParser) Post() bool

type Handler

type Handler interface {
	HandleConnection(parser *HTTPParser, rest []byte, c net.Conn)
}

func AdaptServeHTTP

func AdaptServeHTTP(h http.Handler) Handler

type Redirector

type Redirector interface {
	Redirect(hp *HTTPParser) (string, string, error)
}

type Response

type Response struct {
	// contains filtered or unexported fields
}

func NewResponse

func NewResponse(c net.Conn) *Response

func (*Response) AddHeader

func (r *Response) AddHeader(key, val []byte)

func (*Response) AddStringHeader

func (r *Response) AddStringHeader(key, val string)

func (*Response) WriteBodyBytes

func (r *Response) WriteBodyBytes(body []byte)

func (*Response) WriteBodySizedStream

func (r *Response) WriteBodySizedStream(size int, reader io.Reader)

func (*Response) WriteBodyStream

func (r *Response) WriteBodyStream(size int, reader io.Reader)

func (*Response) WriteBodyString

func (r *Response) WriteBodyString(body string)

func (*Response) WriteHeaders

func (r *Response) WriteHeaders()

func (*Response) WriteStatus

func (r *Response) WriteStatus(code int)

type ReverseProxy

type ReverseProxy struct {
	// contains filtered or unexported fields
}

func NewReverseProxy

func NewReverseProxy(dir Redirector) *ReverseProxy

func (*ReverseProxy) HandleConnection

func (r *ReverseProxy) HandleConnection(hp *HTTPParser, rest []byte, c net.Conn)

type Server

type Server struct {
	Handler Handler
}

func (*Server) ListenAndServe

func (s *Server) ListenAndServe(addr string) error

func (*Server) ListenAndServeTLS

func (srv *Server) ListenAndServeTLS(addr, certFile, keyFile string) error

ListenAndServeTLS listens on the TCP network address srv.Addr and then calls Serve to handle requests on incoming TLS connections.

Filenames containing a certificate and matching private key for the server must be provided. If the certificate is signed by a certificate authority, the certFile should be the concatenation of the server's certificate followed by the CA's certificate.

If addr is blank, ":https" is used.

func (*Server) Serve

func (s *Server) Serve(l net.Listener) error