fastrpc

package module
Version: v0.0.0-...-0b1a558 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Oct 1, 2017 License: MIT Imports: 17 Imported by: 4

README

Build Status GoDoc Go Report

fastrpc

Building blocks for fast rpc systems.

Features

  • Optimized for speed.
  • Zero memory allocations in hot paths.
  • Compression saves network bandwidth.

How does it work?

It just sends batched rpc requests and responses over a single compressed connection. This solves the following issues:

  • High network bandwidth usage.
  • High network packets rate.
  • A lot of open TCP connections.

Benchmark results

GOMAXPROCS=1 go test -bench=. -benchmem
goos: linux
goarch: amd64
pkg: github.com/valyala/fastrpc
BenchmarkCoarseTimeNow             	500000000	         3.91 ns/op	       0 B/op	       0 allocs/op
BenchmarkTimeNow                   	30000000	        47.1 ns/op	       0 B/op	       0 allocs/op
BenchmarkEndToEndNoDelay1          	  500000	      3442 ns/op	  75.81 MB/s	       0 B/op	       0 allocs/op
BenchmarkEndToEndNoDelay10         	  500000	      3440 ns/op	  75.85 MB/s	       2 B/op	       0 allocs/op
BenchmarkEndToEndNoDelay100        	  500000	      3448 ns/op	  75.69 MB/s	       2 B/op	       0 allocs/op
BenchmarkEndToEndNoDelay1000       	  500000	      3440 ns/op	  75.86 MB/s	       8 B/op	       0 allocs/op
BenchmarkEndToEndNoDelay10K        	  300000	      3960 ns/op	  65.90 MB/s	      79 B/op	       0 allocs/op
BenchmarkEndToEndDelay1ms          	  500000	      3395 ns/op	  76.87 MB/s	       6 B/op	       0 allocs/op
BenchmarkEndToEndDelay2ms          	  500000	      3519 ns/op	  74.16 MB/s	       6 B/op	       0 allocs/op
BenchmarkEndToEndDelay4ms          	  300000	      4562 ns/op	  57.20 MB/s	      10 B/op	       0 allocs/op
BenchmarkEndToEndDelay8ms          	  200000	      8909 ns/op	  29.30 MB/s	      15 B/op	       0 allocs/op
BenchmarkEndToEndDelay16ms         	  100000	     16647 ns/op	  15.68 MB/s	      31 B/op	       0 allocs/op
BenchmarkEndToEndCompressNone      	  500000	      3376 ns/op	  77.29 MB/s	       6 B/op	       0 allocs/op
BenchmarkEndToEndCompressFlate     	  200000	      8154 ns/op	  32.01 MB/s	      49 B/op	       0 allocs/op
BenchmarkEndToEndCompressSnappy    	  500000	      3665 ns/op	  71.20 MB/s	      10 B/op	       0 allocs/op
BenchmarkEndToEndTLSCompressNone   	  300000	      3892 ns/op	  67.06 MB/s	      15 B/op	       0 allocs/op
BenchmarkEndToEndTLSCompressFlate  	  200000	      8474 ns/op	  30.80 MB/s	      51 B/op	       0 allocs/op
BenchmarkEndToEndTLSCompressSnappy 	  300000	      3851 ns/op	  67.77 MB/s	      18 B/op	       0 allocs/op
BenchmarkEndToEndPipeline1         	  500000	      3249 ns/op	  80.31 MB/s	       0 B/op	       0 allocs/op
BenchmarkEndToEndPipeline10        	  500000	      3232 ns/op	  80.75 MB/s	       2 B/op	       0 allocs/op
BenchmarkEndToEndPipeline100       	  500000	      3237 ns/op	  80.62 MB/s	       2 B/op	       0 allocs/op
BenchmarkEndToEndPipeline1000      	  500000	      3215 ns/op	  81.17 MB/s	       8 B/op	       0 allocs/op
BenchmarkSendNowait                	 3000000	       451 ns/op	       0 B/op	       0 allocs/op

GOMAXPROCS=4 go test -bench=. -benchmem
goos: linux
goarch: amd64
pkg: github.com/valyala/fastrpc
BenchmarkCoarseTimeNow-4               	1000000000	         2.37 ns/op	       0 B/op	       0 allocs/op
BenchmarkTimeNow-4                     	100000000	        17.7 ns/op	       0 B/op	       0 allocs/op
BenchmarkEndToEndNoDelay1-4            	 1000000	      1726 ns/op	 151.14 MB/s	       1 B/op	       0 allocs/op
BenchmarkEndToEndNoDelay10-4           	 1000000	      1757 ns/op	 148.47 MB/s	       1 B/op	       0 allocs/op
BenchmarkEndToEndNoDelay100-4          	 1000000	      1768 ns/op	 147.59 MB/s	       2 B/op	       0 allocs/op
BenchmarkEndToEndNoDelay1000-4         	 1000000	      1610 ns/op	 162.09 MB/s	      11 B/op	       0 allocs/op
BenchmarkEndToEndNoDelay10K-4          	 1000000	      1825 ns/op	 142.94 MB/s	      41 B/op	       0 allocs/op
BenchmarkEndToEndDelay1ms-4            	 1000000	      1656 ns/op	 157.57 MB/s	       8 B/op	       0 allocs/op
BenchmarkEndToEndDelay2ms-4            	 1000000	      1715 ns/op	 152.10 MB/s	       8 B/op	       0 allocs/op
BenchmarkEndToEndDelay4ms-4            	 1000000	      1635 ns/op	 159.62 MB/s	       7 B/op	       0 allocs/op
BenchmarkEndToEndDelay8ms-4            	  500000	      2322 ns/op	 112.36 MB/s	      15 B/op	       0 allocs/op
BenchmarkEndToEndDelay16ms-4           	  300000	      4944 ns/op	  52.78 MB/s	      27 B/op	       0 allocs/op
BenchmarkEndToEndCompressNone-4        	 1000000	      1688 ns/op	 154.58 MB/s	       8 B/op	       0 allocs/op
BenchmarkEndToEndCompressFlate-4       	  500000	      3129 ns/op	  83.40 MB/s	      29 B/op	       0 allocs/op
BenchmarkEndToEndCompressSnappy-4      	 1000000	      1720 ns/op	 151.68 MB/s	       9 B/op	       0 allocs/op
BenchmarkEndToEndTLSCompressNone-4     	 1000000	      1780 ns/op	 146.58 MB/s	       9 B/op	       0 allocs/op
BenchmarkEndToEndTLSCompressFlate-4    	  500000	      3133 ns/op	  83.29 MB/s	      29 B/op	       0 allocs/op
BenchmarkEndToEndTLSCompressSnappy-4   	 1000000	      1758 ns/op	 148.45 MB/s	      10 B/op	       0 allocs/op
BenchmarkEndToEndPipeline1-4           	 1000000	      1638 ns/op	 159.27 MB/s	       1 B/op	       0 allocs/op
BenchmarkEndToEndPipeline10-4          	 1000000	      1646 ns/op	 158.56 MB/s	       1 B/op	       0 allocs/op
BenchmarkEndToEndPipeline100-4         	 1000000	      1623 ns/op	 160.78 MB/s	       2 B/op	       0 allocs/op
BenchmarkEndToEndPipeline1000-4        	 1000000	      1386 ns/op	 188.20 MB/s	      10 B/op	       0 allocs/op
BenchmarkSendNowait-4                  	 5000000	       327 ns/op	       1 B/op	       0 allocs/op

Users

Documentation

Index

Constants

View Source
const (
	// DefaultMaxPendingRequests is the default number of pending requests
	// a single Client may queue before sending them to the server.
	//
	// This parameter may be overridden by Client.MaxPendingRequests.
	DefaultMaxPendingRequests = 1000

	// DefaultConcurrency is the default maximum number of concurrent
	// Server.Handler goroutines the server may run.
	DefaultConcurrency = 10000
)
View Source
const (
	// DefaultReadBufferSize is the default size for read buffers.
	DefaultReadBufferSize = 64 * 1024

	// DefaultWriteBufferSize is the default size for write buffers.
	DefaultWriteBufferSize = 64 * 1024
)
View Source
const (
	// CompressNone disables connection compression.
	//
	// CompressNone may be used in the following cases:
	//
	//   * If network bandwidth between client and server is unlimited.
	//   * If client and server are located on the same physical host.
	//   * If other CompressType values consume a lot of CPU resources.
	//
	CompressNone = CompressType(1)

	// CompressFlate uses compress/flate with default
	// compression level for connection compression.
	//
	// CompressFlate may be used in the following cases:
	//
	//     * If network bandwidth between client and server is limited.
	//     * If client and server are located on distinct physical hosts.
	//     * If both client and server have enough CPU resources
	//       for compression processing.
	//
	CompressFlate = CompressType(0)

	// CompressSnappy uses snappy compression.
	//
	// CompressSnappy vs CompressFlate comparison:
	//
	//     * CompressSnappy consumes less CPU resources.
	//     * CompressSnappy consumes more network bandwidth.
	//
	CompressSnappy = CompressType(2)
)

Variables

View Source
var (
	// ErrTimeout is returned from timed out calls.
	ErrTimeout = fasthttp.ErrTimeout

	// ErrPendingRequestsOverflow is returned when Client cannot send
	// more requests to the server due to Client.MaxPendingRequests limit.
	ErrPendingRequestsOverflow = errors.New("Pending requests overflow. Increase Client.MaxPendingRequests, " +
		"reduce requests rate or speed up the server")
)

Functions

This section is empty.

Types

type Client

type Client struct {
	// SniffHeader is the header written to each connection established
	// to the server.
	//
	// It is expected that the server replies with the same header.
	SniffHeader string

	// ProtocolVersion is the version of RequestWriter and ResponseReader.
	//
	// The ProtocolVersion must be changed each time RequestWriter
	// or ResponseReader changes the underlying format.
	ProtocolVersion byte

	// NewResponse must return new response object.
	NewResponse func() ResponseReader

	// Addr is the Server address to connect to.
	Addr string

	// CompressType is the compression type used for requests.
	//
	// CompressFlate is used by default.
	CompressType CompressType

	// Dial is a custom function used for connecting to the Server.
	//
	// fasthttp.Dial is used by default.
	Dial func(addr string) (net.Conn, error)

	// TLSConfig is TLS (aka SSL) config used for establishing encrypted
	// connection to the server.
	//
	// Encrypted connections may be used for transferring sensitive
	// information over untrusted networks.
	//
	// By default connection to the server isn't encrypted.
	TLSConfig *tls.Config

	// MaxPendingRequests is the maximum number of pending requests
	// the client may issue until the server responds to them.
	//
	// DefaultMaxPendingRequests is used by default.
	MaxPendingRequests int

	// MaxBatchDelay is the maximum duration before pending requests
	// are sent to the server.
	//
	// Requests' batching may reduce network bandwidth usage and CPU usage.
	//
	// By default requests are sent immediately to the server.
	MaxBatchDelay time.Duration

	// Maximum duration for full response reading (including body).
	//
	// This also limits idle connection lifetime duration.
	//
	// By default response read timeout is unlimited.
	ReadTimeout time.Duration

	// Maximum duration for full request writing (including body).
	//
	// By default request write timeout is unlimited.
	WriteTimeout time.Duration

	// ReadBufferSize is the size for read buffer.
	//
	// DefaultReadBufferSize is used by default.
	ReadBufferSize int

	// WriteBufferSize is the size for write buffer.
	//
	// DefaultWriteBufferSize is used by default.
	WriteBufferSize int

	// Prioritizes new requests over old requests if MaxPendingRequests pending
	// requests is reached.
	PrioritizeNewRequests bool
	// contains filtered or unexported fields
}

Client sends rpc requests to the Server over a single connection.

Use multiple clients for establishing multiple connections to the server if a single connection processing consumes 100% of a single CPU core on either multi-core client or server.

func (*Client) DoDeadline

func (c *Client) DoDeadline(req RequestWriter, resp ResponseReader, deadline time.Time) error

DoDeadline sends the given request to the server set in Client.Addr.

ErrTimeout is returned if the server didn't return response until the given deadline.

func (*Client) PendingRequests

func (c *Client) PendingRequests() int

PendingRequests returns the number of pending requests at the moment.

This function may be used either for informational purposes or for load balancing purposes.

func (*Client) SendNowait

func (c *Client) SendNowait(req RequestWriter, releaseReq func(req RequestWriter)) bool

SendNowait schedules the given request for sending to the server set in Client.Addr.

req cannot be used after SendNowait returns and until releaseReq is called. releaseReq is called when the req is no longer needed and may be re-used.

req cannot be re-used if releaseReq is nil.

Returns true if the request is successfully scheduled for sending, otherwise returns false.

Response for the given request is ignored.

type CompressType

type CompressType byte

CompressType is a compression type used for connections.

type HandlerCtx

type HandlerCtx interface {
	// ConcurrencyLimitError must set the response
	// to 'concurrency limit exceeded' error.
	ConcurrencyLimitError(concurrency int)

	// Init must prepare ctx for reading the next request.
	Init(conn net.Conn, logger fasthttp.Logger)

	// ReadRequest must read request from br.
	ReadRequest(br *bufio.Reader) error

	// WriteResponse must write response to bw.
	WriteResponse(bw *bufio.Writer) error
}

HandlerCtx is an interface implementing context passed to Server.Handler

type RequestWriter

type RequestWriter interface {
	// WriteRequest must write request to bw.
	WriteRequest(bw *bufio.Writer) error
}

RequestWriter is an interface for writing rpc request to buffered writer.

type ResponseReader

type ResponseReader interface {
	// ReadResponse must read response from br.
	ReadResponse(br *bufio.Reader) error
}

ResponseReader is an interface for reading rpc response from buffered reader.

type Server

type Server struct {
	// SniffHeader is the header read from each client connection.
	//
	// The server sends the same header to each client.
	SniffHeader string

	// ProtocolVersion is the version of HandlerCtx.ReadRequest
	// and HandlerCtx.WriteResponse.
	//
	// The ProtocolVersion must be changed each time HandlerCtx.ReadRequest
	// or HandlerCtx.WriteResponse changes the underlying format.
	ProtocolVersion byte

	// NewHandlerCtx must return new HandlerCtx
	NewHandlerCtx func() HandlerCtx

	// Handler must process incoming requests.
	//
	// The handler must return either ctx passed to the call
	// or new non-nil ctx.
	//
	// The handler may return ctx passed to the call only if the ctx
	// is no longer used after returning from the handler.
	// Otherwise new ctx must be returned.
	Handler func(ctx HandlerCtx) HandlerCtx

	// CompressType is the compression type used for responses.
	//
	// CompressFlate is used by default.
	CompressType CompressType

	// Concurrency is the maximum number of concurrent goroutines
	// with Server.Handler the server may run.
	//
	// DefaultConcurrency is used by default.
	Concurrency int

	// TLSConfig is TLS (aka SSL) config used for accepting encrypted
	// client connections.
	//
	// Encrypted connections may be used for transferring sensitive
	// information over untrusted networks.
	//
	// By default server accepts only unencrypted connections.
	TLSConfig *tls.Config

	// MaxBatchDelay is the maximum duration before ready responses
	// are sent to the client.
	//
	// Responses' batching may reduce network bandwidth usage and CPU usage.
	//
	// By default responses are sent immediately to the client.
	MaxBatchDelay time.Duration

	// Maximum duration for reading the full request (including body).
	//
	// This also limits the maximum lifetime for idle connections.
	//
	// By default request read timeout is unlimited.
	ReadTimeout time.Duration

	// Maximum duration for writing the full response (including body).
	//
	// By default response write timeout is unlimited.
	WriteTimeout time.Duration

	// ReadBufferSize is the size for read buffer.
	//
	// DefaultReadBufferSize is used by default.
	ReadBufferSize int

	// WriteBufferSize is the size for write buffer.
	//
	// DefaultWriteBufferSize is used by default.
	WriteBufferSize int

	// Logger, which is used by the Server.
	//
	// Standard logger from log package is used by default.
	Logger fasthttp.Logger

	// PipelineRequests enables requests' pipelining.
	//
	// Requests from a single client are processed serially
	// if is set to true.
	//
	// Enabling requests' pipelining may be useful in the following cases:
	//
	//   - if requests from a single client must be processed serially;
	//   - if the Server.Handler doesn't block and maximum throughput
	//     must be achieved for requests' processing.
	//
	// By default requests from a single client are processed concurrently.
	PipelineRequests bool
	// contains filtered or unexported fields
}

Server accepts rpc requests from Client.

func (*Server) Serve

func (s *Server) Serve(ln net.Listener) error

Serve serves rpc requests accepted from the given listener.

Directories

Path Synopsis
Package tlv provides 'Type-Length-Value' building blocks for fastrpc.
Package tlv provides 'Type-Length-Value' building blocks for fastrpc.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
t or T : Toggle theme light dark auto
y or Y : Canonical URL