shed

README

🏚 shed

shed is a Go and Ruby library implementing cross-service timeout/deadline propagation and load shedding.

What?

timeout/deadline propagation means advertising the client-side timeout for HTTP requests. shed uses the X-Client-Timeout-Ms header to do this.

load shedding means dropping requests early when under load in order to free up resources. This can be done throughout the lifetime of the request, or before processing the request (e.g. if the request has been queued for longer than its client timeout).

Why?

Uncontrolled performance degradation of a service can quickly lead to resource exhaustion and cascading failures. Client progations and load-shedding along with other techniques (such as setting appropriate timeouts, retries, circuit breakers, etc.) can improve the manner in which your service fail under load.

Propagating client timeouts and shedding requests based on this allows your services to make more informed decisions about which requests are still worth processing, saving your limited and already over-utilised resources for requests that are still meaningful to process. This avoid wasted resources, improves availability under load, and speeds up recovery times by controlling request queueing.

In experiment run in CGA1123/loadshedding-experiment-ruby load-shedding across a single service hop with client timeout propagation improved availability of services by a factor of 10 under load.

Documentation

Overview

Package shed implements client and server middleware to propagate and respect client timeouts between services.

Index

• Constants
• func Client(c *http.Client, opts ...RoundTripperOpt) *http.Client
• func DefaultTimeoutMiddleware(n http.Handler, timeout func(*http.Request) time.Duration) http.Handler
• func PropagateMiddleware(n http.Handler, opts ...PropagateMiddlewareOpt) http.Handler
• func RoundTripper(n http.RoundTripper, opts ...RoundTripperOpt) http.RoundTripper
• type PropagateMiddlewareOpt
  • func WithDelta(f func(*http.Request) time.Duration) PropagateMiddlewareOpt
• type RoundTripperOpt
  • func WithMaxTimeout(d time.Duration) RoundTripperOpt

Constants

const (
	// Header contains the header key expected to be set by incoming requests
	// in order to propagate timeouts across the network, it is expected to be
	// a string parseable into an int64 which represents the timeout of the
	// client in milliseconds.
	Header = "X-Client-Timeout-Ms"
)

Functions

func Client

func Client(c *http.Client, opts ...RoundTripperOpt) *http.Client

Client builds a new *http.Client from the given *http.Client, wrapping the given client's Transport using RoundTripper.

func DefaultTimeoutMiddleware

func DefaultTimeoutMiddleware(n http.Handler, timeout func(*http.Request) time.Duration) http.Handler

DefaultTimeoutMiddleware wraps the given handler with a default context deadline propagated via the request context.

The timeout function can be used to have dynamic request based upper bounds for requests. If this function returns a time.Duration that is not strictly greater than 0, no timeout will be set.

func PropagateMiddleware

func PropagateMiddleware(n http.Handler, opts ...PropagateMiddlewareOpt) http.Handler

PropagateMiddleware builds a new http.Handler middleware which sets a context timeout on incoming requests if the client has propagated its timeout via the `X-Client-Timeout-Ms` header.

func RoundTripper

func RoundTripper(n http.RoundTripper, opts ...RoundTripperOpt) http.RoundTripper

RoundTripper builds a new http.RoundTripper which propagates context deadlines over the network via the `X-Client-Timeout-Ms` request header.

Types

type PropagateMiddlewareOpt

type PropagateMiddlewareOpt func(*propagateMiddleware)

PropagateMiddlewareOpt is a function which can modify the behaviour of the shed middleware.

func WithDelta

func WithDelta(f func(*http.Request) time.Duration) PropagateMiddlewareOpt

WithDelta allows for adjusting the timeout set by the Middleware, in order to account for time spent in the network or on various server queues.

The value returned by this function will by subtracted from the `X-Client-Timeout-Ms` value.

type RoundTripperOpt

type RoundTripperOpt func(*roundTripper)

RoundTripperOpt is a function which can modify the behaviour of the shed client transport middleware.

func WithMaxTimeout

func WithMaxTimeout(d time.Duration) RoundTripperOpt

WithMaxTimeout will set a default X-Client-Timeout-Ms if it is lower than any context.Context deadline on the request.

This is intended to be used in cases where some other timeouts are set on the client, e.g. ResponseHeaderTimeout.