circuit

package module
v2.2.2+incompatible Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Oct 31, 2019 License: MIT Imports: 12 Imported by: 0

README

circuitbreaker

Circuitbreaker provides an easy way to use the Circuit Breaker pattern in a Go program.

Circuit breakers are typically used when your program makes remote calls. Remote calls can often hang for a while before they time out. If your application makes a lot of these requests, many resources can be tied up waiting for these time outs to occur. A circuit breaker wraps these remote calls and will trip after a defined amount of failures or time outs occur. When a circuit breaker is tripped any future calls will avoid making the remote call and return an error to the caller. In the meantime, the circuit breaker will periodically allow some calls to be tried again and will close the circuit if those are successful.

You can read more about this pattern and how it's used at:

GoDoc

Installation

  go get github.com/rubyist/circuitbreaker

Examples

Here is a quick example of what circuitbreaker provides

// Creates a circuit breaker that will trip if the function fails 10 times
cb := circuit.NewThresholdBreaker(10)

events := cb.Subscribe()
go func() {
  for {
    e := <-events
    // Monitor breaker events like BreakerTripped, BreakerReset, BreakerFail, BreakerReady
  }
}()

cb.Call(func() error {
	// This is where you'll do some remote call
	// If it fails, return an error
}, 0)

Circuitbreaker can also wrap a time out around the remote call.

// Creates a circuit breaker that will trip after 10 failures
// using a time out of 5 seconds
cb := circuit.NewThresholdBreaker(10)

cb.Call(func() error {
  // This is where you'll do some remote call
  // If it fails, return an error
}, time.Second * 5) // This will time out after 5 seconds, which counts as a failure

// Proceed as above

Circuitbreaker can also trip based on the number of consecutive failures.

// Creates a circuit breaker that will trip if 10 consecutive failures occur
cb := circuit.NewConsecutiveBreaker(10)

// Proceed as above

Circuitbreaker can trip based on the error rate.

// Creates a circuit breaker based on the error rate
cb := circuit.NewRateBreaker(0.95, 100) // trip when error rate hits 95%, with at least 100 samples

// Proceed as above

If it doesn't make sense to wrap logic in Call(), breakers can be handled manually.

cb := circuit.NewThresholdBreaker(10)

for {
  if cb.Ready() {
    // Breaker is not tripped, proceed
    err := doSomething()
    if err != nil {
      cb.Fail() // This will trip the breaker once it's failed 10 times
      continue
    }
    cb.Success()
  } else {
    // Breaker is in a tripped state.
  }
}

Circuitbreaker also provides a wrapper around http.Client that will wrap a time out around any request.

// Passing in nil will create a regular http.Client.
// You can also build your own http.Client and pass it in
client := circuit.NewHTTPClient(time.Second * 5, 10, nil)

resp, err := client.Get("http://example.com/resource.json")

See the godoc for more examples.

Bugs, Issues, Feedback

Right here on GitHub: https://github.com/rubyist/circuitbreaker

Documentation

Overview

Package circuit implements the Circuit Breaker pattern. It will wrap a function call (typically one which uses remote services) and monitors for failures and/or time outs. When a threshold of failures or time outs has been reached, future calls to the function will not run. During this state, the breaker will periodically allow the function to run and, if it is successful, will start running the function again.

Circuit includes three types of circuit breakers:

A Threshold Breaker will trip when the failure count reaches a given threshold. It does not matter how long it takes to reach the threshold and the failures do not need to be consecutive.

A Consecutive Breaker will trip when the consecutive failure count reaches a given threshold. It does not matter how long it takes to reach the threshold, but the failures do need to be consecutive.

When wrapping blocks of code with a Breaker's Call() function, a time out can be specified. If the time out is reached, the breaker's Fail() function will be called.

Other types of circuit breakers can be easily built by creating a Breaker and adding a custom TripFunc. A TripFunc is called when a Breaker Fail()s and receives the breaker as an argument. It then returns true or false to indicate whether the breaker should trip.

The package also provides a wrapper around an http.Client that wraps all of the http.Client functions with a Breaker.

Index

Examples

Constants

This section is empty.

Variables

View Source 
var (
	ErrBreakerOpen    = errors.New("breaker open")
	ErrBreakerTimeout = errors.New("breaker time out")
)

Error codes returned by Call

View Source 
var (
	// DefaultWindowTime is the default time the window covers, 10 seconds.
	DefaultWindowTime = time.Millisecond * 10000

	// DefaultWindowBuckets is the default number of buckets the window holds, 10.
	DefaultWindowBuckets = 10
)

Functions

This section is empty.

Types

type Breaker 

type Breaker struct {
	// BackOff is the backoff policy that is used when determining if the breaker should
	// attempt to retry. A breaker created with NewBreaker will use an exponential backoff
	// policy by default.
	BackOff backoff.BackOff

	// ShouldTrip is a TripFunc that determines whether a Fail() call should trip the breaker.
	// A breaker created with NewBreaker will not have a ShouldTrip by default, and thus will
	// never automatically trip.
	ShouldTrip TripFunc

	// Clock is used for controlling time in tests.
	Clock clock.Clock
	// contains filtered or unexported fields
}

Breaker is the base of a circuit breaker. It maintains failure and success counters as well as the event subscribers.

Example (Events) 
// This example demonstrates the BreakerTripped and BreakerReset callbacks. These are
// available on all breaker types.
breaker := NewThresholdBreaker(1)
events := breaker.Subscribe()

go func() {
	for {
		e := <-events
		switch e {
		case BreakerTripped:
			log.Println("breaker tripped")
		case BreakerReset:
			log.Println("breaker reset")
		case BreakerFail:
			log.Println("breaker fail")
		case BreakerReady:
			log.Println("breaker ready")
		}
	}
}()

breaker.Fail()
breaker.Reset()

Output:

func NewBreaker 

func NewBreaker() *Breaker

NewBreaker creates a base breaker with an exponential backoff and no TripFunc

func NewBreakerWithOptions 

func NewBreakerWithOptions(options *Options) *Breaker

NewBreakerWithOptions creates a base breaker with a specified backoff, clock and TripFunc

func NewConsecutiveBreaker 

func NewConsecutiveBreaker(threshold int64) *Breaker

NewConsecutiveBreaker creates a Breaker with a ConsecutiveTripFunc.

Example 
// This example sets up a FrequencyBreaker that will trip if remoteCall returns
// an error 10 times in a row within a period of 2 minutes.
breaker := NewConsecutiveBreaker(10)
err := breaker.Call(remoteCall, 0)
if err != nil {
	log.Fatal(err)
}

Output:

func NewRateBreaker 

func NewRateBreaker(rate float64, minSamples int64) *Breaker

NewRateBreaker creates a Breaker with a RateTripFunc.

func NewThresholdBreaker 

func NewThresholdBreaker(threshold int64) *Breaker

NewThresholdBreaker creates a Breaker with a ThresholdTripFunc.

Example 
// This example sets up a ThresholdBreaker that will trip if remoteCall returns
// an error 10 times in a row. The error returned by Call() will be the error
// returned by remoteCall, unless the breaker has been tripped, in which case
// it will return ErrBreakerOpen.
breaker := NewThresholdBreaker(10)
err := breaker.Call(remoteCall, 0)
if err != nil {
	log.Fatal(err)
}

Output:
Example (Manual) 
// This example demonstrates the manual use of a ThresholdBreaker. The breaker
// will trip when Fail is called 10 times in a row.
breaker := NewThresholdBreaker(10)
if breaker.Ready() {
	err := remoteCall()
	if err != nil {
		breaker.Fail()
		log.Fatal(err)
	} else {
		breaker.Success()
	}
}

Output:
Example (Timeout) 
// This example sets up a ThresholdBreaker that will trip if remoteCall
// returns an error OR takes longer than one second 10 times in a row. The
// error returned by Call() will be the error returned by remoteCall with
// two exceptions: if remoteCall takes longer than one second the return
// value will be ErrBreakerTimeout, if the breaker has been tripped the
// return value will be ErrBreakerOpen.
breaker := NewThresholdBreaker(10)
err := breaker.Call(remoteCall, time.Second)
if err != nil {
	log.Fatal(err)
}

Output:

func (*Breaker) AddListener 

func (cb *Breaker) AddListener(listener chan ListenerEvent)

AddListener adds a channel of ListenerEvents on behalf of a listener. The listener channel must be buffered.

func (*Breaker) Break 

func (cb *Breaker) Break()

Break trips the circuit breaker and prevents it from auto resetting. Use this when manual control over the circuit breaker state is needed.

func (*Breaker) Call 

func (cb *Breaker) Call(circuit func() error, timeout time.Duration) error

Call wraps a function the Breaker will protect. A failure is recorded whenever the function returns an error. If the called function takes longer than timeout to run, a failure will be recorded.

func (*Breaker) CallContext 

func (cb *Breaker) CallContext(ctx context.Context, circuit func() error, timeout time.Duration) error

CallContext is same as Call but if the ctx is canceled after the circuit returned an error, the error will not be marked as a failure because the call was canceled intentionally.

func (*Breaker) ConsecFailures 

func (cb *Breaker) ConsecFailures() int64

ConsecFailures returns the number of consecutive failures that have occured.

func (*Breaker) ErrorRate 

func (cb *Breaker) ErrorRate() float64

ErrorRate returns the current error rate of the Breaker, expressed as a floating point number (e.g. 0.9 for 90%), since the last time the breaker was Reset.

func (*Breaker) Fail 

func (cb *Breaker) Fail()

Fail is used to indicate a failure condition the Breaker should record. It will increment the failure counters and store the time of the last failure. If the breaker has a TripFunc it will be called, tripping the breaker if necessary.

func (*Breaker) Failures 

func (cb *Breaker) Failures() int64

Failures returns the number of failures for this circuit breaker.

func (*Breaker) Ready 

func (cb *Breaker) Ready() bool

Ready will return true if the circuit breaker is ready to call the function. It will be ready if the breaker is in a reset state, or if it is time to retry the call for auto resetting.

func (*Breaker) RemoveListener 

func (cb *Breaker) RemoveListener(listener chan ListenerEvent) bool

RemoveListener removes a channel previously added via AddListener. Once removed, the channel will no longer receive ListenerEvents. Returns true if the listener was found and removed.

func (*Breaker) Reset 

func (cb *Breaker) Reset()

Reset will reset the circuit breaker. After Reset() is called, Tripped() will return false.

func (*Breaker) ResetCounters 

func (cb *Breaker) ResetCounters()

ResetCounters will reset only the failures, consecFailures, and success counters

func (*Breaker) Stop 

func (cb *Breaker) Stop()

Stop stops all go-routines to process events

func (*Breaker) Subscribe 

func (cb *Breaker) Subscribe() <-chan BreakerEvent

Subscribe returns a channel of BreakerEvents. Whenever the breaker changes state, the state will be sent over the channel. See BreakerEvent for the types of events.

func (*Breaker) Success 

func (cb *Breaker) Success()

Success is used to indicate a success condition the Breaker should record. If the success was triggered by a retry attempt, the breaker will be Reset().

func (*Breaker) Successes 

func (cb *Breaker) Successes() int64

Successes returns the number of successes for this circuit breaker.

func (*Breaker) Trip 

func (cb *Breaker) Trip()

Trip will trip the circuit breaker. After Trip() is called, Tripped() will return true.

func (*Breaker) Tripped 

func (cb *Breaker) Tripped() bool

Tripped returns true if the circuit breaker is tripped, false if it is reset.

type BreakerEvent 

type BreakerEvent int

BreakerEvent indicates the type of event received over an event channel 

const (
	// BreakerTripped is sent when a breaker trips
	BreakerTripped BreakerEvent = iota

	// BreakerReset is sent when a breaker resets
	BreakerReset

	// BreakerFail is sent when Fail() is called
	BreakerFail

	// BreakerReady is sent when the breaker enters the half open state and is ready to retry
	BreakerReady

	// stops breaker's subscribers
	BreakerStop
)

type HTTPClient 

type HTTPClient struct {
	Client         *http.Client
	BreakerTripped func()
	BreakerReset   func()
	BreakerLookup  func(*HTTPClient, interface{}) *Breaker
	Panel          *Panel
	// contains filtered or unexported fields
}

HTTPClient is a wrapper around http.Client that provides circuit breaker capabilities.

By default, the client will use its defaultBreaker. A BreakerLookup function may be provided to allow different breakers to be used based on the circumstance. See the implementation of NewHostBasedHTTPClient for an example of this.

Example 
// This example sets up an HTTP client wrapped in a ThresholdBreaker. The
// breaker will trip with the same behavior as ThresholdBreaker.
client := NewHTTPClient(time.Second*5, 10, nil)

resp, err := client.Get("http://example.com/resource.json")
if err != nil {
	log.Fatal(err)
}
resource, err := ioutil.ReadAll(resp.Body)
resp.Body.Close()
if err != nil {
	log.Fatal(err)
}
fmt.Printf("%s", resource)

Output:

func NewHTTPClient 

func NewHTTPClient(timeout time.Duration, threshold int64, client *http.Client) *HTTPClient

NewHTTPClient provides a circuit breaker wrapper around http.Client. It wraps all of the regular http.Client functions. Specifying 0 for timeout will give a breaker that does not check for time outs.

func NewHTTPClientWithBreaker 

func NewHTTPClientWithBreaker(breaker *Breaker, timeout time.Duration, client *http.Client) *HTTPClient

NewHTTPClientWithBreaker provides a circuit breaker wrapper around http.Client. It wraps all of the regular http.Client functions using the provided Breaker.

func NewHostBasedHTTPClient 

func NewHostBasedHTTPClient(timeout time.Duration, threshold int64, client *http.Client) *HTTPClient

NewHostBasedHTTPClient provides a circuit breaker wrapper around http.Client. This client will use one circuit breaker per host parsed from the request URL. This allows you to use a single HTTPClient for multiple hosts with one host's breaker not affecting the other hosts.

func (*HTTPClient) Do 

func (c *HTTPClient) Do(req *http.Request) (*http.Response, error)

Do wraps http.Client Do()

func (*HTTPClient) Get 

func (c *HTTPClient) Get(url string) (*http.Response, error)

Get wraps http.Client Get()

func (*HTTPClient) Head 

func (c *HTTPClient) Head(url string) (*http.Response, error)

Head wraps http.Client Head()

func (*HTTPClient) Post 

func (c *HTTPClient) Post(url string, bodyType string, body io.Reader) (*http.Response, error)

Post wraps http.Client Post()

func (*HTTPClient) PostForm 

func (c *HTTPClient) PostForm(url string, data url.Values) (*http.Response, error)

PostForm wraps http.Client PostForm()

type ListenerEvent 

type ListenerEvent struct {
	CB    *Breaker
	Event BreakerEvent
}

ListenerEvent includes a reference to the circuit breaker and the event.

type Options 

type Options struct {
	BackOff       backoff.BackOff
	Clock         clock.Clock
	ShouldTrip    TripFunc
	WindowTime    time.Duration
	WindowBuckets int
}

Options holds breaker configuration options.

type Panel 

type Panel struct {
	Statter      Statter
	StatsPrefixf string

	Circuits map[string]*Breaker
	// contains filtered or unexported fields
}

Panel tracks a group of circuit breakers by name.

Example 
// This example demonstrates using a Panel to aggregate and manage circuit breakers.
breaker1 := NewThresholdBreaker(10)
breaker2 := NewRateBreaker(0.95, 100)

panel := NewPanel()
panel.Add("breaker1", breaker1)
panel.Add("breaker2", breaker2)

// Elsewhere in the code ...
b1, _ := panel.Get("breaker1")
b1.Call(func() error {
	// Do some work
	return nil
}, 0)

b2, _ := panel.Get("breaker2")
b2.Call(func() error {
	// Do some work
	return nil
}, 0)

Output:
Example (Stats) 
// This example demonstrates how to push circuit breaker stats to statsd via a Panel.
// This example uses g2s. Anything conforming to the Statter interface can be used.
s, err := g2s.Dial("udp", "statsd-server:8125")
if err != nil {
	log.Fatal(err)
}

breaker := NewThresholdBreaker(10)
panel := NewPanel()
panel.Statter = s
panel.StatsPrefixf = "sys.production.%s"
panel.Add("x", breaker)

breaker.Trip()  // sys.production.circuit.x.tripped
breaker.Reset() // sys.production.circuit.x.reset, sys.production.circuit.x.trip-time
breaker.Fail()  // sys.production.circuit.x.fail
breaker.Ready() // sys.production.circuit.x.ready (if it's tripped and ready to retry)

Output:

func NewPanel 

func NewPanel() *Panel

NewPanel creates a new Panel

func (*Panel) Add 

func (p *Panel) Add(name string, cb *Breaker)

Add sets the name as a reference to the given circuit breaker.

func (*Panel) Get 

func (p *Panel) Get(name string) (*Breaker, bool)

Get retrieves a circuit breaker by name. If no circuit breaker exists, it returns the NoOp one and sets ok to false.

func (*Panel) Subscribe 

func (p *Panel) Subscribe() <-chan PanelEvent

Subscribe returns a channel of PanelEvents. Whenever a breaker changes state, the PanelEvent will be sent over the channel. See BreakerEvent for the types of events.

type PanelEvent 

type PanelEvent struct {
	Name  string
	Event BreakerEvent
}

PanelEvent wraps a BreakerEvent and provides the string name of the breaker

type Statter 

type Statter interface {
	Counter(sampleRate float32, bucket string, n ...int)
	Timing(sampleRate float32, bucket string, d ...time.Duration)
	Gauge(sampleRate float32, bucket string, value ...string)
}

Statter interface provides a way to gather statistics from breakers

type TripFunc 

type TripFunc func(*Breaker) bool

TripFunc is a function called by a Breaker's Fail() function and determines whether the breaker should trip. It will receive the Breaker as an argument and returns a boolean. By default, a Breaker has no TripFunc.

func ConsecutiveTripFunc 

func ConsecutiveTripFunc(threshold int64) TripFunc

ConsecutiveTripFunc returns a TripFunc that trips whenever the consecutive failure count meets the threshold.

func RateTripFunc 

func RateTripFunc(rate float64, minSamples int64) TripFunc

RateTripFunc returns a TripFunc that trips whenever the error rate hits the threshold. The error rate is calculated as such: f = number of failures s = number of successes e = f / (f + s) The error rate is calculated over a sliding window of 10 seconds (by default) This TripFunc will not trip until there have been at least minSamples events.

func ThresholdTripFunc 

func ThresholdTripFunc(threshold int64) TripFunc

ThresholdTripFunc returns a TripFunc with that trips whenever the failure count meets the threshold.

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.