README

circuitbreaker

Circuitbreaker provides an easy way to use the Circuit Breaker pattern in a Go program.

Circuit breakers are typically used when your program makes remote calls. Remote calls can often hang for a while before they time out. If your application makes a lot of these requests, many resources can be tied up waiting for these time outs to occur. A circuit breaker wraps these remote calls and will trip after a defined amount of failures or time outs occur. When a circuit breaker is tripped any future calls will avoid making the remote call and return an error to the caller. In the meantime, the circuit breaker will periodically allow some calls to be tried again and will close the circuit if those are successful.

You can read more about this pattern and how it's used at:

GoDoc

Installation

  go get github.com/rubyist/circuitbreaker

Examples

Here is a quick example of what circuitbreaker provides

// Creates a circuit breaker that will trip if the function fails 10 times
cb := circuit.NewThresholdBreaker(10)

events := cb.Subscribe()
go func() {
  for {
    e := <-events
    // Monitor breaker events like BreakerTripped, BreakerReset, BreakerFail, BreakerReady
  }
}()

cb.Call(func() error {
	// This is where you'll do some remote call
	// If it fails, return an error
}, 0)

Circuitbreaker can also wrap a time out around the remote call.

// Creates a circuit breaker that will trip after 10 failures
// using a time out of 5 seconds
cb := circuit.NewThresholdBreaker(10)

cb.Call(func() error {
  // This is where you'll do some remote call
  // If it fails, return an error
}, time.Second * 5) // This will time out after 5 seconds, which counts as a failure

// Proceed as above

Circuitbreaker can also trip based on the number of consecutive failures.

// Creates a circuit breaker that will trip if 10 consecutive failures occur
cb := circuit.NewConsecutiveBreaker(10)

// Proceed as above

Circuitbreaker can trip based on the error rate.

// Creates a circuit breaker based on the error rate
cb := circuit.NewRateBreaker(0.95, 100) // trip when error rate hits 95%, with at least 100 samples

// Proceed as above

If it doesn't make sense to wrap logic in Call(), breakers can be handled manually.

cb := circuit.NewThresholdBreaker(10)

for {
  if cb.Ready() {
    // Breaker is not tripped, proceed
    err := doSomething()
    if err != nil {
      cb.Fail() // This will trip the breaker once it's failed 10 times
      continue
    }
    cb.Success()
  } else {
    // Breaker is in a tripped state.
  }
}

Circuitbreaker also provides a wrapper around http.Client that will wrap a time out around any request.

// Passing in nil will create a regular http.Client.
// You can also build your own http.Client and pass it in
client := circuit.NewHTTPClient(time.Second * 5, 10, nil)

resp, err := client.Get("http://example.com/resource.json")

See the godoc for more examples.

Bugs, Issues, Feedback

Right here on GitHub: https://github.com/rubyist/circuitbreaker

Expand ▾ Collapse ▴

Documentation

Overview

    Package circuit implements the Circuit Breaker pattern. It will wrap a function call (typically one which uses remote services) and monitors for failures and/or time outs. When a threshold of failures or time outs has been reached, future calls to the function will not run. During this state, the breaker will periodically allow the function to run and, if it is successful, will start running the function again.

    Circuit includes three types of circuit breakers:

    A Threshold Breaker will trip when the failure count reaches a given threshold. It does not matter how long it takes to reach the threshold and the failures do not need to be consecutive.

    A Consecutive Breaker will trip when the consecutive failure count reaches a given threshold. It does not matter how long it takes to reach the threshold, but the failures do need to be consecutive.

    When wrapping blocks of code with a Breaker's Call() function, a time out can be specified. If the time out is reached, the breaker's Fail() function will be called.

    Other types of circuit breakers can be easily built by creating a Breaker and adding a custom TripFunc. A TripFunc is called when a Breaker Fail()s and receives the breaker as an argument. It then returns true or false to indicate whether the breaker should trip.

    The package also provides a wrapper around an http.Client that wraps all of the http.Client functions with a Breaker.

    Index

    Examples

    Constants

    This section is empty.

    Variables

    View Source
    var (
    	ErrBreakerOpen    = errors.New("breaker open")
    	ErrBreakerTimeout = errors.New("breaker time out")
    )

      Error codes returned by Call

      View Source
      var (
      	// DefaultWindowTime is the default time the window covers, 10 seconds.
      	DefaultWindowTime = time.Millisecond * 10000
      
      	// DefaultWindowBuckets is the default number of buckets the window holds, 10.
      	DefaultWindowBuckets = 10
      )

      Functions

      This section is empty.

      Types

      type Breaker

      type Breaker struct {
      	// BackOff is the backoff policy that is used when determining if the breaker should
      	// attempt to retry. A breaker created with NewBreaker will use an exponential backoff
      	// policy by default.
      	BackOff backoff.BackOff
      
      	// ShouldTrip is a TripFunc that determines whether a Fail() call should trip the breaker.
      	// A breaker created with NewBreaker will not have a ShouldTrip by default, and thus will
      	// never automatically trip.
      	ShouldTrip TripFunc
      
      	// Clock is used for controlling time in tests.
      	Clock clock.Clock
      	// contains filtered or unexported fields
      }

        Breaker is the base of a circuit breaker. It maintains failure and success counters as well as the event subscribers.

        Example (Events)
        Output:
        
        

        func NewBreaker

        func NewBreaker() *Breaker

          NewBreaker creates a base breaker with an exponential backoff and no TripFunc

          func NewBreakerWithOptions

          func NewBreakerWithOptions(options *Options) *Breaker

            NewBreakerWithOptions creates a base breaker with a specified backoff, clock and TripFunc

            func NewConsecutiveBreaker

            func NewConsecutiveBreaker(threshold int64) *Breaker

              NewConsecutiveBreaker creates a Breaker with a ConsecutiveTripFunc.

              Example
              Output:
              
              

              func NewRateBreaker

              func NewRateBreaker(rate float64, minSamples int64) *Breaker

                NewRateBreaker creates a Breaker with a RateTripFunc.

                func NewThresholdBreaker

                func NewThresholdBreaker(threshold int64) *Breaker

                  NewThresholdBreaker creates a Breaker with a ThresholdTripFunc.

                  Example
                  Output:
                  
                  
                  Example (Manual)
                  Output:
                  
                  
                  Example (Timeout)
                  Output:
                  
                  

                  func (*Breaker) AddListener

                  func (cb *Breaker) AddListener(listener chan ListenerEvent)

                    AddListener adds a channel of ListenerEvents on behalf of a listener. The listener channel must be buffered.

                    func (*Breaker) Break

                    func (cb *Breaker) Break()

                      Break trips the circuit breaker and prevents it from auto resetting. Use this when manual control over the circuit breaker state is needed.

                      func (*Breaker) Call

                      func (cb *Breaker) Call(circuit func() error, timeout time.Duration) error

                        Call wraps a function the Breaker will protect. A failure is recorded whenever the function returns an error. If the called function takes longer than timeout to run, a failure will be recorded.

                        func (*Breaker) CallContext

                        func (cb *Breaker) CallContext(ctx context.Context, circuit func() error, timeout time.Duration) error

                          CallContext is same as Call but if the ctx is canceled after the circuit returned an error, the error will not be marked as a failure because the call was canceled intentionally.

                          func (*Breaker) ConsecFailures

                          func (cb *Breaker) ConsecFailures() int64

                            ConsecFailures returns the number of consecutive failures that have occured.

                            func (*Breaker) ErrorRate

                            func (cb *Breaker) ErrorRate() float64

                              ErrorRate returns the current error rate of the Breaker, expressed as a floating point number (e.g. 0.9 for 90%), since the last time the breaker was Reset.

                              func (*Breaker) Fail

                              func (cb *Breaker) Fail()

                                Fail is used to indicate a failure condition the Breaker should record. It will increment the failure counters and store the time of the last failure. If the breaker has a TripFunc it will be called, tripping the breaker if necessary.

                                func (*Breaker) Failures

                                func (cb *Breaker) Failures() int64

                                  Failures returns the number of failures for this circuit breaker.

                                  func (*Breaker) Ready

                                  func (cb *Breaker) Ready() bool

                                    Ready will return true if the circuit breaker is ready to call the function. It will be ready if the breaker is in a reset state, or if it is time to retry the call for auto resetting.

                                    func (*Breaker) RemoveListener

                                    func (cb *Breaker) RemoveListener(listener chan ListenerEvent) bool

                                      RemoveListener removes a channel previously added via AddListener. Once removed, the channel will no longer receive ListenerEvents. Returns true if the listener was found and removed.

                                      func (*Breaker) Reset

                                      func (cb *Breaker) Reset()

                                        Reset will reset the circuit breaker. After Reset() is called, Tripped() will return false.

                                        func (*Breaker) ResetCounters

                                        func (cb *Breaker) ResetCounters()

                                          ResetCounters will reset only the failures, consecFailures, and success counters

                                          func (*Breaker) Subscribe

                                          func (cb *Breaker) Subscribe() <-chan BreakerEvent

                                            Subscribe returns a channel of BreakerEvents. Whenever the breaker changes state, the state will be sent over the channel. See BreakerEvent for the types of events.

                                            func (*Breaker) Success

                                            func (cb *Breaker) Success()

                                              Success is used to indicate a success condition the Breaker should record. If the success was triggered by a retry attempt, the breaker will be Reset().

                                              func (*Breaker) Successes

                                              func (cb *Breaker) Successes() int64

                                                Successes returns the number of successes for this circuit breaker.

                                                func (*Breaker) Trip

                                                func (cb *Breaker) Trip()

                                                  Trip will trip the circuit breaker. After Trip() is called, Tripped() will return true.

                                                  func (*Breaker) Tripped

                                                  func (cb *Breaker) Tripped() bool

                                                    Tripped returns true if the circuit breaker is tripped, false if it is reset.

                                                    type BreakerEvent

                                                    type BreakerEvent int

                                                      BreakerEvent indicates the type of event received over an event channel

                                                      const (
                                                      	// BreakerTripped is sent when a breaker trips
                                                      	BreakerTripped BreakerEvent = iota
                                                      
                                                      	// BreakerReset is sent when a breaker resets
                                                      	BreakerReset BreakerEvent = iota
                                                      
                                                      	// BreakerFail is sent when Fail() is called
                                                      	BreakerFail BreakerEvent = iota
                                                      
                                                      	// BreakerReady is sent when the breaker enters the half open state and is ready to retry
                                                      	BreakerReady BreakerEvent = iota
                                                      )

                                                      type HTTPClient

                                                      type HTTPClient struct {
                                                      	Client         *http.Client
                                                      	BreakerTripped func()
                                                      	BreakerReset   func()
                                                      	BreakerLookup  func(*HTTPClient, interface{}) *Breaker
                                                      	Panel          *Panel
                                                      	// contains filtered or unexported fields
                                                      }

                                                        HTTPClient is a wrapper around http.Client that provides circuit breaker capabilities.

                                                        By default, the client will use its defaultBreaker. A BreakerLookup function may be provided to allow different breakers to be used based on the circumstance. See the implementation of NewHostBasedHTTPClient for an example of this.

                                                        Example
                                                        Output:
                                                        
                                                        

                                                        func NewHTTPClient

                                                        func NewHTTPClient(timeout time.Duration, threshold int64, client *http.Client) *HTTPClient

                                                          NewHTTPClient provides a circuit breaker wrapper around http.Client. It wraps all of the regular http.Client functions. Specifying 0 for timeout will give a breaker that does not check for time outs.

                                                          func NewHTTPClientWithBreaker

                                                          func NewHTTPClientWithBreaker(breaker *Breaker, timeout time.Duration, client *http.Client) *HTTPClient

                                                            NewHTTPClientWithBreaker provides a circuit breaker wrapper around http.Client. It wraps all of the regular http.Client functions using the provided Breaker.

                                                            func NewHostBasedHTTPClient

                                                            func NewHostBasedHTTPClient(timeout time.Duration, threshold int64, client *http.Client) *HTTPClient

                                                              NewHostBasedHTTPClient provides a circuit breaker wrapper around http.Client. This client will use one circuit breaker per host parsed from the request URL. This allows you to use a single HTTPClient for multiple hosts with one host's breaker not affecting the other hosts.

                                                              func (*HTTPClient) Do

                                                              func (c *HTTPClient) Do(req *http.Request) (*http.Response, error)

                                                                Do wraps http.Client Do()

                                                                func (*HTTPClient) Get

                                                                func (c *HTTPClient) Get(url string) (*http.Response, error)

                                                                  Get wraps http.Client Get()

                                                                  func (*HTTPClient) Head

                                                                  func (c *HTTPClient) Head(url string) (*http.Response, error)

                                                                    Head wraps http.Client Head()

                                                                    func (*HTTPClient) Post

                                                                    func (c *HTTPClient) Post(url string, bodyType string, body io.Reader) (*http.Response, error)

                                                                      Post wraps http.Client Post()

                                                                      func (*HTTPClient) PostForm

                                                                      func (c *HTTPClient) PostForm(url string, data url.Values) (*http.Response, error)

                                                                        PostForm wraps http.Client PostForm()

                                                                        type ListenerEvent

                                                                        type ListenerEvent struct {
                                                                        	CB    *Breaker
                                                                        	Event BreakerEvent
                                                                        }

                                                                          ListenerEvent includes a reference to the circuit breaker and the event.

                                                                          type Options

                                                                          type Options struct {
                                                                          	BackOff       backoff.BackOff
                                                                          	Clock         clock.Clock
                                                                          	ShouldTrip    TripFunc
                                                                          	WindowTime    time.Duration
                                                                          	WindowBuckets int
                                                                          }

                                                                            Options holds breaker configuration options.

                                                                            type Panel

                                                                            type Panel struct {
                                                                            	Statter      Statter
                                                                            	StatsPrefixf string
                                                                            
                                                                            	Circuits map[string]*Breaker
                                                                            	// contains filtered or unexported fields
                                                                            }

                                                                              Panel tracks a group of circuit breakers by name.

                                                                              Example
                                                                              Output:
                                                                              
                                                                              
                                                                              Example (Stats)
                                                                              Output:
                                                                              
                                                                              

                                                                              func NewPanel

                                                                              func NewPanel() *Panel

                                                                                NewPanel creates a new Panel

                                                                                func (*Panel) Add

                                                                                func (p *Panel) Add(name string, cb *Breaker)

                                                                                  Add sets the name as a reference to the given circuit breaker.

                                                                                  func (*Panel) Get

                                                                                  func (p *Panel) Get(name string) (*Breaker, bool)

                                                                                    Get retrieves a circuit breaker by name. If no circuit breaker exists, it returns the NoOp one and sets ok to false.

                                                                                    func (*Panel) Subscribe

                                                                                    func (p *Panel) Subscribe() <-chan PanelEvent

                                                                                      Subscribe returns a channel of PanelEvents. Whenever a breaker changes state, the PanelEvent will be sent over the channel. See BreakerEvent for the types of events.

                                                                                      type PanelEvent

                                                                                      type PanelEvent struct {
                                                                                      	Name  string
                                                                                      	Event BreakerEvent
                                                                                      }

                                                                                        PanelEvent wraps a BreakerEvent and provides the string name of the breaker

                                                                                        type Statter

                                                                                        type Statter interface {
                                                                                        	Counter(sampleRate float32, bucket string, n ...int)
                                                                                        	Timing(sampleRate float32, bucket string, d ...time.Duration)
                                                                                        	Gauge(sampleRate float32, bucket string, value ...string)
                                                                                        }

                                                                                          Statter interface provides a way to gather statistics from breakers

                                                                                          type TripFunc

                                                                                          type TripFunc func(*Breaker) bool

                                                                                            TripFunc is a function called by a Breaker's Fail() function and determines whether the breaker should trip. It will receive the Breaker as an argument and returns a boolean. By default, a Breaker has no TripFunc.

                                                                                            func ConsecutiveTripFunc

                                                                                            func ConsecutiveTripFunc(threshold int64) TripFunc

                                                                                              ConsecutiveTripFunc returns a TripFunc that trips whenever the consecutive failure count meets the threshold.

                                                                                              func RateTripFunc

                                                                                              func RateTripFunc(rate float64, minSamples int64) TripFunc

                                                                                                RateTripFunc returns a TripFunc that trips whenever the error rate hits the threshold. The error rate is calculated as such: f = number of failures s = number of successes e = f / (f + s) The error rate is calculated over a sliding window of 10 seconds (by default) This TripFunc will not trip until there have been at least minSamples events.

                                                                                                func ThresholdTripFunc

                                                                                                func ThresholdTripFunc(threshold int64) TripFunc

                                                                                                  ThresholdTripFunc returns a TripFunc with that trips whenever the failure count meets the threshold.