shift

package module
Version: v1.0.0 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Oct 31, 2020 License: Apache-2.0 Imports: 6 Imported by: 0

README

🎚Shift

GoDoc Build Status Coverage Status Go Report Card GitHub license

Shift package is an optioned circuit breaker implementation.

For those who are new to the concept, a brief summary:

  • circuit breaker has 3 states: close, half-open and open
  • when it is in open state, something bad is going on the executions and to prevent bad invokations, the circuit breaker gave a break to new invokations, and returns error
  • when it is in half-open state, then there could be a chance for recovery from bad state and circuit breaker evaluates criterias to trip to next states
  • when it is in close state, then everything is working as expected

State changes in circuit breaker:

  • close -> open: close can trip to open state
  • close <- half-open -> open: half-open can trip to both close and open states
  • open -> half-open: open can trip to half-open

If you are interested in learning what is circuit breaker or sample use cases for circuit breakers then you can refer to References section of this README.md file.

API

Shift package follows semantic versioning 2.x rules on releases and tags. To access to the current package version, shift.Version constant can be used.

Features

  • Configurable with optioned plugable components
  • Comes with built-in execution timeout feature which cancels the execution by optioned timeout duration
  • Comes with built-in bucketted counter feature which counts the stats by given durationed buckets
  • Allows subscribing state change, failure and success events
  • Allows overriding the current state with callbacks
  • Allows overriding reset timer which can be implemented using an exponential backoff algorithm or any other algorithm when needed
  • Allows overriding counter which can allow using an external counter for managing the stats
  • Allows adding optional restrictors by execution like max concurrent runs

Installation

Via go packages:

go get github.com/mustafafuran/shift

Usage

Basic with defaults

On configurations 3 options are critical to have a healthy circuit breaker, so on any configuration it is highly recommended to specify at least the following 3 options with desired numbers.

// Trip from Close to Open state under 95.0% success ratio at minimum of
// 20 invokations on configured duration(see WithCounter for durationed stats)
shift.WithOpener(StateClose, 95.0, 20),

// Trip from Half-Open to Open state under 75.0% success ratio at minimum of
// 10 invokations on configured duration(see WithCounter for durationed stats)
shift.WithOpener(StateHalfOpen, 75.0, 10),

// Trip from Half-Open to Close state on 90.0% of success ratio at minumum of
// 10 invokations on configured duration(see WithCounter for durationed stats)
shift.WithCloser(90.0, 10),

It is also recommended to have multiple circuit breakers for each invoker with their own specific configurations depending on the SLA requirements. For example: A circuit breaker for Github, another for Twitter, and yet another for Facebook API client.

Execute with a function implementation
import (
	"context"
	"fmt"

	"github.com/mustafafuran/shift"
)

func NewCircuitBreaker() *shift.Shift {
	cb, err := shift.New(
		"a-name-for-the-breaker",

		// Trip from Close to Open state under 95.0% success ratio at minimum of
		// 20 invokations on configured duration(see WithCounter for durationed
		// stats)
		shift.WithOpener(StateClose, 95.0, 20),

		// Trip from Half-Open to Open state under 75.0% success ratio at
		// minimum of 10 invokations on configured duration(see WithCounter for
		// durationed stats)
		shift.WithOpener(StateHalfOpen, 75.0, 10),

		// Trip from Half-Open to Close state on 90.0% of success ratio at
		// minumum of 10 invokations on configured duration(see WithCounter for
		// durationed stats)
		shift.WithCloser(90.0, 10),
	)
	if err != nil {
		panic(err)
	}
	return cb
}

func DoSomethingWithFn(ctx context.Context, cb *shift.CircuitBreaker) string {
	var fn shift.Operate = func(ctx context.Context) (interface{}, error) {
		// do something in here
		return "foo", nil
	}
	res, err := cb.Run(ctx, fn)
	if err != nil {
		// maybe read from cache to set the res again?
	}

	// convert your res into your actual data
	data := res.(string)
	return data
}

func main() {
	cb := NewCircuitBreaker()
	data := DoSomethingWithFn(ctx, cb)
	fmt.Printf("data: %s\n", data)
}
Configure for max concurrent runnables

Shift allows adding restrictors like max concurrent runnables to prevent execution of the invokes on developer defined conditions. Restrictors do not effect the current state, but they can block the execution depending on their own internal state values. If a restrictor blocks an execution then it returns an error and On Failure Handlers get executed in order.

import (
	"github.com/mustafafuran/shift"
	"github.com/mustafafuran/shift/restrictor"
)

func NewCircuitBreaker() *shift.CircuitBreaker {
	restrictor, err := restrictor.NewConcurrentRunRestrictor("concurrent_runs", 100)
	if err != nil {
		return err
	}

	cb, err := shift.New(
		"twitter-cli",

		// Restrictors
		shift.WithRestrictors(restrictor),

		// Trippers
		shift.WithOpener(StateClose, 95.0, 20),
		shift.WithOpener(StateHalfOpen, 75.0, 10),
		shift.WithCloser(90.0, 10),

		// ... other options
	)
	if err != nil {
		panic(err)
	}
	return cb
}
Creating a reset timer based on errors

Any reset timer strategy can be implemented on top of shift.Timer interface. The default timer strategy does not intentionally implement any use case specific strategy like exponential backoff. Since the decision of reset time incrementation should be taken depending on error reasons, the best decider for each instance of CircuitBreaker would be the developers. In case, if it is good to just have a constant timeout duration, the shift/timer.ConstantTimer implementation should simply help to configure your reset timeout duration.


import (
	"github.com/mustafafuran/shift"
	"github.com/mustafafuran/shift/timer"
)

func NewCircuitBreaker() *shift.CircuitBreaker {
	timer, err := timer.NewConstantTimer(5 * time.Second)
	if err != nil {
		panic(err)
	}

	cb, err := shift.New(
		"twitter-cli",
		// Reset Timer
		shift.WithResetTimer(timer),

		// Trippers
		shift.WithOpener(StateClose, 95.0, 20),
		shift.WithOpener(StateHalfOpen, 75.0, 10),
		shift.WithCloser(90.0, 10),

		// ... other options
	)
	if err != nil {
		panic(err)
	}
	return cb
}
Creating a counter based on your bucketing needs

Any counter strategy can be implemented on top of shift.Counter interface. The default counter strategy is using a bucketing mechanism to bucket time and add/drop metrics into the stats. The default Counter uses 1 second durationed 10 buckets. There are two possible options to modify the Counter based on your needs:

  1. Create a new counter instance and pass as counter option

  2. Create your own counter implementations and pass the instance as counter option


import (
	"github.com/mustafafuran/shift"
	"github.com/mustafafuran/shift/counter"
)

func NewCircuitBreaker() *shift.CircuitBreaker {
	// The TimeBucketCounter automatically drops a the oldest bucket after
	// filling the available last bucket and then shift left the buckets, so a
	// new space is freeing up for a new bucket

	// 60 buckets each holds the stats for 2 seconds
	capacity, duration := 60, 2000 * time.Millisecond
	counter, err := counter.TimeBucketCounter(capacity, duration)
	if err != nil {
		panic(err)
	}

	cb, err := shift.New(
		"twitter-cli",
		// Counter
		shift.WithCounter(counter),

		// Trippers
		shift.WithOpener(StateClose, 95.0, 20),
		shift.WithOpener(StateHalfOpen, 75.0, 10),
		shift.WithCloser(90.0, 10),

		// ... other options
	)
	if err != nil {
		panic(err)
	}
	return cb
}
Events

Shift package allows adding multiple hooks on failure, success and state change circuit breaker events. Both success and failure events come with a context which holds state and stats;

  • State Change Event: Allows attaching handlers on the circuit breaker state changes
  • Failure Event: Allows attaching handlers on the circuit breaker execution results with an error
  • Success Event: Allows attaching handlers on the circuit breaker execution results without an error
Configure with On State Change Handlers
// a printer handler
var printer shift.OnStateChange = func(from, to shift.State, stats shift.Stats) {
	fmt.Printf("State changed from %s, to %s, %+v", from, to, stats)
}

// another handler
var another shift.OnStateChange = func(from, to shift.State, stats shift.Stats) {
	// do sth
}

cb, err := shift.New(
	"a-name",
	shift.WithStateChangeHandlers(printer, another),
	// ... other options
)
Configure with On Failure Handlers
// a printer handler
var printer shift.OnFailure = func(ctx context.Context, err error) {
	state := ctx.Value(CtxState).(State)
	stats := ctx.Value(CtxStats).(Stats)

	fmt.Printf("execution erred on state(%s) with %s and stats are %+v", state, err, stats)
}

// another handler
var another shift.OnFailure = func(ctx context.Context, err error) {
	// do sth: maybe increment an external metric when the execution err
}

// yetAnother handler
var yetAnother shift.OnFailure = func(ctx context.Context, err error) {
	// do sth
}

cb, err := shift.New(
	"a-name",
	// appends the failure handlers provided
	shift.WithFailureHandlers(StateClose, printer, another, yetAnother),
	shift.WithFailureHandlers(StateHalfOpen, printer, another),

	// Trippers
	shift.WithOpener(StateClose, 95.0, 20),
	shift.WithOpener(StateHalfOpen, 75.0, 10),
	shift.WithCloser(90.0, 10),

	// ... other options
)
Configure with On Success Handlers
// a printer handler
var printer shift.OnSuccess = func(ctx context.Context, data interface{}) {
	state := ctx.Value(CtxState).(State)
	stats := ctx.Value(CtxStats).(Stats)

	fmt.Printf("execution succeeded on %s and resulted with %+v and stats are %+v", state, data, stats)
}

// another handler
var another shift.OnSuccess = func(ctx context.Context, data interface{}) {
	// do sth: maybe increment an external metric when the execution succeeds
}

cb, err := shift.New(
	"a-name",

	// Appends the success handlers for a given state
	shift.WithSuccessHandlers(StateClose, printer, another),
	shift.WithSuccessHandlers(StateHalfOpen, printer),

	// Trippers
	shift.WithOpener(StateClose, 95.0, 20),
	shift.WithOpener(StateHalfOpen, 75.0, 10),
	shift.WithCloser(90.0, 10),

	// ... other options
)
Advanced configuration options

Please refer to GoDoc for more options and configurations.

Examples

Shift Examples Repository

Contributing

All contributors should follow Contributing Guidelines before creating pull requests.

New features

All features SHOULD be optional and SHOULD NOT change the API contracts. Please refer to API section of the README.md for more information.

Unit tests

Test coverage is very important for this kind of important piece of infrastructure softwares. Any change MUST cover all use cases with race condition checks.

To run unit tests locally, you can use Makefile short cuts:

make test_race # test against race conditions
make test # test and write the coverage results to `./coverage.out` file
make coverage # display the coverage in format
make all # run all three above in order

References

Credits

Mustafa Turan

License

Apache License 2.0

Copyright (c) 2020 Mustafa Turan

Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.

Documentation

Index

Constants

View Source 
const (
	// CtxState holds state context key
	CtxState = ctxKey("state")

	// CtxStats holds stats context key
	CtxStats = ctxKey("stats")
)
View Source 
const (
	// Version matches with the current version of the package
	Version = "1.0.0"
)

Variables

This section is empty.

Functions

This section is empty.

Types

type Counter added in v1.0.0 

type Counter interface {
	Increment(metric string)
	Stats(metrics ...string) map[string]uint32
	Reset()
}

Counter is an interface to increment, reset and fetch invocation stats

type FailureHandler added in v1.0.0 

type FailureHandler interface {
	Handle(context.Context, error)
}

FailureHandler is an interface to handle failure events

type FailureThresholdReachedError added in v1.0.0 

type FailureThresholdReachedError struct{}

FailureThresholdReachedError is a error type for failure threshold

func (*FailureThresholdReachedError) Error added in v1.0.0 

func (e *FailureThresholdReachedError) Error() string

type InvalidOptionError 

type InvalidOptionError struct {
	Name    string
	Message string
}

InvalidOptionError is a error tyoe for options

func (*InvalidOptionError) Error 

func (e *InvalidOptionError) Error() string

type InvocationError added in v1.0.0 

type InvocationError struct {
	Name string
	Err  error
}

InvocationError is an error type to wrap invocation errors

func (*InvocationError) Error added in v1.0.0 

func (e *InvocationError) Error() string

func (*InvocationError) Unwrap added in v1.0.0 

func (e *InvocationError) Unwrap() error

type InvocationTimeoutError added in v1.0.0 

type InvocationTimeoutError struct {
	Duration time.Duration
}

InvocationTimeoutError is a error type for invocation timeouts

func (*InvocationTimeoutError) Error added in v1.0.0 

func (e *InvocationTimeoutError) Error() string

type IsAlreadyInDesiredStateError added in v1.0.0 

type IsAlreadyInDesiredStateError struct {
	Name  string
	State State
}

IsAlreadyInDesiredStateError is an error type for stating the current state is already in desired state

func (*IsAlreadyInDesiredStateError) Error added in v1.0.0 

func (e *IsAlreadyInDesiredStateError) Error() string

type IsOnOpenStateError added in v1.0.0 

type IsOnOpenStateError struct{}

IsOnOpenStateError is a error type for open state

func (*IsOnOpenStateError) Error added in v1.0.0 

func (e *IsOnOpenStateError) Error() string

type OnFailure 

type OnFailure func(context.Context, error)

OnFailure is a function to run as a callback on any error like timeout and invocation errors

func (OnFailure) Handle 

func (fn OnFailure) Handle(ctx context.Context, err error)

Handle implements FailureHandler for OnFailure func

type OnStateChange 

type OnStateChange func(from, to State, stats Stats)

OnStateChange is a function to run on any state changes

func (OnStateChange) Handle 

func (fn OnStateChange) Handle(from, to State, stats Stats)

Handle implements StateChangeHandler for OnStateChange func

type OnSuccess 

type OnSuccess func(context.Context, interface{})

OnSuccess is a function to run on any successful invocation

func (OnSuccess) Handle 

func (fn OnSuccess) Handle(ctx context.Context, res interface{})

Handle implements SuccessHandler for OnSuccess func

type Operate 

type Operate func(context.Context) (interface{}, error)

Operate is a function that runs the operation

func (Operate) Execute 

func (o Operate) Execute(ctx context.Context) (interface{}, error)

Execute implements Operator interface for any Operate function for free

type Operator 

type Operator interface {
	Execute(context.Context) (interface{}, error)
}

Operator is an interface for circuit breaker operations

type Option 

type Option func(*Shift) error

Option is a type for circuit breaker options

func WithCloser added in v1.0.0 

func WithCloser(minSuccessRatio float32, minRequests uint32) Option

WithCloser builds an option to set the default success criteria trip to 'close' state. (If the success criteria matches then the circuit breaker trips to the 'close' state.)

As runtime behaviour, it appends a success handler for the given state to trip circuit breaker into the 'close' state when the given thresholds reached

Definitions of the params are state: StateHalfOpen(always half-open it is a hidden param) minSuccessRatio: min success ratio to trip the circuit breaker to close state minRequests: min number of requests before checking the ratio

Params with example: state: StateHalfOpen, minSuccessRatio: 99.5%, minRequests: 1000 The above configuration means that: On 'half-open' state, at min 1000 requests, if it counts 995 success then will trip to 'close' state

func WithCounter added in v1.0.0 

func WithCounter(c Counter) Option

WithCounter builds option to set stats counter

func WithFailureHandlers added in v1.0.0 

func WithFailureHandlers(state State, handlers ...FailureHandler) Option

WithFailureHandlers builds option to set on failure handlers, the provided handlers will be evaluate in the given order as option

func WithInitialState 

func WithInitialState(state State) Option

WithInitialState builds option to set initial state

func WithInvocationTimeout 

func WithInvocationTimeout(duration time.Duration) Option

WithInvocationTimeout builds option to set invocation timeout duration

func WithOpener added in v1.0.0 

func WithOpener(state State, minSuccessRatio float32, minRequests uint32) Option

WithOpener builds an option to set the default failure criteria to trip to 'open' state. (If the failure criteria matches then the circuit breaker trips to the 'open' state.)

As runtime behaviour, it prepends a failure handler for the given state to trip circuit breaker into the 'open' state when the given thresholds reached.

Definitions of the params are state: StateClose, StateHalfOpen minSuccessRatio: min success ratio ratio to keep the Circuit Breaker as is minRequests: min number of requests before checking the ratio

Params with example: state: StateClose, minSuccessRatio: 95%, minRequests: 10 The above configuration means that: On 'close' state, at min 10 requests, if it calculates the success ratio less than or equal to 95% then will trip to 'open' state

func WithResetTimer 

func WithResetTimer(t Timer) Option

WithResetTimer builds option to set reset timer

func WithRestrictors 

func WithRestrictors(restrictors ...Restrictor) Option

WithRestrictors builds option to set restrictors to restrict the invocations Restrictors does not effect the current state, but they can block the invocation depending on its own internal state values. If a restrictor blocks an invocation then it returns an error and `On Failure Handlers` get executed in order.

func WithStateChangeHandlers added in v1.0.0 

func WithStateChangeHandlers(handlers ...StateChangeHandler) Option

WithStateChangeHandlers builds option to set state change handlers, the provided handlers will be evaluate in the given order as option

func WithSuccessHandlers added in v1.0.0 

func WithSuccessHandlers(state State, handlers ...SuccessHandler) Option

WithSuccessHandlers builds option to set on failure handlers, the provided handlers will be evaluate in the given order as option

type Restrictor 

type Restrictor interface {
	// Check checks if restriction allows to run current invocation and errors
	// if not allowed the invocation
	Check(context.Context) (bool, error)

	// Defer executes exit rules of the restrictor right after the run process
	Defer()
}

Restrictor allows adding restriction to circuit breaker

type Shift added in v1.0.0 

type Shift struct {
	// contains filtered or unexported fields
}

Shift is an optioned circuit breaker implementation

func New added in v1.0.0 

func New(name string, opts ...Option) (*Shift, error)

New inits a new Circuit Breaker with given name and options

func (*Shift) Run added in v1.0.0 

func (s *Shift) Run(ctx context.Context, o Operator) (interface{}, error)

Run executes the given func with circuit breaker

func (*Shift) Trip added in v1.0.0 

func (s *Shift) Trip(to State, reasons ...error) error

Trip to desired state

type State 

type State int8

State circuit breaker state holder 

const (
	// StateUnknown is an unknown state for circuit breaker
	StateUnknown State = iota
	// StateClose close state for circuit breaker
	StateClose
	// StateHalfOpen half-open state for circuit breaker
	StateHalfOpen
	// StateOpen open state for circuit breaker
	StateOpen
)

func (State) String 

func (s State) String() string

type StateChangeHandler added in v1.0.0 

type StateChangeHandler interface {
	Handle(from, to State, stats Stats)
}

StateChangeHandler is an interface to handle state change events

type Stats added in v1.0.0 

type Stats struct {
	SuccessCount, FailureCount, TimeoutCount, RejectCount uint32
}

Stats is a structure which holds cb invocation metrics

type SuccessHandler added in v1.0.0 

type SuccessHandler interface {
	Handle(context.Context, interface{})
}

SuccessHandler is an interface to handle success events

type Timer 

type Timer interface {
	// Next returns the current duration and sets the next duration according to
	// the given error
	Next(error) time.Duration

	// Reset resets the current duration to the initial duration
	Reset()
}

Timer is an interface to set reset time duration dynamically depending on the occurred error on the invocation

type UnknownStateError added in v1.0.0 

type UnknownStateError struct {
	State State
}

UnknownStateError is a error tyoe for states

func (*UnknownStateError) Error added in v1.0.0 

func (e *UnknownStateError) Error() string

Source Files

View all Source files

Directories

Path Synopsis
Package mock is a generated GoMock package.
 Package mock is a generated GoMock package.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.