repeat

package module
v1.5.1 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Feb 13, 2020 License: MIT Imports: 4 Imported by: 13

README

Repeat

Alt text GoDoc Build Status Go Report Status GoCover

Go implementation of different backoff strategies useful for retrying operations and heartbeating.

Examples

Backoff

Let's imagine that we need to do a REST call on remote server but it could fail with a bunch of different issues. We can repeat failed operation using exponential backoff policies.

Exponential backoff is an algorithm that uses feedback to multiplicatively decrease the rate of some process, in order to gradually find an acceptable rate.

The example below tries to repeat operation 10 times using a full jitter backoff. See algorithm details here.

    // An example operation that do some useful stuff.
    // It fails five first times.
    var last time.Time
    op := func(c int) error {
        printInfo(c, &last)
        if c < 5 {
            return repeat.HintTemporary(errors.New("can't connect to a server"))
        }
        return nil
    }

    // Repeat op on any error, with 10 retries, with a backoff.
    err := repeat.Repeat(
        // Our op with additional call counter.
        repeat.FnWithCounter(op),
        // Force the repetition to stop in case the previous operation
        // returns nil.
        repeat.StopOnSuccess(),
        // 10 retries max.
        repeat.LimitMaxTries(10),
        // Specify a delay that uses a backoff.
        repeat.WithDelay(
            repeat.FullJitterBackoff(500*time.Millisecond).Set(),
        ),
    )

The example of output:

Attempt #0, Delay 0s
Attempt #1, Delay 373.617912ms
Attempt #2, Delay 668.004225ms
Attempt #3, Delay 1.220076558s
Attempt #4, Delay 2.716156336s
Attempt #5, Delay 6.458431017s
Repetition process is finished with: <nil>
Backoff with timeout

The example below is almost the same as the previous one. It adds one important feature - possibility to cancel operation repetition using context's timeout.

    // A context with cancel.
    // Repetition will be cancelled in 3 seconds.
    ctx, cancelFunc := context.WithCancel(context.Background())
    go func() {
        time.Sleep(3 * time.Second)
        cancelFunc()
    }()

    // Repeat op on any error, with 10 retries, with a backoff.
    err := repeat.Repeat(
        ...
        // Specify a delay that uses a backoff.
        repeat.WithDelay(
            repeat.FullJitterBackoff(500*time.Millisecond).Set(),
            repeat.SetContext(ctx),
        ),
        ...
    )

The example of output:

Attempt #0, Delay 0s
Attempt #1, Delay 358.728046ms
Attempt #2, Delay 845.361787ms
Attempt #3, Delay 61.527485ms
Repetition process is finished with: context canceled
Heartbeating

Let's imagine we need to periodically report execution progress to remote server. The example below repeats the operation each second until it will be cancelled using passed context.

    // An example operation that do heartbeat.
    var last time.Time
    op := func(c int) error {
        printInfo(c, &last)
        return nil
    }

    // A context with cancel.
    // Repetition will be cancelled in 7 seconds.
    ctx, cancelFunc := context.WithCancel(context.Background())
    go func() {
        time.Sleep(7 * time.Second)
        cancelFunc()
    }()

    err := repeat.Repeat(
        // Heartbeating op.
        repeat.FnWithCounter(op),
        // Delay with fixed backoff and context.
        repeat.WithDelay(
            repeat.FixedBackoff(time.Second).Set(),
            repeat.SetContext(ctx),
        ),
    )

The example of output:

Attempt #0, Delay 0s
Attempt #1, Delay 1.001129426s
Attempt #2, Delay 1.000155727s
Attempt #3, Delay 1.001131014s
Attempt #4, Delay 1.000500428s
Attempt #5, Delay 1.0008985s
Attempt #6, Delay 1.000417057s
Repetition process is finished with: context canceled
Heartbeating with error timeout

The example below is almost the same as the previous one but it will be cancelled using special error timeout. This timeout resets each time the operations return nil.

    // An example operation that do heartbeat.
    // It fails 5 times after 3 successfull tries.
    var last time.Time
    op := func(c int) error {
        printInfo(c, &last)
        if c > 3 && c < 8 {
            return repeat.HintTemporary(errors.New("can't connect to a server"))
        }
        return nil
    }

    err := repeat.Repeat(
        // Heartbeating op.
        repeat.FnWithCounter(op),
        // Delay with fixed backoff and error timeout.
        repeat.WithDelay(
            repeat.FixedBackoff(time.Second).Set(),
            repeat.SetErrorsTimeout(3*time.Second),
        ),
    )

The example of output:

Attempt #0, Delay 0s
Attempt #1, Delay 1.001634616s
Attempt #2, Delay 1.004912408s
Attempt #3, Delay 1.001021358s
Attempt #4, Delay 1.001249459s
Attempt #5, Delay 1.004320833s
Repetition process is finished with: can't connect to a server

Documentation

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

func Cause 

func Cause(err error) error

Cause extracts the cause error from TemporaryError and StopError or return the passed one.

func Done added in v1.4.1 

func Done(e error) error

Done does nothing, returns nil.

func ExponentialBackoffAlgorithm 

func ExponentialBackoffAlgorithm(initialDelay time.Duration, maxDelay time.Duration, multiplier float64, jitter float64) func() time.Duration

ExponentialBackoffAlgorithm implements classic caped exponential backoff.

Example (initialDelay=1, maxDelay=30, Multiplier=2, Jitter=0.5): Attempt Delay ------- -------------------------- 0 1 + random [-0.5...0.5] 1 2 + random [-1...1] 2 4 + random [-2...2] 3 8 + random [-4...4] 4 16 + random [-8...8] 5 32 + random [-16...16] 6 64 + random [-32...32] = 30

func FixedBackoffAlgorithm 

func FixedBackoffAlgorithm(delay time.Duration) func() time.Duration

FixedBackoffAlgorithm implements backoff with a fixed delay.

func FullJitterBackoffAlgorithm 

func FullJitterBackoffAlgorithm(baseDelay time.Duration, maxDelay time.Duration) func() time.Duration

FullJitterBackoffAlgorithm implements caped exponential backoff with jitter. Algorithm is fast because it does not use floating point arithmetics.

Details: https://aws.amazon.com/blogs/architecture/exponential-backoff-and-jitter/

Example (BaseDelay=1, maxDelay=30): Call Delay ------- ---------------- 1 random [0...1] 2 random [0...2] 3 random [0...4] 4 random [0...8] 5 random [0...16] 6 random [0...30] 7 random [0...30]

func HintStop 

func HintStop(e error) error

HintStop makes a StopError.

func HintTemporary 

func HintTemporary(e error) error

HintTemporary makes a TemporaryError.

func IsStop added in v1.4.1 

func IsStop(e error) bool

IsStop checks if passed error is StopError.

func IsTemporary added in v1.4.1 

func IsTemporary(e error) bool

IsTemporary checks if passed error is TemporaryError.

func Nope added in v1.4.1 

func Nope(e error) error

Nope does nothing, returns input error.

func Once added in v1.4.1 

func Once(ops ...Operation) error

Once composes the operations and executes the result once.

It is guaranteed that the first op will be called at least once.

func Repeat 

func Repeat(ops ...Operation) error

Repeat repeat operations until one of them stops the repetition.

It is guaranteed that the first op will be called at least once.

func SetContext 

func SetContext(ctx context.Context) func(*DelayOptions)

SetContext allows to set a context instead of default one.

func SetContextHintStop added in v1.4.1 

func SetContextHintStop() func(*DelayOptions)

SetContextHintStop instructs to use HintStop(nil) instead of context error in case of context expiration.

func SetErrorsTimeout 

func SetErrorsTimeout(t time.Duration) func(*DelayOptions)

SetErrorsTimeout specifies the maximum timeout for repetition in case of error. This timeout is reset each time when the repetition operation is successfully completed.

Default value is maximum time.Duration value.

Types

type DelayOptions 

type DelayOptions struct {
	ErrorsTimeout   time.Duration
	Backoff         func() time.Duration
	Context         context.Context
	ContextHintStop bool
}

DelayOptions holds parameters for a heartbeat process.

type ExponentialBackoffBuilder 

type ExponentialBackoffBuilder struct {
	// MaxDelay specifies maximum value of a delay calculated by the
	// algorithm.
	//
	// Default value is maximum time.Duration value.
	MaxDelay time.Duration

	// InitialDelay specifies an initial delay for the algorithm.
	//
	// Default value is equal to 1 second.
	InitialDelay time.Duration

	// Miltiplier specifies a multiplier for the last calculated
	// or specified delay.
	//
	// Default value is 2.
	Multiplier float64

	// Jitter specifies randomization factor [0..1].
	//
	// Default value is 0.
	Jitter float64
	// contains filtered or unexported fields
}

ExponentialBackoffBuilder is an option builder.

func ExponentialBackoff 

func ExponentialBackoff(initialDelay time.Duration) *ExponentialBackoffBuilder

ExponentialBackoff create a builder for Delay's option.

func (*ExponentialBackoffBuilder) Set 

func (s *ExponentialBackoffBuilder) Set() func(*DelayOptions)

Set creates a Delay' option.

func (*ExponentialBackoffBuilder) WithInitialDelay 

func (s *ExponentialBackoffBuilder) WithInitialDelay(d time.Duration) *ExponentialBackoffBuilder

WithInitialDelay allows to set InitialDelay.

InitialDelay specifies an initial delay for the algorithm.

func (*ExponentialBackoffBuilder) WithJitter 

func (s *ExponentialBackoffBuilder) WithJitter(j float64) *ExponentialBackoffBuilder

WithJitter allows to set Jitter.

Jitter specifies randomization factor [0..1].

Default value is 0.

func (*ExponentialBackoffBuilder) WithMaxDelay 

func (s *ExponentialBackoffBuilder) WithMaxDelay(d time.Duration) *ExponentialBackoffBuilder

WithMaxDelay allows to set MaxDelay.

MaxDelay specifies the maximum value of a delay calculated by the algorithm.

Default value is maximum time.Duration value.

func (*ExponentialBackoffBuilder) WithMultiplier 

func (s *ExponentialBackoffBuilder) WithMultiplier(m float64) *ExponentialBackoffBuilder

WithMultiplier allows to set Multiplier.

Miltiplier specifies a multiplier for the last calculated

Default value is 2.

type FixedBackoffBuilder 

type FixedBackoffBuilder struct {
	// Delay specifyes fixed delay value.
	Delay time.Duration
}

FixedBackoffBuilder is an option builder.

func FixedBackoff 

func FixedBackoff(delay time.Duration) *FixedBackoffBuilder

FixedBackoff create a builder for Delay's option.

func (*FixedBackoffBuilder) Set 

func (s *FixedBackoffBuilder) Set() func(*DelayOptions)

Set creates a Delay' option.

type FullJitterBackoffBuilder 

type FullJitterBackoffBuilder struct {
	// MaxDelay specifies maximum value of a delay calculated by the
	// algorithm.
	//
	// Default value is maximum time.Duration value.
	MaxDelay time.Duration

	// BaseDelay specifies base of an exponent.
	BaseDelay time.Duration
}

FullJitterBackoffBuilder is an option builder.

func FullJitterBackoff 

func FullJitterBackoff(baseDelay time.Duration) *FullJitterBackoffBuilder

FullJitterBackoff create a builder for Delay's option.

func (*FullJitterBackoffBuilder) Set 

func (s *FullJitterBackoffBuilder) Set() func(*DelayOptions)

Set creates a Delay' option.

func (*FullJitterBackoffBuilder) WithBaseDelay 

func (s *FullJitterBackoffBuilder) WithBaseDelay(d time.Duration) *FullJitterBackoffBuilder

WithBaseDelay allows to set BaseDelay.

BaseDelay specifies base of an exponent.

func (*FullJitterBackoffBuilder) WithMaxDelay 

func (s *FullJitterBackoffBuilder) WithMaxDelay(d time.Duration) *FullJitterBackoffBuilder

WithMaxDelay allows to set MaxDelay.

MaxDelay specifies the maximum value of a delay calculated by the algorithm.

Default value is maximum time.Duration value.

type OpWrapper added in v1.4.1 

type OpWrapper func(Operation) Operation

OpWrapper is the type of function for repetition.

func WrStopOnContextError added in v1.4.1 

func WrStopOnContextError(ctx context.Context) OpWrapper

WrStopOnContextError stops an operation in case of context error.

func WrWith added in v1.5.0 

func WrWith(c, d Operation) OpWrapper

WrWith returns wrapper that calls C (constructor) at first, then ops, then D (destructor). D will be called in any case if C returns nil.

type Operation 

type Operation func(error) error

Operation is the type of function for repetition.

func Compose 

func Compose(ops ...Operation) Operation

Compose composes all passed operations into a single one.

func Fn 

func Fn(op func() error) Operation

Fn wraps operation with no arguments.

func FnDone added in v1.4.1 

func FnDone(op Operation) Operation

FnDone returns nil even if wrapped op returns an error.

func FnES added in v1.4.1 

func FnES(op func(error)) Operation

FnES wraps operation with no return value.

func FnHintStop added in v1.4.1 

func FnHintStop(op Operation) Operation

FnHintStop hints all operation errors as StopError.

func FnHintTemporary added in v1.4.1 

func FnHintTemporary(op Operation) Operation

FnHintTemporary hints all operation errors as temporary.

func FnNope added in v1.4.1 

func FnNope(op Operation) Operation

FnNope does not call pass op, returns input error.

func FnOnError added in v1.4.1 

func FnOnError(op Operation) Operation

FnOnError executes operation in case error is NOT nil.

func FnOnSuccess added in v1.4.1 

func FnOnSuccess(op Operation) Operation

FnOnSuccess executes operation in case of error is nil.

func FnOnlyOnce added in v1.4.1 

func FnOnlyOnce(op Operation) Operation

FnOnlyOnce executes op only once permanently.

func FnPanic added in v1.4.1 

func FnPanic(op Operation) Operation

FnPanic panics if op returns any error other than nil, TemporaryError and StopError.

func FnRepeat added in v1.4.1 

func FnRepeat(ops ...Operation) Operation

FnRepeat is a Repeat operation.

func FnS added in v1.4.1 

func FnS(op func()) Operation

FnS wraps operation with no arguments and return value.

func FnWithCounter 

func FnWithCounter(op func(int) error) Operation

FnWithCounter wraps operation with counter only.

func FnWithErrorAndCounter 

func FnWithErrorAndCounter(op func(error, int) error) Operation

FnWithErrorAndCounter wraps operation and adds call counter.

func Forward added in v1.4.1 

func Forward(op Operation) Operation

Forward returns the passed operation.

func LimitMaxTries 

func LimitMaxTries(max int) Operation

LimitMaxTries returns true if attempt number is less then max.

func StopOnSuccess 

func StopOnSuccess() Operation

StopOnSuccess returns true in case of error is nil.

func WithDelay 

func WithDelay(options ...func(hb *DelayOptions)) Operation

WithDelay constructs HeartbeatPredicate.

type Repeater added in v1.4.1 

type Repeater interface {
	Once(...Operation) error
	Repeat(...Operation) error
	Compose(...Operation) Operation
	FnRepeat(...Operation) Operation
}

Repeater represents general package concept.

func Cpp added in v1.4.1 

func Cpp(c, d Operation) Repeater

Cpp returns object that calls C (constructor) at first, then ops, then D (destructor). D will be called in any case if C returns nil.

Note! Cpp panics if D returns non nil error. Wrap it using Done if you log D's error or handle it somehow else.

func NewRepeater added in v1.4.1 

func NewRepeater() Repeater

NewRepeater sets up everything to be able to repeat operations.

func NewRepeaterExt added in v1.5.0 

func NewRepeaterExt(opw, copw OpWrapper) Repeater

NewRepeaterExt returns object that wraps all ops with with the given opw and wraps composed operation with the given copw.

func With added in v1.5.0 

func With(c, d Operation) Repeater

With returns object that calls C (constructor) at first, then ops, then D (destructor). D will be called in any case if C returns nil.

Note! D is able to hide original error an return nil or return error event if the original error is nil.

func WithContext added in v1.4.1 

func WithContext(ctx context.Context) Repeater

WithContext repeat operations until one of them stops the repetition or context will be canceled.

It is guaranteed that the first op will be called at least once.

func Wrap added in v1.4.1 

func Wrap(opw OpWrapper) Repeater

Wrap returns object that wraps all repeating ops with passed OpWrapper.

func WrapOnce added in v1.4.1 

func WrapOnce(copw OpWrapper) Repeater

WrapOnce returns object that wraps all repeating ops combined into a single op with passed OpWrapper calling it once.

type StopError 

type StopError struct {
	Cause error
}

StopError allows to stop repetition process without specifying a separate error.

This error never returns to the caller as is, only wrapped error.

func (*StopError) Error 

func (e *StopError) Error() string

type TemporaryError 

type TemporaryError struct {
	Cause error
}

TemporaryError allows not to stop repetitions process right now.

This error never returns to the caller as is, only wrapped error.

func (*TemporaryError) Error 

func (e *TemporaryError) Error() string

Source Files

View all Source files

Directories

Path Synopsis
examples
backoff
backoff_with_timeout
heartbeat
heartbeat_with_error_timeout

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.