retry

package
v2.0.3 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Apr 19, 2021 License: Apache-2.0 Imports: 12 Imported by: 0

Documentation

Overview

`retry` provides client-side request retry logic for gRPC.

Client-Side Request Retry Interceptor

It allows for automatic retry, inside the generated gRPC code of requests based on the gRPC status of the reply. It supports unary (1:1), and server stream (1:n) requests.

By default the interceptors *are disabled*, preventing accidental use of retries. You can easily override the number of retries (setting them to more than 0) with a `grpc.ClientOption`, e.g.:

myclient.Ping(ctx, goodPing, grpc_retry.WithMax(5))

Other default options are: retry on `ResourceExhausted` and `Unavailable` gRPC codes, use a 50ms linear backoff with 10% jitter.

For chained interceptors, the retry interceptor will call every interceptor that follows it whenever when a retry happens.

Please see examples for more advanced use.

Example (Initialization)

Simple example of using the default interceptor configuration.

package main

import (
	"google.golang.org/grpc"

	"github.com/grpc-ecosystem/go-grpc-middleware/v2/interceptors/retry"
)

func main() {
	_, _ = grpc.Dial("myservice.example.com",
		grpc.WithStreamInterceptor(retry.StreamClientInterceptor()),
		grpc.WithUnaryInterceptor(retry.UnaryClientInterceptor()),
	)
}
Output:

Example (InitializationWithExponentialBackoff)

Example with an exponential backoff starting with 100ms.

Each next interval is the previous interval multiplied by 2.

package main

import (
	"time"

	"google.golang.org/grpc"

	"github.com/grpc-ecosystem/go-grpc-middleware/v2/interceptors/retry"
)

func main() {
	opts := []retry.CallOption{
		retry.WithBackoff(retry.BackoffExponential(100 * time.Millisecond)),
	}
	_, _ = grpc.Dial("myservice.example.com",
		grpc.WithStreamInterceptor(retry.StreamClientInterceptor(opts...)),
		grpc.WithUnaryInterceptor(retry.UnaryClientInterceptor(opts...)),
	)
}
Output:

Example (InitializationWithOptions)

Complex example with a 100ms linear backoff interval, and retry only on NotFound and Unavailable.

package main

import (
	"time"

	"google.golang.org/grpc"
	"google.golang.org/grpc/codes"

	"github.com/grpc-ecosystem/go-grpc-middleware/v2/interceptors/retry"
)

func main() {
	opts := []retry.CallOption{
		retry.WithBackoff(retry.BackoffLinear(100 * time.Millisecond)),
		retry.WithCodes(codes.NotFound, codes.Aborted),
	}
	_, _ = grpc.Dial("myservice.example.com",
		grpc.WithStreamInterceptor(retry.StreamClientInterceptor(opts...)),
		grpc.WithUnaryInterceptor(retry.UnaryClientInterceptor(opts...)),
	)
}
Output:

Example (SimpleCall)

Simple example of an idempotent `ServerStream` call, that will be retried automatically 3 times.

package main

import (
	"context"
	"io"
	"time"

	"google.golang.org/grpc"

	"github.com/grpc-ecosystem/go-grpc-middleware/v2/interceptors/retry"
	"github.com/grpc-ecosystem/go-grpc-middleware/v2/testing/testpb"
)

var cc *grpc.ClientConn

func main() {
	ctx, cancel := context.WithTimeout(context.TODO(), 1*time.Second)
	defer cancel()

	client := testpb.NewTestServiceClient(cc)
	stream, _ := client.PingList(ctx, &testpb.PingListRequest{}, retry.WithMax(3))

	for {
		_, err := stream.Recv() // retries happen here
		if err == io.EOF {
			break
		} else if err != nil {
			return
		}
	}
}
Output:

Index

Examples

Constants

View Source
const (
	AttemptMetadataKey = "x-retry-attempty"
)

Variables

View Source
var (
	// DefaultRetriableCodes is a set of well known types gRPC codes that should be retri-able.
	//
	// `ResourceExhausted` means that the user quota, e.g. per-RPC limits, have been reached.
	// `Unavailable` means that system is currently unavailable and the client should retry again.
	DefaultRetriableCodes = []codes.Code{codes.ResourceExhausted, codes.Unavailable}
)

Functions

func StreamClientInterceptor

func StreamClientInterceptor(optFuncs ...CallOption) grpc.StreamClientInterceptor

StreamClientInterceptor returns a new retrying stream client interceptor for server side streaming calls.

The default configuration of the interceptor is to not retry *at all*. This behaviour can be changed through options (e.g. WithMax) on creation of the interceptor or on call (through grpc.CallOptions).

Retry logic is available *only for ServerStreams*, i.e. 1:n streams, as the internal logic needs to buffer the messages sent by the client. If retry is enabled on any other streams (ClientStreams, BidiStreams), the retry interceptor will fail the call.

func UnaryClientInterceptor

func UnaryClientInterceptor(optFuncs ...CallOption) grpc.UnaryClientInterceptor

UnaryClientInterceptor returns a new retrying unary client interceptor.

The default configuration of the interceptor is to not retry *at all*. This behaviour can be changed through options (e.g. WithMax) on creation of the interceptor or on call (through grpc.CallOptions).

Types

type BackoffFunc

type BackoffFunc func(attempt uint) time.Duration

BackoffFunc denotes a family of functions that control the backoff duration between call retries.

They are called with an identifier of the attempt, and should return a time the system client should hold off for. If the time returned is longer than the `context.Context.Deadline` of the request the deadline of the request takes precedence and the wait will be interrupted before proceeding with the next iteration.

func BackoffExponential

func BackoffExponential(scalar time.Duration) BackoffFunc

BackoffExponential produces increasing intervals for each attempt.

The scalar is multiplied times 2 raised to the current attempt. So the first retry with a scalar of 100ms is 100ms, while the 5th attempt would be 1.6s.

func BackoffExponentialWithJitter

func BackoffExponentialWithJitter(scalar time.Duration, jitterFraction float64) BackoffFunc

BackoffExponentialWithJitter creates an exponential backoff like BackoffExponential does, but adds jitter.

func BackoffLinear

func BackoffLinear(waitBetween time.Duration) BackoffFunc

BackoffLinear is very simple: it waits for a fixed period of time between calls.

func BackoffLinearWithJitter

func BackoffLinearWithJitter(waitBetween time.Duration, jitterFraction float64) BackoffFunc

BackoffLinearWithJitter waits a set period of time, allowing for jitter (fractional adjustment).

For example waitBetween=1s and jitter=0.10 can generate waits between 900ms and 1100ms.

type BackoffFuncContext

type BackoffFuncContext func(ctx context.Context, attempt uint) time.Duration

BackoffFuncContext denotes a family of functions that control the backoff duration between call retries.

They are called with an identifier of the attempt, and should return a time the system client should hold off for. If the time returned is longer than the `context.Context.Deadline` of the request the deadline of the request takes precedence and the wait will be interrupted before proceeding with the next iteration. The context can be used to extract request scoped metadata and context values.

type CallOption

type CallOption struct {
	grpc.EmptyCallOption // make sure we implement private after() and before() fields so we don't panic.
	// contains filtered or unexported fields
}

CallOption is a grpc.CallOption that is local to grpc_retry.

func Disable

func Disable() CallOption

Disable disables the retry behaviour on this call, or this interceptor.

Its semantically the same to `WithMax`

func WithBackoff

func WithBackoff(bf BackoffFunc) CallOption

WithBackoff sets the `BackoffFunc` used to control time between retries.

func WithBackoffContext

func WithBackoffContext(bf BackoffFuncContext) CallOption

WithBackoffContext sets the `BackoffFuncContext` used to control time between retries.

func WithCodes

func WithCodes(retryCodes ...codes.Code) CallOption

WithCodes sets which codes should be retried.

Please *use with care*, as you may be retrying non-idempotent calls.

You cannot automatically retry on Cancelled and Deadline, please use `WithPerRetryTimeout` for these.

func WithMax

func WithMax(maxRetries uint) CallOption

WithMax sets the maximum number of retries on this call, or this interceptor.

func WithOnRetryCallback

func WithOnRetryCallback(fn OnRetryCallback) CallOption

WithOnRetryCallback sets the callback to use when a retry occurs.

By default, when no callback function provided, we will just print a log to trace

func WithPerRetryTimeout

func WithPerRetryTimeout(timeout time.Duration) CallOption

WithPerRetryTimeout sets the RPC timeout per call (including initial call) on this call, or this interceptor.

The context.Deadline of the call takes precedence and sets the maximum time the whole invocation will take, but WithPerRetryTimeout can be used to limit the RPC time per each call.

For example, with context.Deadline = now + 10s, and WithPerRetryTimeout(3 * time.Seconds), each of the retry calls (including the initial one) will have a deadline of now + 3s.

A value of 0 disables the timeout overrides completely and returns to each retry call using the parent `context.Deadline`.

Note that when this is enabled, any DeadlineExceeded errors that are propagated up will be retried.

Example

This is an example of an `Unary` call that will also retry on deadlines.

Because the passed in context has a `5s` timeout, the whole `Ping` invocation should finish within that time. However, by default all retried calls will use the parent context for their deadlines. This means, that unless you shorten the deadline of each call of the retry, you won't be able to retry the first call at all.

`WithPerRetryTimeout` allows you to shorten the deadline of each retry call, allowing you to fit multiple retries in the single parent deadline.

package main

import (
	"context"
	"time"

	"google.golang.org/grpc"

	"github.com/grpc-ecosystem/go-grpc-middleware/v2/interceptors/retry"
	"github.com/grpc-ecosystem/go-grpc-middleware/v2/testing/testpb"
)

var cc *grpc.ClientConn

func main() {
	ctx, cancel := context.WithTimeout(context.TODO(), 5*time.Second)
	defer cancel()

	client := testpb.NewTestServiceClient(cc)
	_, _ = client.Ping(
		ctx,
		&testpb.PingRequest{},
		retry.WithMax(3),
		retry.WithPerRetryTimeout(1*time.Second))
}
Output:

type OnRetryCallback

type OnRetryCallback func(ctx context.Context, attempt uint, err error)

OnRetryCallback is the type of function called when a retry occurs.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL