limiter

Documentation

Overview

Package limiter implements load shedding for servers.

Currently only supports setting a hard limit on a number of concurrently processed requests. Over-engineered to support more features in the future without significantly altering its API.

Index

• Variables
• func NewModule(opts *ModuleOptions) module.Module
• func NewModuleFromFlags() module.Module
• func NewUnaryServerInterceptor(l *Limiter) grpc.UnaryServerInterceptor
• func PeerLabelFromAuthState(ctx context.Context) string
• type Limiter
  • func New(opts Options) (*Limiter, error)
  • func (l *Limiter) CheckRequest(ctx context.Context, ri *RequestInfo) (done func(), err error)
  • func (l *Limiter) ReportMetrics(ctx context.Context)
• type ModuleOptions
  • func (o *ModuleOptions) Register(f *flag.FlagSet)
• type Options
• type RequestInfo

Variables

var ErrLimitReached = errors.New("the server limit reached")

ErrLimitReached is returned by CheckRequest when some limit is reached.

var ModuleName = module.RegisterName("go.chromium.org/luci/server/limiter")

ModuleName can be used to refer to this module when declaring dependencies.

Functions

func NewModule

func NewModule(opts *ModuleOptions) module.Module

NewModule returns a server module that installs default limiters applied to all routes/services in the server.

func NewModuleFromFlags

func NewModuleFromFlags() module.Module

NewModuleFromFlags is a variant of NewModule that initializes options through command line flags.

Calling this function registers flags in flag.CommandLine. They are usually parsed in server.Main(...).

func NewUnaryServerInterceptor

func NewUnaryServerInterceptor(l *Limiter) grpc.UnaryServerInterceptor

NewUnaryServerInterceptor returns a grpc.UnaryServerInterceptor that uses the given limiter to accept or drop gRPC requests.

func PeerLabelFromAuthState

func PeerLabelFromAuthState(ctx context.Context) string

PeerLabelFromAuthState looks at the auth.State in the context and derives a peer label from it.

Currently returns one of "unknown", "anonymous", "authenticated".

TODO(vadimsh): Have a small group with a whitelist of identities that are OK to use as peer label directly.

Types

type Limiter

type Limiter struct {
	// contains filtered or unexported fields
}

Limiter is a stateful runtime object that decides whether to accept or reject requests based on the current load (calculated from requests that went through it).

It is also responsible for maintaining monitoring metrics that describe its state and what requests it rejects.

May be running in advisory mode, in which it will do all the usual processing and logging, but won't actually reject the requests.

All methods are safe for concurrent use.

func New

func New(opts Options) (*Limiter, error)

New returns a new limiter.

Returns an error if options are invalid.

func (*Limiter) CheckRequest

func (l *Limiter) CheckRequest(ctx context.Context, ri *RequestInfo) (done func(), err error)

CheckRequest should be called before processing a request.

If it returns an error, the request should be declined as soon as possible with Unavailable/HTTP 503 status and the given error (which is an annotated ErrLimitReached).

If it succeeds, the request should be processed as usual, and the returned callback called afterwards to notify the limiter the processing is done.

func (*Limiter) ReportMetrics

func (l *Limiter) ReportMetrics(ctx context.Context)

ReportMetrics updates all limiter's gauge metrics to match the current state.

Must be called periodically (at least once per every metrics flush).

type ModuleOptions

type ModuleOptions struct {
	MaxConcurrentRPCs int64 // limit on a number of incoming concurrent RPCs (default is 100000, i.e. unlimited)
	AdvisoryMode      bool  // if set, don't enforce MaxConcurrentRPCs, but still report violations
}

ModuleOptions contains configuration of the server module that installs default limiters applied to all routes/services in the server.

func (*ModuleOptions) Register

func (o *ModuleOptions) Register(f *flag.FlagSet)

Register registers the command line flags.

type Options

type Options struct {
	Name                  string // used for metric fields, logs and error messages
	AdvisoryMode          bool   // if true, don't actually reject requests, just log
	MaxConcurrentRequests int64  // a hard limit on a number of concurrent requests
}

Options contains configuration of a single Limiter instance.

type RequestInfo

type RequestInfo struct {
	CallLabel string // an RPC or an endpoint being called (if known)
	PeerLabel string // who's making the request (if known), see also peer.go
}

RequestInfo holds information about a single inbound request.

Used by the limiter to decide whether to accept or reject the request.

Pretty sparse now, but in the future will contain fields like cost, a QoS class and an attempt count, which will help the limiter to decide what requests to drop.

Fields `CallLabel` and `PeerLabel` are intentionally pretty generic, since they will be used only as labels in internal maps and metric fields. Their internal structure and meaning are not important to the limiter, but the cardinality of the set of their possible values must be reasonably bounded.