Package limiter implements load shedding for servers.

    Currently only supports setting a hard limit on a number of concurrently processed requests. Over-engineered to support more features in the future without significantly altering its API.



    This section is empty.


    View Source
    var ErrLimitReached = errors.New("the server limit reached")

      ErrLimitReached is returned by CheckRequest when some limit is reached.

      View Source
      var ModuleName = module.RegisterName("")

        ModuleName can be used to refer to this module when declaring dependencies.


        func NewModule

        func NewModule(opts *ModuleOptions) module.Module

          NewModule returns a server module that installs default limiters applied to all routes/services in the server.

          func NewModuleFromFlags

          func NewModuleFromFlags() module.Module

            NewModuleFromFlags is a variant of NewModule that initializes options through command line flags.

            Calling this function registers flags in flag.CommandLine. They are usually parsed in server.Main(...).

            func NewUnaryServerInterceptor

            func NewUnaryServerInterceptor(l *Limiter) grpc.UnaryServerInterceptor

              NewUnaryServerInterceptor returns a grpc.UnaryServerInterceptor that uses the given limiter to accept or drop gRPC requests.

              func PeerLabelFromAuthState

              func PeerLabelFromAuthState(ctx context.Context) string

                PeerLabelFromAuthState looks at the auth.State in the context and derives a peer label from it.

                Currently returns one of "unknown", "anonymous", "authenticated".

                TODO(vadimsh): Have a small group with a whitelist of identities that are OK to use as peer label directly.


                type Limiter

                type Limiter struct {
                	// contains filtered or unexported fields

                  Limiter is a stateful runtime object that decides whether to accept or reject requests based on the current load (calculated from requests that went through it).

                  It is also responsible for maintaining monitoring metrics that describe its state and what requests it rejects.

                  May be running in advisory mode, in which it will do all the usual processing and logging, but won't actually reject the requests.

                  All methods are safe for concurrent use.

                  func New

                  func New(opts Options) (*Limiter, error)

                    New returns a new limiter.

                    Returns an error if options are invalid.

                    func (*Limiter) CheckRequest

                    func (l *Limiter) CheckRequest(ctx context.Context, ri *RequestInfo) (done func(), err error)

                      CheckRequest should be called before processing a request.

                      If it returns an error, the request should be declined as soon as possible with Unavailable/HTTP 503 status and the given error (which is an annotated ErrLimitReached).

                      If it succeeds, the request should be processed as usual, and the returned callback called afterwards to notify the limiter the processing is done.

                      func (*Limiter) ReportMetrics

                      func (l *Limiter) ReportMetrics(ctx context.Context)

                        ReportMetrics updates all limiter's gauge metrics to match the current state.

                        Must be called periodically (at least once per every metrics flush).

                        type ModuleOptions

                        type ModuleOptions struct {
                        	MaxConcurrentRPCs int64 // limit on a number of incoming concurrent RPCs (default is 100000, i.e. unlimited)
                        	AdvisoryMode      bool  // if set, don't enforce MaxConcurrentRPCs, but still report violations

                          ModuleOptions contains configuration of the server module that installs default limiters applied to all routes/services in the server.

                          func (*ModuleOptions) Register

                          func (o *ModuleOptions) Register(f *flag.FlagSet)

                            Register registers the command line flags.

                            type Options

                            type Options struct {
                            	Name                  string // used for metric fields, logs and error messages
                            	AdvisoryMode          bool   // if true, don't actually reject requests, just log
                            	MaxConcurrentRequests int64  // a hard limit on a number of concurrent requests

                              Options contains configuration of a single Limiter instance.

                              type RequestInfo

                              type RequestInfo struct {
                              	CallLabel string // an RPC or an endpoint being called (if known)
                              	PeerLabel string // who's making the request (if known), see also peer.go

                                RequestInfo holds information about a single inbound request.

                                Used by the limiter to decide whether to accept or reject the request.

                                Pretty sparse now, but in the future will contain fields like cost, a QoS class and an attempt count, which will help the limiter to decide what requests to drop.

                                Fields `CallLabel` and `PeerLabel` are intentionally pretty generic, since they will be used only as labels in internal maps and metric fields. Their internal structure and meaning are not important to the limiter, but the cardinality of the set of their possible values must be reasonably bounded.