Package limiter implements load shedding for servers.

    Currently only supports setting a hard limit on a number of concurrently processed requests. Over-engineered to support more features in the future without significantly altering its API.



    This section is empty.


    View Source
    var ErrLimitReached = errors.New("the server limit reached")

      ErrLimitReached is returned by CheckRequest when some limit is reached.


      func NewModule

      func NewModule(opts *ModuleOptions) module.Module

        NewModule returns a server module that installs default limiters applied to all routes/services in the server.

        func NewModuleFromFlags

        func NewModuleFromFlags() module.Module

          NewModuleFromFlags is a variant of NewModule that initializes options through command line flags.

          Calling this function registers flags in flag.CommandLine. They are usually parsed in server.Main(...).

          func NewUnaryServerInterceptor

          func NewUnaryServerInterceptor(l *Limiter) grpc.UnaryServerInterceptor

            NewUnaryServerInterceptor returns a grpc.UnaryServerInterceptor that uses the given limiter to accept or drop gRPC requests.

            func PeerLabelFromAuthState

            func PeerLabelFromAuthState(ctx context.Context) string

              PeerLabelFromAuthState looks at the auth.State in the context and derives a peer label from it.

              Currently returns one of "unknown", "anonymous", "authenticated".

              TODO(vadimsh): Have a small group with a whitelist of identities that are OK to use as peer label directly.


              type Limiter

              type Limiter struct {
              	// contains filtered or unexported fields

                Limiter is a stateful runtime object that decides whether to accept or reject requests based on the current load (calculated from requests that went through it).

                It is also responsible for maintaining monitoring metrics that describe its state and what requests it rejects.

                May be running in advisory mode, in which it will do all the usual processing and logging, but won't actually reject the requests.

                All methods are safe for concurrent use.

                func New

                func New(opts Options) (*Limiter, error)

                  New returns a new limiter.

                  Returns an error if options are invalid.

                  func (*Limiter) CheckRequest

                  func (l *Limiter) CheckRequest(ctx context.Context, ri *RequestInfo) (done func(), err error)

                    CheckRequest should be called before processing a request.

                    If it returns an error, the request should be declined as soon as possible with Unavailable/HTTP 503 status and the given error (which is an annotated ErrLimitReached).

                    If it succeeds, the request should be processed as usual, and the returned callback called afterwards to notify the limiter the processing is done.

                    func (*Limiter) ReportMetrics

                    func (l *Limiter) ReportMetrics(ctx context.Context)

                      ReportMetrics updates all limiter's gauge metrics to match the current state.

                      Must be called periodically (at least once per every metrics flush).

                      type ModuleOptions

                      type ModuleOptions struct {
                      	MaxConcurrentRPCs int64 // limit on a number of incoming concurrent RPCs (default is 100000, i.e. unlimited)
                      	AdvisoryMode      bool  // if set, don't enforce MaxConcurrentRPCs, but still report violations

                        ModuleOptions contains configuration of the server module that installs default limiters applied to all routes/services in the server.

                        func (*ModuleOptions) Register

                        func (o *ModuleOptions) Register(f *flag.FlagSet)

                          Register registers the command line flags.

                          type Options

                          type Options struct {
                          	Name                  string // used for metric fields, logs and error messages
                          	AdvisoryMode          bool   // if true, don't actually reject requests, just log
                          	MaxConcurrentRequests int64  // a hard limit on a number of concurrent requests

                            Options contains configuration of a single Limiter instance.

                            type RequestInfo

                            type RequestInfo struct {
                            	CallLabel string // an RPC or an endpoint being called (if known)
                            	PeerLabel string // who's making the request (if known), see also peer.go

                              RequestInfo holds information about a single inbound request.

                              Used by the limiter to decide whether to accept or reject the request.

                              Pretty sparse now, but in the future will contain fields like cost, a QoS class and an attempt count, which will help the limiter to decide what requests to drop.

                              Fields `CallLabel` and `PeerLabel` are intentionally pretty generic, since they will be used only as labels in internal maps and metric fields. Their internal structure and meaning are not important to the limiter, but the cardinality of the set of their possible values must be reasonably bounded.