Documentation
¶
Overview ¶
Package ratelimiting provides a per-key rate limiter interface using the token bucket algorithm.
RateLimiter is deliberately narrow: Allow says yes or no for a key, and Close releases whatever the implementation held. Implementations live in this package (in-memory), ratelimiting/redis (shared, sliding window), and ratelimiting/noop. Select one from configuration with ratelimiting/config.
Refusals ¶
Allow expresses a refusal as (false, nil), because the caller is usually deciding what to do next rather than propagating a failure. ErrRateLimited exists for the callers that have to hand a refusal back as an error instead — an http.RoundTripper has nowhere else to put it. errors/http maps that sentinel to 429 and errors/grpc to RESOURCE_EXHAUSTED, so a refusal crosses the wire as itself rather than as whatever the fallback mapping produces. errors/http maps back too: ErrorForCode turns the E116 in a response envelope into this same sentinel, which is what lets a typed client hand its caller the value an in-process caller would have gotten.
Retry hints ¶
RetryHinter is the optional half of the interface: an implementation that can say when a refused key will next be allowed implements it, and callers ask through RetryAfterFor rather than asserting for it themselves. It is separate from RateLimiter because a limiter fronting a third party often cannot answer, and an invented Retry-After is worse than none — clients obey it.
What the in-memory limiter keeps ¶
The in-memory limiter holds one token bucket per key, and the keys are whatever the caller limits on — client addresses and principal IDs, for the two obvious choices, neither of which has a bounded key space. So it reclaims them: a key that has gone unconsulted for twice its window is dropped on the next pass of a sweeper the constructor starts and Close stops.
The window is the time a bucket takes to refill a full burst at the steady rate, which is the same quantity ratelimiting/redis turns into the length of its sliding window and expires its keys against. It is derived rather than configured because it is the point past which a bucket has nothing left to remember: a key that returns after the TTL is handed a full burst, which is exactly what the bucket it left behind would have refilled to. Eviction is therefore invisible to callers, and Close is not optional — an unclosed limiter keeps its sweeper, and itself, alive for the life of the process.
The TTL cannot cover one case: a flood of distinct keys inside a single window, where nothing has been idle long enough to reclaim. That is what DefaultMaxLimiters bounds, evicting the least recently seen — the buckets closest to being refilled, and so the cheapest to forget. Unlike a TTL eviction this one does forgive whatever the evicted keys still owed, which is why the two are counted separately: a non-zero rate of capacity evictions says the bound is being hit and some keys are getting their allowance back early. Raise it with WithMaxLimiters.
Guarding a service ¶
The transport adapters are ratelimiting/http (a routing.Middleware answering 429, with Retry-After) and ratelimiting/grpc (a unary interceptor answering RESOURCE_EXHAUSTED, with RetryInfo). Outbound, httpclient.WithRateLimit spends from a limiter before each request. All three take the same RateLimiter, so one configured limiter can govern a service's inbound and outbound traffic alike.
Not quota ¶
This package answers "too fast right now". "Too much this month" is metering: the two have different remedies — wait versus buy more — and conflating them tells a client to retry when it should stop.
Index ¶
Examples ¶
Constants ¶
const DefaultMaxLimiters = 100_000
DefaultMaxLimiters bounds how many per-key limiters the in-memory limiter holds at once unless WithMaxLimiters says otherwise.
It is a memory ceiling, not a tuning parameter: at a few dozen bytes per entry this is single-digit megabytes, which is far above any legitimate count of distinct keys active within one window and far below what it costs to let a public endpoint's key space grow unchecked.
Variables ¶
var ErrRateLimited = errors.New("rate limited")
ErrRateLimited reports that a limiter refused an operation. Allow expresses a refusal as (false, nil) because the caller is usually deciding what to do next; this sentinel exists for the callers that have to hand the refusal back as an error instead — an http.RoundTripper, for one, has nowhere else to put it. Callers branch on it with errors.Is rather than on a bare false.
Functions ¶
func RetryAfterFor ¶
RetryAfterFor asks limiter when key will next be allowed, returning (0, false) when it cannot say — either because it does not implement RetryHinter or because it has no estimate for this key.
It exists so that the transport adapters share one type assertion instead of each writing their own, which is what keeps an unhinted refusal behave identically over HTTP and over gRPC.
A negative duration is reported as no hint rather than passed along: it would mean the key is already allowed, and telling a client to come back in the past is the same as telling it nothing.
Types ¶
type InMemoryRateLimiter ¶
type InMemoryRateLimiter struct {
// contains filtered or unexported fields
}
InMemoryRateLimiter is the process-local RateLimiter, backed by a token bucket per key. It is exported, and returned by NewInMemoryRateLimiter, so a caller who has chosen it can depend on that choice rather than on the interface every limiter shares.
func NewInMemoryRateLimiter ¶
func NewInMemoryRateLimiter(requestsPerSec float64, burstSize int, opts ...Option) (*InMemoryRateLimiter, error)
NewInMemoryRateLimiter returns a RateLimiter that uses per-key limiters in memory.
The returned limiter owns a goroutine that reclaims the limiters of keys that have stopped arriving, so Close is not optional: a limiter that is never closed keeps that goroutine, and itself, alive for the life of the process. See the package documentation for what is retained and for how long.
Example ¶
package main
import (
"context"
"fmt"
"github.com/primandproper/platform-go/v12/ratelimiting"
)
func main() {
limiter, err := ratelimiting.NewInMemoryRateLimiter(10.0, 5)
if err != nil {
panic(err)
}
// Closing is what stops the sweeper that reclaims the limiters of keys that
// have stopped arriving.
defer limiter.Close()
var allowed bool
allowed, err = limiter.Allow(context.Background(), "user-123")
if err != nil {
panic(err)
}
fmt.Println(allowed)
}
Output: true
func (*InMemoryRateLimiter) Close ¶
func (r *InMemoryRateLimiter) Close() error
Close stops the sweeper and drops every per-key limiter.
It is safe to call more than once, and it waits for the sweeper to exit, so a caller that closes a limiter holds no goroutine of ours afterwards.
func (*InMemoryRateLimiter) RetryAfter ¶
RetryAfter reports how long key's bucket needs to hold a whole token again.
It reads the bucket rather than reserving from it. rate.Limiter.Reserve answers the same question, but spends the token to do it — so asking when to come back would itself push the answer further out, and a refused caller that asked twice would be told to wait longer for having asked.
It does not count as touching the key either: this is the refusal path, so the Allow that produced the refusal has already stamped the key, and a hint that kept a bucket resident would let a client hold a limiter open by asking when it may return.
A key with no bucket yet reports no hint rather than zero: the caller is about to be allowed, so there is nothing to wait for and nothing to say.
type Option ¶
type Option func(*options)
Option configures the in-memory rate limiter this package constructs. The zero configuration works: an absent metrics provider records nothing, and the limiter reclaims idle keys against the wall clock under the default bound.
func WithClock ¶
WithClock supplies the clock the limiter ages its per-key state against.
It measures idleness and paces the sweep, and nothing else — the token buckets themselves are golang.org/x/time/rate's, which read the wall clock on their own. So this makes eviction testable, not the limiting itself.
func WithMaxLimiters ¶
WithMaxLimiters bounds how many per-key limiters are held at once, replacing DefaultMaxLimiters.
The bound only ever binds on a flood of distinct keys inside a single window, where the idle TTL has had nothing to reclaim yet. Evicting under it hands the affected keys a full burst early, so raising it trades memory for a limit that holds under key-cardinality pressure.
A non-positive n removes the bound, leaving the idle TTL as the only thing that reclaims memory. That is a deliberate choice for a limiter whose key space is known to be small — a fixed set of tenants, say — and a bad one for anything keyed on client addresses.
func WithMetricsProvider ¶
WithMetricsProvider attaches a metrics provider for the limiter's allowed and rejected counters.
func WithTracerProvider ¶
WithTracerProvider attaches a tracer provider, so the limiter's spans are children of the request that consulted it.
type RateLimiter ¶
RateLimiter limits the rate of operations per key.
type RetryHinter ¶
type RetryHinter interface {
// RetryAfter estimates how long key must wait before Allow would say yes
// again. ok is false when there is no estimate to give — including for a
// key this limiter has not seen.
//
// It is an estimate by construction: nothing reserves the capacity it
// describes, so another caller may spend it first. Treat it as a floor
// under how long to wait, not as a promise about what happens after.
RetryAfter(ctx context.Context, key string) (time.Duration, bool)
}
RetryHinter is the optional half of RateLimiter: an implementation able to say when a refused key will next be allowed implements it, and the callers that can pass such a hint on — an HTTP middleware writing Retry-After, a gRPC interceptor attaching RetryInfo — ask for it through RetryAfterFor.
It is a second interface rather than a third method on RateLimiter because not every limiter can answer it. One fronting a third party knows only what that party told it, which is often nothing. Widening RateLimiter would force those implementations to invent a number, and an invented Retry-After is worse than none at all: clients obey it, so a wrong one either wastes the capacity that was actually free or sends everyone back at the same instant.
The refusal itself is still expressed by Allow. A hint is an improvement on a refusal, never a substitute for one.
Source Files
Directories
¶
| Path | Synopsis |
|---|---|
|
Package ratelimitingcfg selects and builds a rate limiter from configuration: the in-process memory limiter, the Redis-backed one, or noop.
|
Package ratelimitingcfg selects and builds a rate limiter from configuration: the in-process memory limiter, the Redis-backed one, or noop. |
|
Package grpc adapts ratelimiting to inbound gRPC.
|
Package grpc adapts ratelimiting to inbound gRPC. |
|
Package http adapts ratelimiting to inbound HTTP.
|
Package http adapts ratelimiting to inbound HTTP. |
|
Package noop is the ratelimiting.RateLimiter that never limits: Allow returns true for every key, and there is no counter, window, or store behind it to consult.
|
Package noop is the ratelimiting.RateLimiter that never limits: Allow returns true for every key, and there is no counter, window, or store behind it to consult. |
|
Package redis implements ratelimiting.RateLimiter as a sliding window kept in a Redis sorted set.
|
Package redis implements ratelimiting.RateLimiter as a sliding window kept in a Redis sorted set. |