limiter

README

Limiter

Dead simple rate limit middleware for Go.

• Simple API
• "Store" approach for backend
• Redis support (but not tied too)
• Middlewares: HTTP and Gin

Installation

Using Dep

$ dep ensure -add github.com/blackcatzzzz/limiter@v3-dep

Issue: https://github.com/blackcatzzzz/limiter/issues/57

Usage

In five steps:

• Create a limiter.Rate instance (the number of requests per period)
• Create a limiter.Store instance (see Redis or In-Memory)
• Create a limiter.Limiter instance that takes store and rate instances as arguments
• Create a middleware instance using the middleware of your choice
• Give the limiter instance to your middleware initializer

Example:

// Create a rate with the given limit (number of requests) for the given
// period (a time.Duration of your choice).
import "github.com/blackcatzzzz/limiter"

rate := limiter.Rate{
    Period: 1 * time.Hour,
    Limit:  1000,
}

// You can also use the simplified format "<limit>-<period>"", with the given
// periods:
//
// * "S": second
// * "M": minute
// * "H": hour
// * "D": day
//
// Examples:
//
// * 5 reqs/second: "5-S"
// * 10 reqs/minute: "10-M"
// * 1000 reqs/hour: "1000-H"
// * 2000 reqs/day: "2000-D"
//
rate, err := limiter.NewRateFromFormatted("1000-H")
if err != nil {
    panic(err)
}

// Then, create a store. Here, we use the bundled Redis store. Any store
// compliant to limiter.Store interface will do the job. The defaults are
// "limiter" as Redis key prefix and a maximum of 3 retries for the key under
// race condition.
import "github.com/blackcatzzzz/limiter/drivers/store/redis"

store, err := redis.NewStore(client)
if err != nil {
    panic(err)
}

// Alternatively, you can pass options to the store with the "WithOptions"
// function. For example, for Redis store:
import "github.com/blackcatzzzz/limiter/drivers/store/redis"

store, err := redis.NewStoreWithOptions(pool, limiter.StoreOptions{
    Prefix:   "your_own_prefix",
    MaxRetry: 4,
})
if err != nil {
    panic(err)
}

// Or use a in-memory store with a goroutine which clears expired keys.
import "github.com/blackcatzzzz/limiter/drivers/store/memory"

store := memory.NewStore()

// Then, create the limiter instance which takes the store and the rate as arguments.
// Now, you can give this instance to any supported middleware.
instance := limiter.New(store, rate)

See middleware examples:

• HTTP
• Gin
• Beego
• Chi

How it works

The ip address of the request is used as a key in the store.

If the key does not exist in the store we set a default value with an expiration period.

You will find two stores:

• Redis: rely on TTL and incrementing the rate limit on each request.
• In-Memory: rely on a fork of go-cache with a goroutine to clear expired keys using a default interval.

When the limit is reached, a 429 HTTP status code is sent.

Why Yet Another Package

You could ask us: why yet another rate limit package?

Because existing packages did not suit our needs.

We tried a lot of alternatives:

1. Throttled. This package uses the generic cell-rate algorithm. To cite the documentation: "The algorithm has been slightly modified from its usual form to support limiting with an additional quantity parameter, such as for limiting the number of bytes uploaded". It is brillant in term of algorithm but documentation is quite unclear at the moment, we don't need burst feature for now, impossible to get a correct After-Retry (when limit exceeds, we can still make a few requests, because of the max burst) and it only supports http.Handler middleware (we use Gin). Currently, we only need to return 429 and X-Ratelimit-* headers for n reqs/duration.

2. Speedbump. Good package but maybe too lightweight. No Reset support, only one middleware for Gin framework and too Redis-coupled. We rather prefer to use a "store" approach.

3. Tollbooth. Good one too but does both too much and too little. It limits by remote IP, path, methods, custom headers and basic auth usernames... but does not provide any Redis support (only in-memory) and a ready-to-go middleware that sets X-Ratelimit-* headers. tollbooth.LimitByRequest(limiter, r) only returns an HTTP code.

4. ratelimit. Probably the closer to our needs but, once again, too lightweight, no middleware available and not active (last commit was in August 2014). Some parts of code (Redis) comes from this project. It should deserve much more love.

There are other many packages on GitHub but most are either too lightweight, too old (only support old Go versions) or unmaintained. So that's why we decided to create yet another one.

Contributing

• Ping us on twitter:
  • @oibafsellig
  • @thoas
  • @novln_
• Fork the project
• Fix bugs

Don't hesitate ;)

Documentation

Index

• Constants
• Variables
• func GetIP(r *http.Request, options ...Options) net.IP
• func GetIPWithMask(r *http.Request, options ...Options) net.IP
• type Context
• type Limiter
  • func New(store Store, rate Rate, options ...Option) *Limiter
  • func (limiter *Limiter) Get(ctx context.Context, key string) (Context, error)
  • func (limiter *Limiter) GetIP(r *http.Request) net.IP
  • func (limiter *Limiter) GetIPKey(r *http.Request) string
  • func (limiter *Limiter) GetIPWithMask(r *http.Request) net.IP
  • func (limiter *Limiter) Peek(ctx context.Context, key string) (Context, error)
  • func (limiter *Limiter) Reset(ctx context.Context, key string) (Context, error)
• type Option
  • func WithIPv4Mask(mask net.IPMask) Option
  • func WithIPv6Mask(mask net.IPMask) Option
  • func WithTrustForwardHeader(enable bool) Option
• type Options
• type Rate
  • func NewRateFromFormatted(formatted string) (Rate, error)
• type Store
• type StoreOptions

Constants

const (
	// DefaultPrefix is the default prefix to use for the key in the store.
	DefaultPrefix = "limiter"

	// DefaultMaxRetry is the default maximum number of key retries under
	// race condition (mainly used with database-based stores).
	DefaultMaxRetry = 3

	// DefaultCleanUpInterval is the default time duration for cleanup.
	DefaultCleanUpInterval = 30 * time.Second
)

Variables

var (
	// DefaultIPv4Mask defines the default IPv4 mask used to obtain user IP.
	DefaultIPv4Mask = net.CIDRMask(32, 32)
	// DefaultIPv6Mask defines the default IPv6 mask used to obtain user IP.
	DefaultIPv6Mask = net.CIDRMask(128, 128)
)

Functions

func GetIP

func GetIP(r *http.Request, options ...Options) net.IP

GetIP returns IP address from request. If options is defined and TrustForwardHeader is true, it will lookup IP in X-Forwarded-For and X-Real-IP headers.

func GetIPWithMask

func GetIPWithMask(r *http.Request, options ...Options) net.IP

GetIPWithMask returns IP address from request by applying a mask.

Types

type Context

type Context struct {
	Limit     int64
	Remaining int64
	Reset     int64
	Reached   bool
}

Context is the limit context.

type Limiter

type Limiter struct {
	Store   Store
	Rate    Rate
	Options Options
}

Limiter is the limiter instance.

func New

func New(store Store, rate Rate, options ...Option) *Limiter

New returns an instance of Limiter.

func (*Limiter) Get

func (limiter *Limiter) Get(ctx context.Context, key string) (Context, error)

Get returns the limit for given identifier.

func (*Limiter) GetIP

func (limiter *Limiter) GetIP(r *http.Request) net.IP

GetIP returns IP address from request.

func (*Limiter) GetIPKey

func (limiter *Limiter) GetIPKey(r *http.Request) string

GetIPKey extracts IP from request and returns hashed IP to use as store key.

func (*Limiter) GetIPWithMask

func (limiter *Limiter) GetIPWithMask(r *http.Request) net.IP

GetIPWithMask returns IP address from request by applying a mask.

func (*Limiter) Peek

func (limiter *Limiter) Peek(ctx context.Context, key string) (Context, error)

Peek returns the limit for given identifier, without modification on current values.

func (*Limiter) Reset

func (limiter *Limiter) Reset(ctx context.Context, key string) (Context, error)

Reset sets the limit for given identifier to zero.

type Option

type Option func(*Options)

Option is a functional option.

func WithIPv4Mask

func WithIPv4Mask(mask net.IPMask) Option

WithIPv4Mask will configure the limiter to use given mask for IPv4 address.

func WithIPv6Mask

func WithIPv6Mask(mask net.IPMask) Option

WithIPv6Mask will configure the limiter to use given mask for IPv6 address.

func WithTrustForwardHeader

func WithTrustForwardHeader(enable bool) Option

WithTrustForwardHeader will configure the limiter to trust X-Real-IP and X-Forwarded-For headers.

type Options

type Options struct {
	// IPv4Mask defines the mask used to obtain a IPv4 address.
	IPv4Mask net.IPMask
	// IPv6Mask defines the mask used to obtain a IPv6 address.
	IPv6Mask net.IPMask
	// TrustForwardHeader enable parsing of X-Real-IP and X-Forwarded-For headers to obtain user IP.
	TrustForwardHeader bool
}

Options are limiter options.

type Rate

type Rate struct {
	Formatted string
	Period    time.Duration
	Limit     int64
}

Rate is the rate.

func NewRateFromFormatted

func NewRateFromFormatted(formatted string) (Rate, error)

NewRateFromFormatted returns the rate from the formatted version.

type Store

type Store interface {
	// Get returns the limit for given identifier.
	Get(ctx context.Context, key string, rate Rate) (Context, error)
	// Peek returns the limit for given identifier, without modification on current values.
	Peek(ctx context.Context, key string, rate Rate) (Context, error)
	// Reset resets the limit to zero for given identifier.
	Reset(ctx context.Context, key string, rate Rate) (Context, error)
}

Store is the common interface for limiter stores.

type StoreOptions

type StoreOptions struct {
	// Prefix is the prefix to use for the key.
	Prefix string

	// MaxRetry is the maximum number of retry under race conditions.
	MaxRetry int

	// CleanUpInterval is the interval for cleanup.
	CleanUpInterval time.Duration
}

StoreOptions are options for store.