ratelimit

README

ratelimit

A Golang rate limit implementation which allows burst of request during the defined duration.

Differences with 'golang.org/x/time/rate#Limiter'

The original library i.e golang.org/x/time/rate implements classic token bucket algorithm allowing a burst of tokens and a refill that happens at a specified ratio by one unit at a time whereas this implementation is a variant that allows a burst of tokens just like "the token bucket" algorithm, but the refill happens entirely at the defined ratio.

This allows scanners to respect maximum defined rate limits, pause until the allowed interval hits, and then process again at maximum speed. The original library slowed down requests according to the refill ratio.

Example

An Example showing usage of ratelimit as a library is specified below:

package main

import (
	"context"
	"fmt"
	"time"

	"github.com/projectdiscovery/ratelimit"
)

func main() {

	// create a rate limiter by passing context, max tasks/requests , time interval
	limiter := ratelimit.New(context.Background(), 5, time.Duration(10*time.Second))

	save := time.Now()

	for i := 0; i < 10; i++ {
		// run limiter.Take() method before each task
		limiter.Take()
		fmt.Printf("Task %v completed after %v\n", i, time.Since(save))
	}

	/*
		Output:
		Task 0 completed after 4.083µs
		Task 1 completed after 111.416µs
		Task 2 completed after 118µs
		Task 3 completed after 121.083µs
		Task 4 completed after 124.583µs
		Task 5 completed after 10.001356375s
		Task 6 completed after 10.001524791s
		Task 7 completed after 10.001537583s
		Task 8 completed after 10.001542708s
		Task 9 completed after 10.001548666s
	*/
}

Documentation

Index

• Variables
• type Limiter
  • func New(ctx context.Context, max uint, duration time.Duration) *Limiter
  • func NewLeakyBucket(ctx context.Context, max uint, duration time.Duration) *Limiter
  • func NewUnlimited(ctx context.Context) *Limiter
  • func (limiter *Limiter) CanTake() bool
  • func (limiter *Limiter) GetLimit() uint
  • func (limiter *Limiter) SetDuration(d time.Duration)
  • func (limiter *Limiter) SetLimit(max uint)
  • func (limiter *Limiter) Stop()
  • func (limiter *Limiter) Take()
• type MultiLimiter
  • func NewMultiLimiter(ctx context.Context, opts *Options) (*MultiLimiter, error)
  • func (m *MultiLimiter) Add(opts *Options) error
  • func (m *MultiLimiter) AddAndTake(opts *Options)
  • func (m *MultiLimiter) CanTake(key string) bool
  • func (m *MultiLimiter) GetLimit(key string) (uint, error)
  • func (m *MultiLimiter) Stop(keys ...string)
  • func (m *MultiLimiter) Take(key string) error
• type Options
  • func (o *Options) Validate() error
• type Strategy

Variables

var (
	ErrKeyAlreadyExists = errorutil.NewWithTag("MultiLimiter", "key already exists")
	ErrKeyMissing       = errorutil.NewWithTag("MultiLimiter", "key does not exist")
)

Types

type Limiter

type Limiter struct {
	// contains filtered or unexported fields
}

Limiter allows a burst of request during the defined duration

func New

func New(ctx context.Context, max uint, duration time.Duration) *Limiter

New creates a new limiter instance with the tokens amount and the interval

func NewLeakyBucket

func NewLeakyBucket(ctx context.Context, max uint, duration time.Duration) *Limiter

NewUnlimited create a bucket with approximated unlimited tokens

func NewUnlimited

func NewUnlimited(ctx context.Context) *Limiter

NewUnlimited create a bucket with approximated unlimited tokens

func (*Limiter) CanTake

func (limiter *Limiter) CanTake() bool

CanTake checks if the rate limiter has any token

func (*Limiter) GetLimit

func (limiter *Limiter) GetLimit() uint

GetLimit returns current rate limit per given duration

func (*Limiter) SetDuration

func (limiter *Limiter) SetDuration(d time.Duration)

GetLimit returns current rate limit per given duration

func (*Limiter) SetLimit

func (limiter *Limiter) SetLimit(max uint)

GetLimit returns current rate limit per given duration

func (*Limiter) Stop

func (limiter *Limiter) Stop()

Stop the rate limiter canceling the internal context

func (*Limiter) Take

func (limiter *Limiter) Take()

Take one token from the bucket

type MultiLimiter

type MultiLimiter struct {
	// contains filtered or unexported fields
}

MultiLimiter is wrapper around Limiter than can limit based on a key

func NewMultiLimiter

func NewMultiLimiter(ctx context.Context, opts *Options) (*MultiLimiter, error)

NewMultiLimiter : Limits

func (*MultiLimiter) Add

func (m *MultiLimiter) Add(opts *Options) error

Add new bucket with key

func (*MultiLimiter) AddAndTake

func (m *MultiLimiter) AddAndTake(opts *Options)

AddAndTake adds key if not present and then takes token from bucket

func (*MultiLimiter) CanTake

func (m *MultiLimiter) CanTake(key string) bool

CanTake checks if the rate limiter with the given key has any token

func (*MultiLimiter) GetLimit

func (m *MultiLimiter) GetLimit(key string) (uint, error)

GetLimit returns current ratelimit of given key

func (*MultiLimiter) Stop

func (m *MultiLimiter) Stop(keys ...string)

Stop internal limiters with defined keys or all if no key is provided

func (*MultiLimiter) Take

func (m *MultiLimiter) Take(key string) error

Take one token from bucket returns error if key not present

type Options

type Options struct {
	Key         string // Unique Identifier
	IsUnlimited bool
	MaxCount    uint
	Duration    time.Duration
}

Options of MultiLimiter

func (*Options) Validate

func (o *Options) Validate() error

Validate given MultiLimiter Options

type Strategy

type Strategy uint8

const (
	None Strategy = iota
	LeakyBucket
)