ratelimit

README

goprocess/ratelimit - ratelimit children creation

• goprocess: https://github.com/jbenet/goprocess
• Godoc: https://godoc.org/github.com/jbenet/goprocess/ratelimit

Documentation

Overview

Package ratelimit is part of github.com/jbenet/goprocess. It provides a simple process that ratelimits child creation. This is done internally with a channel/semaphore. So the call `RateLimiter.LimitedGo` may block until another child is Closed().

Index

• func LimitChan(limit int) chan struct{}
• type RateLimiter
  • func NewRateLimiter(parent process.Process, limit int) *RateLimiter
  • func (rl *RateLimiter) LimitedGo(f process.ProcessFunc)

Functions

func LimitChan

func LimitChan(limit int) chan struct{}

LimitChan returns a rate-limiting channel. it is the usual, simple, golang-idiomatic rate-limiting semaphore. This function merely initializes it with certain buffer size, and sends that many values, so it is ready to be used.

Types

type RateLimiter

type RateLimiter struct {
	process.Process
	// contains filtered or unexported fields
}

RateLimiter limits the spawning of children. It does so with an internal semaphore. Note that Go will continue to be the unlimited process.Process.Go, and ONLY the added function `RateLimiter.LimitedGo` will honor the limit. This is to improve readability and avoid confusion for the reader, particularly if code changes over time.

func NewRateLimiter

func NewRateLimiter(parent process.Process, limit int) *RateLimiter

func (*RateLimiter) LimitedGo

func (rl *RateLimiter) LimitedGo(f process.ProcessFunc)

LimitedGo creates a new process, adds it as a child, and spawns the ProcessFunc f in its own goroutine, but may block according to the internal rate limit. It is equivalent to:

func(f process.ProcessFunc) {
   <-limitch
   p.Go(func (child process.Process) {
     f(child)
     f.Close() // make sure its children close too!
     limitch<- struct{}{}
   })

/ }

It is useful to construct simple asynchronous workers, children of p, and rate limit their creation, to avoid spinning up too many, too fast. This is great for providing backpressure to producers.