gopool

README

gopool

Introduction

Package gopool is a high-performance goroutine pool which aims to reuse goroutines and limit the number of goroutines.

It is an alternative to the go keyword.

This package is a fork of github.com/bytedance/gopkg/util/gopool.

Features

• High performance
• Auto-recovering panics
• Limit goroutine numbers
• Reuse goroutine stack

QuickStart

Just replace your go func(){...} with gopool.Go(func(){...}).

old:

go func() {
	// do your job
}()

new:

gopool.Go(func() {
	// do your job
})

// or with context
gopool.CtxGo(ctx, func() {
	// do your job
})

Or create a dedicated pool for specific workload:

myPool := gopool.NewPool(&gopool.Config{
	// configuration
})

myPool.Go(func() {
	// do your job
})
myPool.CtxGo(ctx, func() {
	// do your job
})

Or create a pool to execute a handler to process values of a specific type:

myHandler := func(ctx context.Context, arg SomeType) {
	// do your job
}
myPool := gopool.NewTypedPool(myHandler, &gopool.Config{
	// configuration
})

myPool.Go(varOfSomeType)
myPool.CtxGo(ctx, varOfSomeType)

See package doc for more information.

Documentation

Overview

Package gopool is a high-performance goroutine pool which aims to reuse goroutines and limit the number of goroutines.

Index

• func CtxGo(ctx context.Context, f func())
• func Go(f func())
• type Config
  • func NewConfig() *Config
• type Pool
  • func NewPool(config *Config) *Pool
• type TypedPool
  • func NewTypedPool[T any](config *Config, handler func(context.Context, T)) *TypedPool[T]
  • func (p *TypedPool) AdhocWorkerCount() int32
  • func (p *TypedPool) AdhocWorkerLimit() int32
  • func (p *TypedPool[T]) CtxGo(ctx context.Context, arg T)
  • func (p *TypedPool[T]) Go(arg T)
  • func (p *TypedPool) Name() string
  • func (p *TypedPool) PermanentWorkerCount() int32
  • func (p *TypedPool) SetAdhocWorkerLimit(limit int)

Functions

func CtxGo

func CtxGo(ctx context.Context, f func())

CtxGo is preferred over Go.

func Go

func Go(f func())

Go is an alternative to the go keyword, which is able to recover panic, reuse goroutine stack, limit goroutine numbers, etc.

See package doc for detailed introduction.

Types

type Config

type Config struct {

	// Name optionally specifies the name of a pool instance.
	Name string

	// New goroutine will be created if len(queuedTasks) >= ScaleThreshold,
	// it defaults to 1, which means always start a new adhoc worker before
	// reaching the limit of total adhoc worker number.
	ScaleThreshold int

	// PermanentWorkerNum specifies the number of permanent workers to spawn
	// when creating a Pool, it defaults to 0 (no permanent worker).
	// Note that a permanent worker's goroutine stack is reused,
	// the memory won't be freed in the entire program lifetime.
	//
	// Generally you may want to set this to zero for common workloads,
	// tweak it for special workloads which benefits from reusing goroutine stacks.
	PermanentWorkerNum int

	// AdhocWorkerLimit specifies the initial limit of adhoc workers,
	// 0 or negative value means no limit.
	//
	// The limit of adhoc worker number can be changed by calling
	// SetAdhocWorkerLimit.
	AdhocWorkerLimit int

	// PanicHandler specifies a handler when panic occurs.
	// By default, a panic message with stack information is logged.
	PanicHandler func(context.Context, any)
}

Config is used to config a Pool instance.

func NewConfig

func NewConfig() *Config

NewConfig creates a default Config.

type Pool

type Pool = TypedPool[func()]

Pool manages a goroutine pool and tasks for better performance, it reuses goroutines and limits the number of goroutines.

func NewPool

func NewPool(config *Config) *Pool

NewPool creates a new pool with the config.

type TypedPool

type TypedPool[T any] struct {
	// contains filtered or unexported fields
}

TypedPool is a task-specific pool. A TypedPool is like pool, but it executes a handler to process values of a specific type. Compared to Pool, it helps to reduce unnecessary memory allocation of closures when submitting tasks.

func NewTypedPool

func NewTypedPool[T any](config *Config, handler func(context.Context, T)) *TypedPool[T]

NewTypedPool creates a new task-specific pool with given handler and config.

func (*TypedPool) AdhocWorkerCount

func (p *TypedPool) AdhocWorkerCount() int32

AdhocWorkerCount returns the number of running adhoc workers.

func (*TypedPool) AdhocWorkerLimit

func (p *TypedPool) AdhocWorkerLimit() int32

AdhocWorkerLimit returns the current limit of adhoc workers.

func (*TypedPool[T]) CtxGo

func (p *TypedPool[T]) CtxGo(ctx context.Context, arg T)

CtxGo submits a task to the pool, it's preferred over Go.

func (*TypedPool[T]) Go

func (p *TypedPool[T]) Go(arg T)

Go submits a task to the pool.

func (*TypedPool) Name

func (p *TypedPool) Name() string

Name returns the name of a pool.

func (*TypedPool) PermanentWorkerCount

func (p *TypedPool) PermanentWorkerCount() int32

PermanentWorkerCount returns the number of permanent workers.

func (*TypedPool) SetAdhocWorkerLimit

func (p *TypedPool) SetAdhocWorkerLimit(limit int)

SetAdhocWorkerLimit changes the limit of adhoc workers. 0 or negative value means no limit.