itogami

README

Itogami

The best goroutine pool in terms of performance implemented using a lock-free stack

By limiting concurrency with a fixed pool size and recycling goroutines, itogami saves a lot of memory as compared to using unlimited goroutines and also itogami is the fastest and has the least allocs/op among all existing goroutine pool implementations

Benchmarks to support the above claims here

Installation

You need Golang 1.19.x or above

$ go get github.com/alphadose/itogami@v0.3.0

Usage

package main

import (
	"fmt"
	"sync"
	"sync/atomic"
	"time"

	"github.com/alphadose/itogami"
)

const runTimes uint32 = 1000

var sum uint32

func myFunc(i uint32) {
	atomic.AddUint32(&sum, i)
	fmt.Printf("run with %d\n", i)
}

func demoFunc() {
	time.Sleep(10 * time.Millisecond)
	println("Hello World")
}

func examplePool() {
	var wg sync.WaitGroup
	// Use the common pool
	pool := itogami.NewPool(10)

	syncCalculateSum := func() {
		demoFunc()
		wg.Done()
	}
	for i := uint32(0); i < runTimes; i++ {
		wg.Add(1)
		// Submit task to the pool
		pool.Submit(syncCalculateSum)
	}
	wg.Wait()
	println("finished all tasks")
}

func examplePoolWithFunc() {
	var wg sync.WaitGroup
	// Use the pool with a pre-defined function
	pool := itogami.NewPoolWithFunc(10, func(i uint32) {
		myFunc(i)
		wg.Done()
	})
	for i := uint32(0); i < runTimes; i++ {
		wg.Add(1)
		// Invoke the function with a value
		pool.Invoke(i)
	}
	wg.Wait()
	fmt.Printf("finish all tasks, result is %d\n", sum)
}

func main() {
	examplePool()
	examplePoolWithFunc()
}

Benchmarks

Benchmarking was performed against:-

1. Unlimited goroutines
2. Ants
3. Gamma-Zero-Worker-Pool
4. golang.org/x/sync/errgroup
5. Bytedance GoPool

Pool size -> 50k

CPU -> M1, arm64, 8 cores, 3.2 GHz

OS -> darwin

Results were computed from benchstat of 30 cases

name                   time/op
UnlimitedGoroutines-8   331ms ± 4%
ErrGroup-8              515ms ± 9%
AntsPool-8              582ms ± 9%
GammaZeroPool-8         740ms ±13%
BytedanceGoPool-8       572ms ±18%
ItogamiPool-8           337ms ± 1%

name                   alloc/op
UnlimitedGoroutines-8  96.3MB ± 0%
ErrGroup-8              120MB ± 0%
AntsPool-8             22.4MB ± 6%
GammaZeroPool-8        18.8MB ± 1%
BytedanceGoPool-8      82.2MB ± 2%
ItogamiPool-8          25.6MB ± 2%

name                   allocs/op
UnlimitedGoroutines-8   2.00M ± 0%
ErrGroup-8              3.00M ± 0%
AntsPool-8              1.10M ± 2%
GammaZeroPool-8         1.08M ± 0%
BytedanceGoPool-8       2.59M ± 1%
ItogamiPool-8           1.08M ± 0%

The following conclusions can be drawn from the above results:-

1. Itogami is the fastest among all goroutine pool implementations and slightly slower than unlimited goroutines
2. Itogami has the least allocs/op and hence the memory usage scales really well with high load
3. The memory used per operation is in the acceptable range of other pools and drastically lower than unlimited goroutines
4. The tolerance (± %) for Itogami is quite low for all 3 metrics indicating that the algorithm is quite stable overall

Benchmarking code available here

Documentation

Index

• func Fastrand() uint32
• func GetG() unsafe.Pointer
• func ProcPin() int
• func ProcUnpin()
• func Readgstatus(gp unsafe.Pointer) uint32
• type Pool
  • func NewPool(size uint64) *Pool
  • func (self *Pool) Submit(task func())
• type PoolWithFunc
  • func NewPoolWithFunc[T any](size uint64, task func(T)) *PoolWithFunc[T]
  • func (self *PoolWithFunc[T]) Invoke(value T)

Functions

func Fastrand

func Fastrand() uint32

func GetG

func GetG() unsafe.Pointer

GetG returns the pointer to the current goroutine defined in the asm files

func ProcPin

func ProcPin() int

func ProcUnpin

func ProcUnpin()

func Readgstatus

func Readgstatus(gp unsafe.Pointer) uint32

Types

type Pool

type Pool struct {
	// contains filtered or unexported fields
}

Pool represents the thread-pool for performing any kind of task ( type -> func() {} )

func NewPool

func NewPool(size uint64) *Pool

NewPool returns a new thread pool

func (*Pool) Submit

func (self *Pool) Submit(task func())

Submit submits a new task to the pool it first tries to use already parked goroutines from the stack if any if there are no available worker goroutines, it tries to add a new goroutine to the pool if the pool capacity is not exceeded in case the pool capacity hit its maximum limit, this function yields the processor to other goroutines and loops again for finding available workers

type PoolWithFunc

type PoolWithFunc[T any] struct {
	// contains filtered or unexported fields
}

PoolWithFunc is used for spawning workers for a single pre-defined function with myriad inputs useful for throughput bound cases has lower memory usage and allocs per op than the default Pool

( type -> func(T) {} ) where T is a generic parameter

func NewPoolWithFunc

func NewPoolWithFunc[T any](size uint64, task func(T)) *PoolWithFunc[T]

NewPoolWithFunc returns a new PoolWithFunc

func (*PoolWithFunc[T]) Invoke

func (self *PoolWithFunc[T]) Invoke(value T)

Invoke invokes the pre-defined method in PoolWithFunc by assigning the data to an already existing worker or spawning a new worker given queue size is in limits