pool

README

Generic worker pool

Go (1.18+) framework to run a pool of N workers

go get github.com/quakephil/generic-worker-pool@v0.2.1

Pool

• shared input/output channels

workers := pool.New[I, O](input, worker, output)

• channel synchronization for
  • concurrent runs of Process()
  • completion of Output()

result := workers.Wait(concurrency)

Input

func input(in chan<- I) {
  for ... {
    in <- I{ ... }
  }  
}

Worker

func worker(i I) I {
  i.update = ...
  return i
}

Output

func output(results <-chan I) (out O) {
  for result := range results {
    out.update += ...
  }
  return
}

Examples: https://github.com/QuakePhil/generic-worker-pool-examples

Documentation

Overview

Package pool implements a generic worker pool with shared channels.

Index

• type Pool
  • func New[I, O any](input func(chan<- I), worker func(I) I, output func(<-chan I) O) (p Pool[I, O])
  • func (p Pool[I, O]) Wait(concurrency int) O

Types

type Pool

type Pool[I, O any] struct {
	// contains filtered or unexported fields
}

Pool contains channels for the generic worker() to run at limited concurrency.

func New

func New[I, O any](
	input func(chan<- I),
	worker func(I) I,
	output func(<-chan I) O,
) (p Pool[I, O])

New creates the channels and kicks off the input producer and the output consumer.

func (Pool[I, O]) Wait

func (p Pool[I, O]) Wait(concurrency int) O

Wait runs workers concurrently and returns the result of output. If there's no worker, input is sent directly to output.