Documentation ¶
Overview ¶
Package concurrent makes it easy to execute a list of jobs concurrently with a simple closure while using a finite number of goroutines (concurrency width).
Three broad patterns are supported and described below. For each, the package user can easily cap the maximum concurrency width, that is clipped to the maximum number of CPUs on the system in all cases.
This package was created because the author kept finding the need to implement these patterns over and over.
Grouped Execution Pattern ¶
The idea here is to take `n` jobs, split them into groups ("batches" or "chunks"). Each invocation `i` of the closure is given an index range `[m_i, n_i)` that specifies a non-overlapping group. The union of all invocations covers the index range `[0, n)`.
If one of the invocations returns an error, the first error received is returned, but all invocations are executed before returning.
func GroupedExample() { num := int[]{1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16} var sum uint32 concurrent.RunGrouped(len(num), 0, func(m, n int) { localSum := 0 for j := m; j < n; j++ { localSum += num[j] } atomic.AddUint32(&sum, localSum) }) }
Sweep Execution Pattern ¶
The idea here is similar to the Grouped pattern, except that the closure is invoked once per job, and if any invocation returns an error, no more invocations are scheduled. This allows errors to "short circuit" execution and return earlier than Grouped equivalent.
Like the Grouped pattern, the concurrency width is limited, to reduce goroutine overheads.
func SweepExample() { // A trivial example, to contrast to the Grouped pattern example above. // You would almost surely not implement a sum in this manner. ;-) num := int[]{1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16} var sum uint32 concurrent.RunSweep(len(num), 0, func(j int) { atomic.AddUint32(&sum, num[j]) }) }
Runner Pattern ¶
A `Runner` is also provided, that allows jobs to be scheduled without having to know how many jobs are required up-front. The implementation of the Sweep pattern uses this functionality.
Index ¶
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
func RunGrouped ¶
RunGrouped is like RunGroupedErr but without errors.
func RunGroupedErr ¶
RunGroupedErr the given func f concurrently using up to the specified number of maxThreads, or equal to the number of CPUs on the system if passed zero. The number of jobs is given by count, and the range of jobs [m, n) are passed to the callback f.
func RunSweepErr ¶
RunSweepErr will use at most maxThreads (or equal to the number of CPUs on the system if zero), to run func f concurrently, returning the first error received. If an error is reported, some func f may not be executed.
Types ¶
type Runner ¶
Runner runs jobs with a specified maximum limit on concurrency.
func NewRunner ¶
NewRunner returns a new Runner that executes jobs concurrently with at most n jobs running at any one time.
func (*Runner) Errors ¶
Errors returns all of the errors reported by jobs. The order is given by job completion, not submission.
func (*Runner) Failures ¶
Failures returns the number of errors reported by jobs so far. This is useful for callers to fail-fast and stop submitting jobs if an error has been reported.
func (*Runner) Finish ¶
Finish waits for all executing jobs to complete, and returns the number of successful jobs completed.