chunkgroup

README

Chunkgroup

Package chunkgroup provides a way to schedule the execution of a function when a chunk of data is ready.

Example

package main

import (
	"fmt"
	"log"

	"github.com/celrenheit/chunkgroup"
)

func main() {
	cg := chunkgroup.New(10, 5, func(items []int) error {
		fmt.Println(items)
		return nil
	})

	for i := range 105 {
		cg.Add(i)
	}

	if err := cg.Flush(); err != nil {
		log.Fatal(err)
	}
}

This should output something like this (order may be different due to concurrency being over 1):

[40 41 42 43 44 45 46 47 48 49]
[30 31 32 33 34 35 36 37 38 39]
[50 51 52 53 54 55 56 57 58 59]
[10 11 12 13 14 15 16 17 18 19]
[70 71 72 73 74 75 76 77 78 79]
[0 1 2 3 4 5 6 7 8 9]
[60 61 62 63 64 65 66 67 68 69]
[20 21 22 23 24 25 26 27 28 29]
[100 101 102 103 104]
[90 91 92 93 94 95 96 97 98 99]
[80 81 82 83 84 85 86 87 88 89]

Documentation

Index

• type ExecFunc
• type Group
  • func New[T any](size, concurrency int, fn ExecFunc[T]) *Group[T]
  • func WithContext[T any](ctx context.Context, size, concurrency int, fn ExecFunc[T]) (*Group[T], context.Context)
  • func (g *Group[T]) Add(item T)
  • func (g *Group[T]) Flush() error

Types

type ExecFunc

type ExecFunc[T any] func(context.Context, []T) error

ExecFunc is a function that takes a slice of items and returns an error.

type Group

type Group[T any] struct {
	// contains filtered or unexported fields
}

Group is a struct that holds a slice of items and a function to execute on them. The function is executed when the slice reaches its capacity. The function is executed concurrently with a limit of concurrency.

func New

func New[T any](size, concurrency int, fn ExecFunc[T]) *Group[T]

New creates a new Group with the given size and concurrency.

func WithContext

func WithContext[T any](ctx context.Context, size, concurrency int, fn ExecFunc[T]) (*Group[T], context.Context)

WithContext creates a new Group with the given context, size, and concurrency. The context is used to cancel the execution of the function.

func (*Group[T]) Add

func (g *Group[T]) Add(item T)

Add adds an item to the Group.

func (*Group[T]) Flush

func (g *Group[T]) Flush() error

Flush executes the function on the remaining items. It MUST be called after all items have been added.