gobatch

package module
v0.0.0-...-54166d4 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Jul 26, 2019 License: Apache-2.0 Imports: 2 Imported by: 5

README

gobatch

Simple batch library for Golang.

How to Use

Install:

go get github.com/herryg91/gobatch

Example:

func fn1(workerID int, datas []interface{}) (err error) {
    //do something
    return
}

// every 100 datas or 15 second no activity, batch will be processed (fn1 will be run) with 2 worker
mBatch := gobatch.NewMemoryBatch(fn1, 100, time.Second*15, 2)

mBatch.Insert(interface{}{})
mBatch.Insert(interface{}{})
mBatch.Insert(interface{}{})

Need additional param to DoFn? You can do something like this

type additionalParam struct {
	RedisPool *redis.Pool
}

func (p additionalParam) fn1(workerID int, datas []interface{}) (err error) {
	// do something log.Println(p)
	return
}

mBatch := gobatch.NewMemoryBatch(additionalParam{pool}.fn1, 100, time.Second*15, 2)

Future Update:

  • Batch using Redis
  • Batch using File
  • Add / Decrease Worker on the fly

Documentation

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

This section is empty.

Types

type Batch

type Batch struct {
	// contains filtered or unexported fields
}

func NewMemoryBatch

func NewMemoryBatch(flushMaxSize int, flushMaxWait time.Duration, callback BatchFn, workerSize int) *Batch

func (*Batch) Flush

func (b *Batch) Flush()

func (*Batch) Insert

func (b *Batch) Insert(data interface{})

type BatchFn

type BatchFn func(workerID int, datas []interface{})

Directories

Path Synopsis
example
test command

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL