batchprocess

Documentation

Index

• func FNV1av32(key string) uint32
• type BatchItem
• type Batcher
  • func NewBatcher(id int, cfg *BatcherGroupCfg) *Batcher
  • func (b *Batcher) Close()
  • func (b *Batcher) Put(key string, job interface{}) error
  • func (b *Batcher) Start(doBatch DoBatch)
  • func (b *Batcher) Stat() int64
• type BatcherGroup
  • func NewBatcherGroup(cfg *BatcherGroupCfg) BatcherGroup
  • func (bg BatcherGroup) Close()
  • func (bg BatcherGroup) Put(key string, job interface{}) error
  • func (bg BatcherGroup) Start(doBatch DoBatch)
  • func (bg BatcherGroup) Stat()
• type BatcherGroupCfg
• type DoBatch
• type SourceItem

Functions

func FNV1av32

func FNV1av32(key string) uint32

FNV1av32 哈希函数. 参考 http://www.isthe.com/chongo/tech/comp/fnv/index.html#FNV-source

Types

type BatchItem

type BatchItem struct {
	CreatedTime time.Time
	Items       []interface{}
	NextItemIdx int
}

BatchItem 批处理队列中待处理的事务单元

type Batcher

type Batcher struct {
	// contains filtered or unexported fields
}

Batcher 匹处理器, 用于批量处理指定的事务

func NewBatcher

func NewBatcher(id int, cfg *BatcherGroupCfg) *Batcher

NewBatcher 返回Batcher实例.

func (*Batcher) Close

func (b *Batcher) Close()

Close 停止运行批处理器.

func (*Batcher) Put

func (b *Batcher) Put(key string, job interface{}) error

Put 将待处理的事务加入批处理器.

func (*Batcher) Start

func (b *Batcher) Start(doBatch DoBatch)

Start 开始运行批处理器.

func (*Batcher) Stat

func (b *Batcher) Stat() int64

Stat 返回批处理器已经处理的批次.

type BatcherGroup

type BatcherGroup []*Batcher

BatcherGroup 一组匹处理器

func NewBatcherGroup

func NewBatcherGroup(cfg *BatcherGroupCfg) BatcherGroup

NewBatcherGroup 返回BatcherGroup实例.

func (BatcherGroup) Close

func (bg BatcherGroup) Close()

Close 停止运行所有的批处理器.

func (BatcherGroup) Put

func (bg BatcherGroup) Put(key string, job interface{}) error

Put 将待处理的事务加入批处理器组, 并按照关键词分发给指定的批处理器处理.

func (BatcherGroup) Start

func (bg BatcherGroup) Start(doBatch DoBatch)

Start 开始运行所有的批处理器.

func (BatcherGroup) Stat

func (bg BatcherGroup) Stat()

Stat 显示所有批处理器的工作状态.

type BatcherGroupCfg

type BatcherGroupCfg struct {
	BatcherNum         int
	BatcherConcurrency int
	MaxBatchSize       int
	FlushTimeMs        int
	SourceQueueSize    int
}

BatcherGroupCfg 匹处理器组配置

type DoBatch

type DoBatch func(*BatchItem) error

DoBatch 匹处理器处理事务的处理函数

type SourceItem

type SourceItem struct {
	// contains filtered or unexported fields
}

SourceItem 待处理的事务单元