parallel_chunked_flow

README

parallel-chunked-flow

Package parallel chunked flow provides the ability to craete parallel pipeline for sequential data.

License

Licensed under the MIT License

Authors

Copyright(c) 2020 Fred Chien cfsghost@gmail.com

Documentation

Index

• type Chunk
  • func NewChunk(size int) *Chunk
  • func (chunk *Chunk) Initialize()
• type Options
  • func NewOptions() *Options
• type ParallelChunkedFlow
  • func NewParallelChunkedFlow(options *Options) *ParallelChunkedFlow
  • func (pcf *ParallelChunkedFlow) Output() chan interface{}
  • func (pcf *ParallelChunkedFlow) Push(data interface{}) error

Types

type Chunk

type Chunk struct {
	Handler func(interface{}) interface{}
	// contains filtered or unexported fields
}

func NewChunk

func NewChunk(size int) *Chunk

func (*Chunk) Initialize

func (chunk *Chunk) Initialize()

type Options

type Options struct {
	BufferSize int
	ChunkSize  int
	ChunkCount int
	Handler    func(interface{}) interface{}
}

Options represent all of the available options when creating a parallel chunked flow.

func NewOptions

func NewOptions() *Options

NewOptions creates a Options object.

type ParallelChunkedFlow

type ParallelChunkedFlow struct {
	// contains filtered or unexported fields
}

func NewParallelChunkedFlow

func NewParallelChunkedFlow(options *Options) *ParallelChunkedFlow

NewParallelChunckedFlow creates a new parallel chunked flow

func (*ParallelChunkedFlow) Output

func (pcf *ParallelChunkedFlow) Output() chan interface{}

Output will return a channel for receive proccessed data from flow

func (*ParallelChunkedFlow) Push

func (pcf *ParallelChunkedFlow) Push(data interface{}) error

Push will put data to the flow