disksort

Documentation

Overview

Package disksort contains a modified direct runner for running single-bundle pipelines in the current process. Uses a disksort to implement CoGBK operations.

Index

• func Compile(edges []*graph.MultiEdge) (*exec.Plan, error)
• func Execute(ctx context.Context, p *beam.Pipeline) (beam.PipelineResult, error)
• type CoGBK
  • func (n *CoGBK) Down(ctx context.Context) error
  • func (n *CoGBK) FinishBundle(ctx context.Context) error
  • func (n *CoGBK) ID() exec.UnitID
  • func (n *CoGBK) ProcessElement(ctx context.Context, elm *exec.FullValue, _ ...exec.ReStream) error
  • func (n *CoGBK) StartBundle(ctx context.Context, id string, data exec.DataContext) error
  • func (n *CoGBK) String() string
  • func (n *CoGBK) Up(ctx context.Context) error

Functions

func Compile

func Compile(edges []*graph.MultiEdge) (*exec.Plan, error)

Compile translates a pipeline to a multi-bundle execution plan.

func Execute

func Execute(ctx context.Context, p *beam.Pipeline) (beam.PipelineResult, error)

Execute runs the pipeline in-process.

Types

type CoGBK

type CoGBK struct {
	UID  exec.UnitID
	Edge *graph.MultiEdge
	Out  exec.Node
	// contains filtered or unexported fields
}

CoGBK buffers all input and continues on FinishBundle. Use with small single-bundle data only.

func (*CoGBK) Down

func (n *CoGBK) Down(ctx context.Context) error

func (*CoGBK) FinishBundle

func (n *CoGBK) FinishBundle(ctx context.Context) error

func (*CoGBK) ID

func (n *CoGBK) ID() exec.UnitID

func (*CoGBK) ProcessElement

func (n *CoGBK) ProcessElement(ctx context.Context, elm *exec.FullValue, _ ...exec.ReStream) error

func (*CoGBK) StartBundle

func (n *CoGBK) StartBundle(ctx context.Context, id string, data exec.DataContext) error

func (*CoGBK) String

func (n *CoGBK) String() string

func (*CoGBK) Up

func (n *CoGBK) Up(ctx context.Context) error