pipeline

Documentation

Overview

Package for building an execution pipeline and executing it.

Index

• type Pipeline
  • func ImportPipeline(filename string) (Pipeline, error)
  • func NewPipeline(name string, r *r.Server) Pipeline
  • func (p *Pipeline) AddStage(s Stage)
  • func (p *Pipeline) ExecuteStage(stage *Stage) (string, error)
  • func (p *Pipeline) GetResult(format string) string
  • func (p *Pipeline) Print()
  • func (p *Pipeline) Results(resultType string) ([]byte, error)
  • func (p *Pipeline) Run() ([]*Result, error)
  • func (p *Pipeline) RunSequential() ([]*Result, error)
  • func (p *Pipeline) Save() error
• type Result
• type Stage
  • func NewStage(name, function, pkg string, argnames, args []string) Stage
  • func (s *Stage) GetArguments() string
  • func (s Stage) GetDependencies() []string
  • func (s *Stage) Print()
  • func (s *Stage) ReplaceArg(oldarg string, newarg string)

Types

type Pipeline

type Pipeline struct {
	Name    string    "name,omitempty"
	RServer *r.Server "r-server,omitempty"
	Stages  []*Stage  "stages,omitempty"
}

The pipeline. A name, reference to an opencpu server and list of pipeline stages.

func ImportPipeline

func ImportPipeline(filename string) (Pipeline, error)

func NewPipeline

func NewPipeline(name string, r *r.Server) Pipeline

Set up new pipeline with the given name and reference to gopencpu server

func (*Pipeline) AddStage

func (p *Pipeline) AddStage(s Stage)

Appends a pipeline stage to to the end of the pipeline.

func (*Pipeline) ExecuteStage

func (p *Pipeline) ExecuteStage(stage *Stage) (string, error)

Executes a pipeline stage.

func (*Pipeline) GetResult

func (p *Pipeline) GetResult(format string) string

Get the final result from the last stage in a pipeline.

func (*Pipeline) Print

func (p *Pipeline) Print()

Print a pipeline stage.

func (*Pipeline) Results

func (p *Pipeline) Results(resultType string) ([]byte, error)

Return pipeline results, i.e. result of last pipeline stage.

func (*Pipeline) Run

func (p *Pipeline) Run() ([]*Result, error)

Exectues a pipeline. Uses a go routine per pipeline stage making it possible to execute multiple stages simultaneously. Returns a list of results, one per stage.

func (*Pipeline) RunSequential

func (p *Pipeline) RunSequential() ([]*Result, error)

func (*Pipeline) Save

func (p *Pipeline) Save() error

Stores a pipeline as a pipeline description yaml file.

type Result

type Result struct {
	Key   string
	Error error
	// contains filtered or unexported fields
}

For storing the result of a pipeline stage. Key can be used to retrieve plots/results later using the gopencpu package.

type Stage

type Stage struct {
	Name      string            "name,omitempty"
	Package   string            "package,omitempty"
	Function  string            "function,omitempty"
	Arguments map[string]string "arguments,omitempty"
	Depends   []string          "depends,omitempty"
	Session   string            "session,omitempty"
	Output    string            "output,omitempty"
}

A pipeline stage. Name: A unique name for each stage. Function: Function name, e.g. plot. Package: Which statistical package the function comes from, e.g. graphics for plot(). Arguments: A name:value map for arguments to the function. Depends: List of stages it depends on. Session: Reference to the OpenCPU session used to compute the results from this stage. Output: String for storing output from pipeline stage execution.

func NewStage

func NewStage(name, function, pkg string, argnames, args []string) Stage

Creates a new pipeline stage. Argnames is an array of argument names, args are the values for each of the arguments. Assumes that argnames and args are of the same length.

func (*Stage) GetArguments

func (s *Stage) GetArguments() string

Create a string of arguments for usage by the Call method in the gopencpu package

func (Stage) GetDependencies

func (s Stage) GetDependencies() []string

Returns all dependencies for a stage.

func (*Stage) Print

func (s *Stage) Print()

Print a pipeline stage.

func (*Stage) ReplaceArg

func (s *Stage) ReplaceArg(oldarg string, newarg string)

Replace any "from:stage-name" argument values into opencpu references