flow

Documentation

Overview

Package defines a Flow type that represents an entire data pipeline.

Index

• func Run(ctx context.Context, fn FactoryFn) error
• func RunWithConfig[C any](ctx context.Context, fn FactoryFnWithConfig[C], cfgPath string) error
• type Factory
  • func NewFactory(fn FactoryFn) Factory
  • func NewFactoryWithConfig[C any](fn FactoryFnWithConfig[C], cfgPath string) Factory
• type FactoryFn
• type FactoryFnWithConfig
• type Flow
  • func FromTask(task task.Task[struct{}, struct{}]) *Flow
  • func (f *Flow) Run(ctx context.Context) error

Functions

func Run

func Run(ctx context.Context, fn FactoryFn) error

Build a Flow with the passed FactoryFn and run it.

This is a shorthand of creating a Factory with NewFactory, building a Flow with the Factory, and running the built Flow.

func RunWithConfig

func RunWithConfig[C any](ctx context.Context, fn FactoryFnWithConfig[C], cfgPath string) error

Build a Flow with the passed FactoryFnWithConfig and run it.

This is a shorthand of creating a Factory with NewFactoryWithConfig, building a Flow with the Factory, and running the built Flow.

Types

type Factory

type Factory interface {
	Build() (*Flow, error)
}

An object to build a Flow.

func NewFactory

func NewFactory(fn FactoryFn) Factory

Create a Factory with a FactoryFn.

func NewFactoryWithConfig

func NewFactoryWithConfig[C any](fn FactoryFnWithConfig[C], cfgPath string) Factory

Create a Factory with a FactoryFnWithConfig. The passed cfgPath is read when building a Flow, and passed to the FactoryFnWithConfig.

type FactoryFn

type FactoryFn func() (*Flow, error)

A function that defines how to build a Flow.

type FactoryFnWithConfig

type FactoryFnWithConfig[C any] func(cfg *C) (*Flow, error)

A function that defines how to build a Flow with a configuration struct.

type Flow

type Flow struct {
	// contains filtered or unexported fields
}

A type that defines an entire data pipeline.

The behavior of a Flow is defined by a task that doesn't have any input and output. Such a task is typically built as a pipeline that starts with a Source and ends with a Sink.

func FromTask

func FromTask(task task.Task[struct{}, struct{}]) *Flow

Create a Flow with a task that has neither input nor output.

func (*Flow) Run

func (f *Flow) Run(ctx context.Context) error

Run this Flow.