taskflow

package module
Version: v0.2.0 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Mar 14, 2021 License: Unlicense Imports: 19 Imported by: 0

README

taskflow

Create build pipelines in Go

GitHub Release Go Reference go.mod Build Status Go Report Card codecov Mentioned in Awesome Go Gitpod ready-to-code

This package aims to simplify the creation of build pipelines in Go instead of using scripts or Make.

taskflow API is mainly inspired by the testing, http and flag packages.

Check Go Build Pipeline Demo to compare taskflow with Make and Mage.

Star this repository if you find it valuable and worth maintaining.

Table of Contents:

Example

Create a file in your project build/build.go.

Copy and paste the content from below.

package main

import "github.com/pellared/taskflow"

func main() {
	flow := taskflow.New()

	fmt := flow.MustRegister(taskFmt())
	test := flow.MustRegister(taskTest())

	flow.MustRegister(taskflow.Task{
		Name:        "all",
		Description: "build pipeline",
		Dependencies: taskflow.Deps{
			fmt,
			test,
		},
	})

	flow.Main()
}

func taskFmt() taskflow.Task {
	return taskflow.Task{
		Name:        "fmt",
		Description: "go fmt",
		Command:     taskflow.Exec("go", "fmt", "./..."),
	}
}

func taskTest() taskflow.Task {
	return taskflow.Task{
		Name:        "test",
		Description: "go test with race detector and code covarage",
		Command: func(tf *taskflow.TF) {
			if err := tf.Cmd("go", "test", "-race", "-covermode=atomic", "-coverprofile=coverage.out", "./...").Run(); err != nil {
				tf.Errorf("go test: %v", err)
			}
			if err := tf.Cmd("go", "tool", "cover", "-html=coverage.out", "-o", "coverage.html").Run(); err != nil {
				tf.Errorf("go tool cover: %v", err)
			}	
		},
	}
}

Sample usage:

$ go run ./build -h
Usage: [flag(s)] [key=val] task(s)
Flags:
  -v    Verbose output: log all tasks as they are run. Also print all text from Log and Logf calls even if the task succeeds.
Tasks:
  all     build pipeline
  fmt     go fmt
  test    go test with race detector and code covarage

$ go run ./build all
ok     0.453s

$ go run ./build -v all
===== TASK  fmt
Exec: go fmt ./...
----- PASS: fmt (0.06s)
===== TASK  test
Exec: go test -race -covermode=atomic -coverprofile=coverage.out ./...
?       github.com/pellared/taskflow/example    [no test files]
Exec: go tool cover -html=coverage.out -o coverage.html
----- PASS: test (0.11s)
ok      0.176s

Tired of writing go run ./build each time? Just add an alias to your shell. For example, add the line below to ~/.bash_aliases:

alias gake='go run ./build'

Additionally, take a look at taskflow-example and this repository's own build pipeline script - build/build.go.

Features

Task registration

The registered tasks are required to have a non-empty name. For future compatibility, it is strongly suggested to use only the following characters:

  • letters (a-z and A-Z)
  • digits (0-9)
  • underscore (_)
  • hyphens (-)

Do not use the equals sign (=) as it is used for assigning parameters.

A task with a given name can be only registered once.

A task without description is not listed in taskflow's CLI usage.

Task command

Task command is a function which is executed when a task is executed. It is not required to to set a command. Not having a command is very handy when registering "pipelines".

Task dependencies

During task registration it is possible to add a dependency to an already registered task. When taskflow is processed, it makes sure that the dependency is executed before the current task is run. Take note that each task will be executed at most once.

Helpers for running programs

Use func Exec(name string, args ...string) func(*TF) to create a task's command which only runs a single program.

Use func (tf *TF) Cmd(name string, args ...string) *exec.Cmd if within a task's command function when you want to execute more programs or you need more granular control.

Verbose mode

Enable verbose output using the -v CLI flag. It works similar to go test -v. When enabled, the whole output will be streamed. If disabled, only logs from failed task are send to the output.

Use func (*TF) Verbose to check if verbose mode was set within the task's command.

Default task

Default task can be assigned via DefaultTask field in type Taskflow.

When default task is set, then it is run if no task is provied via CLI.

Parameters

The parameters can be set via CLI using the key=val syntax after CLI flags. For example, go run ./build -v ci=true all would run the all task with ci parameter set to "true" in verbose mode.

Default values can be assigned via Params field in type Taskflow.

The task's command can get the parameters using func (*TF) Params.

Task runner

You can use type Runner to execute a single command.

It may be handy during development of a new task, when debugging some issue or if you want to have a test suite for reusable commands.

Supported Go versions

Minimal required Go version is 1.10.

FAQ

Why not use Make?

While Make is currently the de facto standard, it has some pitfalls:

  • Requires to learn Make, which is not so easy.
  • It is hard to develop a Makefile which is truly cross-platform.
  • Debugging and testing Make targets is not fun.

However, if you (and your team) know Make and are happy with it, do not change it.

Make is very powerful and a lot of stuff can be made a lot faster, if you know how to use it.

taskflow is intended to be simpler and easier to learn, while still being able to handle most use cases.

Why not use Mage?

taskflow is intended to be an alternative to Mage.

Mage is a framework/tool which magically discovers the targets from magefiles.

taskflow takes a different approach as it is a regular Go library (package).

This results in following benefits:

  • It is easy to debug. Like a regular Go application.
  • Tasks and helpers are testable. See exec_test.go.
  • Reusing tasks is easy and readable. Just create a function which registers common tasks. Mage does it in a hacky way.
  • API similar to testing so it is possible to use e.g. testify for asserting.

To sum up, taskflow is not magical. Write regular Go code. No build tags or special names for functions.

Why not use Task?

While Task is simpler and easier to use than Make it still has some problems:

  • Requires to learn Task's YAML sturcture and the minimalistic, cross-platform interpreter which it uses.
  • Debugging and testing tasks is not fun.
  • Harder to make some reusable tasks.
  • Requires to "install" the tool. taskflow leverages go run and Go Modules so that you can be sure that everyone uses the same version of taskflow.
Why not use Bazel?

Bazel is a very sophisticated tool which is created to efficiently handle complex and long-running build pipelines. It requires the build target inputs and outputs to be fully specified.

taskflow is just a simple library that is mainly supposed to create a build pipeline consisting of commands like go vet, go test, go build. However, take notice that taskflow is a library. Nothing prevents you from, for example, using Mage's target package to make your build pipeline more efficient.

Contributing

I am open to any feedback and contribution.

Use Discussions or write to me: Robert Pajak @ Gophers Slack.

You can also create an issue or a pull request.

Documentation

Overview

Package taskflow helps implementing build automation. It is intended to be used in concert with the "go run" command, to run a program which implements the build pipeline (called taskflow). A taskflow consists of a set of registered tasks. A task has a name, can have a defined command, which is a function with signature 

func (*taskflow.TF)

and can have dependencies (already defined tasks).

When the taskflow is executed for given tasks, then the tasks' commands are run in the order defined by their dependencies. The task's dependencies are run in a recusrive manner, however each is going to be run at most once.

The taskflow is interupted in case a command fails. Within these functions, use the Error, Fail or related methods to signal failure.

Example 
package main

import (
	"github.com/pellared/taskflow"
)

func main() {
	flow := taskflow.New()

	task1 := flow.MustRegister(taskflow.Task{
		Name:        "task-1",
		Description: "Print Go version",
		Command:     taskflow.Exec("go", "version"),
	})

	task2 := flow.MustRegister(taskflow.Task{
		Name: "task-2",
		Command: func(tf *taskflow.TF) {
			tf.Skip("skipping")
		},
	})

	task3 := flow.MustRegister(taskflow.Task{
		Name: "task-3",
		Command: func(tf *taskflow.TF) {
			tf.Error("hello from", tf.Name())
			tf.Log("this will be printed")
		},
	})

	flow.MustRegister(taskflow.Task{
		Name:         "all",
		Dependencies: taskflow.Deps{task1, task2, task3},
	})

	flow.Main()
}

Output:

Index

Examples

Constants

View Source 
const (
	// CodePass indicates that taskflow passed.
	CodePass = 0
	// CodeFailure indicates that taskflow failed.
	CodeFailure = 1
	// CodeInvalidArgs indicates that got invalid input.
	CodeInvalidArgs = 2
)

Variables

View Source 
var DefaultOutput io.Writer = os.Stdout

DefaultOutput is the default output used by Taskflow if it is not set.

Functions

func Exec 

func Exec(name string, args ...string) func(*TF)

Exec returns a command that will run the named program with the given arguments. The command will pass only if the program if the program runs, has no problems copying stdin, stdout, and stderr, and exits with a zero exit status.

Types

type Deps 

type Deps []RegisteredTask

Deps represents an collection of registered Tasks.

type ParamError 

type ParamError struct {
	Key string // the parameter's key
	Err error  // the reason the conversion failure, e.g. *strconv.NumError
}

ParamError records an error during parameter conversion.

func (*ParamError) Error 

func (e *ParamError) Error() string

func (*ParamError) Unwrap 

func (e *ParamError) Unwrap() error

Unwrap unpacks the wrapped error.

type Params 

type Params map[string]string

Params represents Taskflow parameters used within Taskflow. The default values set in the struct are overridden in Run method.

func (Params) SetBool 

func (p Params) SetBool(key string, v bool)

SetBool sets default value for given parameter using strconv.FormatBool.

func (Params) SetDate 

func (p Params) SetDate(key string, v time.Time, layout string)

SetDate sets default value for given parameter using time.Time.Format.

func (Params) SetDuration 

func (p Params) SetDuration(key string, v time.Duration)

SetDuration sets default value for given parameter using time.Duration.String.

func (Params) SetInt 

func (p Params) SetInt(key string, v int)

SetInt sets default value for given parameter using strconv.Itoa.

func (Params) SetJSON 

func (p Params) SetJSON(key string, v interface{}) error

SetJSON sets default value for given parameter using json.Marshal. If v is nil SetJSON returns an error. An error is also returned if marshaling failed.

func (Params) SetText 

func (p Params) SetText(key string, v encoding.TextMarshaler) error

SetText sets default value for given parameter using value's MarshalText method. If v is nil SetText returns an error. An error is also returned if marshaling failed.

type RegisteredTask 

type RegisteredTask struct {
	// contains filtered or unexported fields
}

RegisteredTask represents a task that has been registered to a Taskflow. It can be used as a dependency for another Task.

type RunResult 

type RunResult struct {
	// contains filtered or unexported fields
}

RunResult contains the results of a Command run.

func (RunResult) Duration 

func (r RunResult) Duration() time.Duration

Duration returns the durations of the Command.

func (RunResult) Failed 

func (r RunResult) Failed() bool

Failed returns true if a command failed. Failure can be caused by invocation of Error, Fail or related methods or a panic.

func (RunResult) Passed 

func (r RunResult) Passed() bool

Passed true if a command passed. It means that it has not failed, nor skipped.

func (RunResult) Skipped 

func (r RunResult) Skipped() bool

Skipped returns true if a command was skipped. Skip is casused by invocation of Skip or related methods.

type Runner 

type Runner struct {
	Ctx      context.Context
	TaskName string
	Output   io.Writer
	Params   Params
	Verbose  bool
}

Runner is used to run a Command.

func (Runner) Run 

func (r Runner) Run(command func(tf *TF)) RunResult

Run runs the command.

type TF 

type TF struct {
	// contains filtered or unexported fields
}

TF is a type passed to Task's Command function to manage task state.

A Task ends when its Command function returns or calls any of the methods FailNow, Fatal, Fatalf, SkipNow, Skip, or Skipf.

All methods must be called only from the goroutine running the Command function.

func (*TF) Cmd 

func (tf *TF) Cmd(name string, args ...string) *exec.Cmd

Cmd is like exec.Command, but it assignes tf's context and assigns Stdout and Stderr to tf's output.

func (*TF) Context 

func (tf *TF) Context() context.Context

Context returns the taskflows' run context.

func (*TF) Error 

func (tf *TF) Error(args ...interface{})

Error is equivalent to Log followed by Fail.

func (*TF) Errorf 

func (tf *TF) Errorf(format string, args ...interface{})

Errorf is equivalent to Logf followed by Fail.

func (*TF) Fail 

func (tf *TF) Fail()

Fail marks the function as having failed but continues execution.

func (*TF) FailNow 

func (tf *TF) FailNow()

FailNow marks the function as having failed and stops its execution by calling runtime.Goexit (which then runs all deferred calls in the current goroutine). It finishes the whole taskflow.

func (*TF) Failed 

func (tf *TF) Failed() bool

Failed reports whether the function has failed.

func (*TF) Fatal 

func (tf *TF) Fatal(args ...interface{})

Fatal is equivalent to Log followed by FailNow.

func (*TF) Fatalf 

func (tf *TF) Fatalf(format string, args ...interface{})

Fatalf is equivalent to Logf followed by FailNow.

func (*TF) Log 

func (tf *TF) Log(args ...interface{})

Log formats its arguments using default formatting, analogous to Println, and prints the text to Output. A final newline is added. The text will be printed only if the task fails or taskflow is run in Verbose mode.

func (*TF) Logf 

func (tf *TF) Logf(format string, args ...interface{})

Logf formats its arguments according to the format, analogous to Printf, and prints the text to Output. A final newline is added. The text will be printed only if the task fails or taskflow is run in Verbose mode.

func (*TF) Name 

func (tf *TF) Name() string

Name returns the name of the running task.

func (*TF) Output 

func (tf *TF) Output() io.Writer

Output returns the io.Writer used to print output.

func (*TF) Params 

func (tf *TF) Params() TFParams

Params returns the key-value parameters.

func (*TF) Skip 

func (tf *TF) Skip(args ...interface{})

Skip is equivalent to Log followed by SkipNow.

func (*TF) SkipNow 

func (tf *TF) SkipNow()

SkipNow marks the task as having been skipped and stops its execution by calling runtime.Goexit (which then runs all deferred calls in the current goroutine). If a test fails (see Error, Errorf, Fail) and is then skipped, it is still considered to have failed. Taskflow will continue at the next task.

func (*TF) Skipf 

func (tf *TF) Skipf(format string, args ...interface{})

Skipf is equivalent to Logf followed by SkipNow.

func (*TF) Skipped 

func (tf *TF) Skipped() bool

Skipped reports whether the task was skipped.

func (*TF) Verbose 

func (tf *TF) Verbose() bool

Verbose returns if verbose mode was set.

type TFParams 

type TFParams struct {
	// contains filtered or unexported fields
}

TFParams represents Taskflow parameters accessible in task's command.

func (TFParams) Bool 

func (p TFParams) Bool(key string) bool

Bool converts the parameter to bool. It fails the task if the conversion failed. False is returned if the parameter was not set. It accepts 1, t, T, TRUE, true, True, 0, f, F, FALSE, false, False.

func (TFParams) Date 

func (p TFParams) Date(key string, layout string) time.Time

Date converts the parameter to time.Time using time.Parse. It fails the task if the conversion failed. Zero value is returned if the parameter was not set. The layout defines the format by showing how the reference time, defined to be 

Mon Jan 2 15:04:05 -0700 MST 2006

would be interpreted if it were the value; it serves as an example of the input format. The same interpretation will then be made to the input string.

func (TFParams) Duration 

func (p TFParams) Duration(key string) time.Duration

Duration converts the parameter to time.Duration using time.ParseDuration. It fails the task if the conversion failed. 0 is returned if the parameter was not set. A duration string is a possibly signed sequence of decimal numbers, each with optional fraction and a unit suffix, such as "300ms", "-1.5h" or "2h45m". Valid time units are "ns", "us" (or "µs"), "ms", "s", "m", "h".

func (TFParams) Float64 

func (p TFParams) Float64(key string) float64

Float64 converts the parameter to float64 accepting decimal and hexadecimal floating-point number syntax. It fails the task if the conversion failed. 0 is returned if the parameter was not set.

func (TFParams) Int 

func (p TFParams) Int(key string) int

Int converts the parameter to int using the Go syntax for integer literals. It fails the task if the conversion failed. 0 is returned if the parameter was not set.

func (TFParams) ParseJSON 

func (p TFParams) ParseJSON(key string, v interface{})

ParseJSON parses the parameter and stores the result in the value pointed to by v by using json.Unmarshal. It fails the task if the conversion failed or v is nil or not a pointer.

func (TFParams) ParseText 

func (p TFParams) ParseText(key string, v encoding.TextUnmarshaler)

ParseText parses the parameter and stores the result in the value pointed to by v by using its UnmarshalText method. It fails the task if the conversion failed or v is nil or not a pointer.

func (TFParams) String 

func (p TFParams) String(key string) string

String returns the parameter as a string.

type Task 

type Task struct {
	Name         string
	Description  string
	Command      func(tf *TF)
	Dependencies Deps
}

Task represents a named task that can be registered. It can consist of a command (function that will be called when task is run) and dependencies (tasks which has to be run before this one).

type Taskflow 

type Taskflow struct {
	Output      io.Writer      // output where text is printed; os.Stdout by default
	Params      Params         // default parameters' values
	Verbose     bool           // when enabled, then the whole output will be always streamed
	DefaultTask RegisteredTask // task which is run when non is explicitly provided
	// contains filtered or unexported fields
}

Taskflow is the root type of the package. Use Register methods to register all tasks and Run or Main method to execute provided tasks.

func New 

func New() *Taskflow

New return an instance of Taskflow with initialized fields.

func (*Taskflow) Main 

func (f *Taskflow) Main()

Main parses the command-line arguments and runs the provided tasks. The usage is printed when invalid arguments are passed.

func (*Taskflow) MustRegister 

func (f *Taskflow) MustRegister(task Task) RegisteredTask

MustRegister registers the task. It panics in case of any error.

func (*Taskflow) Register 

func (f *Taskflow) Register(task Task) (RegisteredTask, error)

Register registers the task.

func (*Taskflow) Run 

func (f *Taskflow) Run(ctx context.Context, args ...string) int

Run runs provided tasks and all their dependencies. Each task is executed at most once.

Source Files

View all

Directories

Path Synopsis

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL