gotask

README

sysvinit like task runner in go

Synopsis

must := func(err error) {
    if err != nil {
        log.Fatal(err)
    }
}
r := gotask.NewRunner()
must(r.QuickAdd("a", initAFunc))
must(r.Add(gotask.New("b", initBFunc, []string{"a"}))

if err := r.RunSync(); err != nil {
    log.Fatal(err)
}

Strange but fairly complex example

In this example, it starts pgsql and redis via systemd, warm-up the cache before start the service.

It uses routines package

var runner = gotask.NewRunner()

func must(e error) {
    if e != nil {
        log.Fatal(e)
    }
}

func systemdStart(srv string) (err error) {
    if err := exec.Command("systemctl", "start", srv).Run(); err != nil {
        return
    }
    
    // check 3 times every 10 sec
    err = routines.TryAtMost(3, routines.RunAtLeast(
        10 * time.Second, 
        func() error { return exec.Command("systemctl", "status", srv).Run() },
    ))
        
    return
}

func init() {
    must(runner.QuickAdd("pgsql", func() error { return systemdStart("postgresql") }))
    must(runner.QuickAdd("redis", func() error { return systemdStart("redis") }))
}

func fillCache() error {
    // TBD
    return nil
}

func init() {
    must(runner.Add("warm-up", fillCache, []string{
        "prepareConn",
    }))
}

var dbConn *sql.DB
var cacheConn *redis.Client

func prepareConn() error {
    // TBD: setup dbConn and cacheConn
    return nil
}

func init() {
    must(runner.Add("prepareConn", prepareConn, []string{
        "pgsql", "redis",
    }))
}

func main() {
    if err := runner.Run(2); err != nil {
        log.Fatal(runner.Errors())
    }
    
    log.Print(myservicepkg.Start())
}

Split tasks into multiple groups

By using Only(), Without(), With() and Skip(), it is possible to split tasks in to several groups.

r := NewRunner()
r.QuickAdd("a", funcA)
r.Add(New("b", funcB, "a")
r.QuickAdd("c", funcC)
r.Add(New("d", funcD, "c", "a")

groupB := r.With("b")
groupB.Run() // runs task a and b

// skip tasks ran in groupB to prevent running same task twice 
r.Skip(groupb.Tasks()...)
r.Run() // runs task c and d

License

Copyright Chung-Ping Jen ronmi.ren@gmail.com 2021-

MPL v2.0

Documentation

Overview

Package gotask provides a tool to run tasks

See Runner and Task for detailed info

Index

• Variables
• type ErrDuplicated
  • func (e ErrDuplicated) Error() string
• type ErrMissing
  • func (e ErrMissing) Error() string
• type Runner
  • func NewRunner() (ret Runner)
• type Task
  • func New(name string, f func() error, depends []string) (ret Task)

Examples

• Runner

Variables

var (
	// ErrCyclic indicates cyclic dependencies are detected
	ErrCyclic = errors.New("cyclic dependencies detected")
	// ErrParsed indicates Runner.Add() is called after Runner.Parse()
	ErrParsed = errors.New("cannot add task after parsed")
)

Types

type ErrDuplicated

type ErrDuplicated string

ErrDuplicated indicates the task has been registered

func (ErrDuplicated) Error

func (e ErrDuplicated) Error() string

type ErrMissing

type ErrMissing string

ErrMissing indicates a dependency is missing

func (ErrMissing) Error

func (e ErrMissing) Error() string

type Runner

type Runner interface {
	// Add a task
	//
	// This might returns following errors:
	//
	//   - ErrDuplicated: a task with same name has been registered
	//   - ErrParsed: calling Add() after Parse()
	//   - ErrCyclic: the task depends itself
	Add(Task) error
	// Helper to add a task without dependency
	QuickAdd(name string, f func() error) error
	// Reset internal state, make it possible to Add() after Parse()
	Reset()
	// Check dependencies and sort tasks. It might return following errors:
	//
	//   - ErrMissing: the depdency is missing
	//   - ErrCyclic: there's a loop in dependency graph
	Parse() error
	// Run tasks one-by-one (synchronously). It almost identical to Run(1), but
	// frees you from race condition.
	//
	// As scheduler is implemented in  map, the order of task is unspecified,
	// only dependencies are ensured.
	//
	// It calls Parse() if needed, so same error applies in addition to errors
	// returned from task.
	//
	// While RunSync() has less overhead compares to Run(). It is suggested only
	// if your tasks are not time-consuming.
	RunSync() error
	// Run the tasks in n goroutines. Set n to 1 will use only 1 groutine, which
	// is roughly same as synchronized executing. Set n to 0 = runtime.NumCPU()
	//
	// As scheduler is implemented in  map, the order of task is unspecified,
	// only dependencies are ensured.
	//
	// It calls Parse() if needed, so same error applies in addition to errors
	// returned from task.
	//
	// Since tasks are run in separated goroutines, you have to take care race
	// conditions yourself.
	//
	// Tasks are grouped by their "depth", tasks in next group will not run
	// before all tasks in current group done. For example
	//
	//    task  | depends | depth
	//   -------+---------+-------------------
	//    a     |         | 0
	//    b     |         | 0
	//    c     | a       | 1 (a+1)
	//    d     | b, c    | 2 (max(b, c) + 1)
	//
	// In this case, c will not run until both a and b are done, even if you set
	// n to 2.
	//
	// To prevent race-conditions, Run() has much more overhead compares to
	// RunSync(). However, RunSync() barely beats Run() in real world scenario
	// as most tasks are time-consuming.
	Run(n uint8) error
	// Gets known errors. Multiple errors might occur concurrently in Run(), but
	// only one is returned. If you need to know all errors, just use this.
	Errors() []error
	// Get the name of known tasks
	Tasks() []string
	// Mark some tasks to be skipped.
	//
	// Skipped tasks will not be executed when calling Run() or RunSync(), just
	// set the task as "successfully executed".
	Skip(name ...string)
	// Only creates a new Runner which contains only specified tasks.
	//
	// Non-exist tasks are ignored silently. Say you have a Runner contains four
	// tasks: a, b, c and d. Calling Only("a", "b", "f") returns a new Runner
	// contains a and b.
	Only(name ...string) Runner
	// Without creates a new Runner without specified tasks.
	//
	// Non-exist tasks are ignored silently. Say you have a Runner contains four
	// tasks: a, b, c and d. Calling Without("a", "b", "f") returns a new Runner
	// contains c and d.
	Without(name ...string) Runner
	// With creates a new Runner contains specified tasks and their deps.
	//
	// Non-exist tasks are ignored silently. Say you have a Runner contains four
	// tasks: a, b, c (depends b) and d. Calling With("a", "c", "f") returns a
	// new Runner contains a, b and c.
	With(name ...string) Runner
}

Runner manages task dependencies and runs the tasks

Currently it caches all dependencies for each task in map while parsing, which cost you some extra memory space (should be few KBs in real world).

Runner is not thread-safe, you MUST NOT share same instance amoung multiple goroutines.

Example

r := NewRunner()
r.Add(New("a", func() error {
	fmt.Println("a")
	return nil
}, []string{"b"}))
r.Add(New("b", func() error {
	fmt.Println("b")
	return nil
}, []string{"c"}))
r.Add(New("c", func() error {
	fmt.Println("c")
	return nil
}, []string{}))

r.Run(3)

Output:
c
b
a

func NewRunner

func NewRunner() (ret Runner)

NewRunner creates a Runner instance

type Task

type Task interface {
	// task name, also used as dependency specification (CASE-SENSITIVE)
	Name() string
	// actually run the task
	Run() error
	// returns dependencies (CASE-SENSITIVE)
	Depends() []string
}

Task represents a task to run

func New

func New(name string, f func() error, depends []string) (ret Task)

New is a helper to create a Task instance