README
¶
Overview
"Go task group" or "Go task graph". Runs a set of functions as a graph with coordination and deduplication.
Good for CLI task orchestration, serving as a Go-based alternative to Make (see #comparison) and a simple, flexible replacement for Mage (see #comparison). May be useful for non-CLI applications.
API documentation: https://pkg.go.dev/github.com/mitranim/gtg.
Main concepts:
type Task interface {
context.Context
TaskGroup
}
type TaskGroup interface {
Task(TaskFunc) Task
}
type TaskFunc func(Task) error
TOC
Usage
(For CLI, see #below.)
This example demonstrates a diamond-shaped task graph:
A
/ \
v v
B C
\ /
v
D
-
Task A waits on tasks B and C.
-
Both B and C wait on D.
-
C allows D to fail and logs the error, while B requires D to succeed.
-
Each function in the group is called exactly once.
import g "github.com/mitranim/gtg"
func main() {
g.MustRun(context.Background(), A)
}
func A(task g.Task) error {
// Allow B and C to run concurrently, wait on both.
g.MustWait(task, g.Par(B, C))
// Any logic here.
return nil
}
func B(task g.Task) error {
return g.Wait(task, D)
}
func C(task g.Task) error {
// Choose one line. Either of them will work.
g.Log(g.Wait(task, D))
g.MustWait(task, g.Opt(D))
// Any logic here.
return nil
}
func D(_ g.Task) error {
return fmt.Errorf("arbitrary failure")
}
CLI Usage
Reusing the A B C D task definitions from the example above:
import g "github.com/mitranim/gtg"
func main() {
g.MustRunCmd(A, B, C, D)
}
Then from the command line:
# Print available tasks.
go run .
# Run a specific task.
go run . a
Comparisons
Comparison with "context"
- Gtg is an extension of
context, adding support for running tasks as a graph, with mutual coordination and deduplication.
Comparison with Make
-
Make is a CLI. Gtg is a library with CLI convenience features.
-
Make runs shell scripts, Gtg runs Go. Both are good for different things.
-
Go scripts are more portable than shell scripts. (My camel-breaking straw was Bash incompatibilities between Ubuntu and MacOS.)
Comparison with Mage
-
Mage is a CLI. Gtg is a library with CLI convenience features.
-
Gtg does not require an external CLI to function, and doesn't need separate installation. Just
go run .which auto-installs all dependencies including Gtg. -
Mage has a build system. Gtg is just a library. No accidental conflicts in imports and declarations. No accidental log suppression or log spam. No need for special system variables. No multi-GiB hidden cache folder stuck forever on your system.
-
Gtg has no implicit control flow. Just handle your errors. It provides
Mustshortcuts which are optional, explicit, and conventional. -
Gtg is compatible with external watchers such as Gow.
-
Gtg is much smaller and simpler. It adds very few concepts: a minor extension of the
context.Contextinterface, and a few utility functions defined in terms of that. -
Gtg is new and immature.
Known Limitations and TODOs
-
CLI usage needs better error logging. Currently we just panic, rendering an error message but also the call stack.
-
Task identity is determined via function pointers, using unsafe hacks. May be unreliable, needs more testing.
-
ChooseandRunCmdallow to run only one task. We should provide shortcuts for selecting N tasks, which can be run concurrently viaParor serially viaSer. -
Sershould produce an error when other tasks cause the requested tasks to run in a different order. Currently this is unchecked.
License
Misc
I'm receptive to suggestions. If this library almost satisfies you but needs changes, open an issue or chat me up. Contacts: https://mitranim.com/#contacts
Documentation
¶
Overview ¶
"Go task group" or "Go task graph". Utility for running tasks (functions with a context) as a single group with mutual coordination and deduplication.
Good for CLI task orchestration, serving as a Go-based alternative to Make and a simple, flexible replacement for Mage (https://github.com/magefile/mage). May be useful for non-CLI applications.
For examples and comparison with other tools, see the readme: https://github.com/mitranim/gtg.
Index ¶
- func Log(err error)
- func Must(err error)
- func MustRun(ctx context.Context, fun TaskFunc)
- func MustRunCmd(funs ...TaskFunc)
- func MustWait(group TaskGroup, fun TaskFunc)
- func Run(ctx context.Context, fun TaskFunc) error
- func RunCmd(funs ...TaskFunc) error
- func TaskTiming(fun TaskFunc) func()
- func Timing(name string) func()
- func Wait(group TaskGroup, fun TaskFunc) error
- type Task
- type TaskFunc
- type TaskGroup
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
func Log ¶
func Log(err error)
Convenience function for CLI. If the error is non-nil, logs it, otherwise ignores it:
Log(Wait(task, AnotherTask))
func Must ¶
func Must(err error)
Panics if the error is non-nil. Allows shorter, cleaner task code, while keeping control flow explicit. Gtg automatically handles panics in tasks, annotating them with task names.
func Run ¶
Creates a new task group/graph. Runs `fun` as the first task in the group, blocks until it finishes, and returns its error.
When this "main" task finishes, the context provided to all tasks in this group is canceled.
func RunCmd ¶
Convenience function for CLI. Selects one task function via `Choose`, using the command line arguments from `os.Args`. Runs this task and returns its error.
CLI scripts can use the `MustRunCmd` shortcut.
func TaskTiming ¶
func TaskTiming(fun TaskFunc) func()
Convenience function for CLI. Logs execution time of a task function. Usage:
func SomeTask(Task) error {
defer TaskTiming(SomeTask)()
return nil
}
Output:
[SomeTask] starting [SomeTask] done in 1μs
Types ¶
type Task ¶
Describes a task, which is a superset of `context.Context` but also belongs to a task group; see the `TaskGroup` interface. Every task in the group provides access to the entire group, able to retrieve and create tasks in it.
Gtg has two "views" of a task: from the "inside" (what's passed to a task function), and from the "outside" (as returned by `TaskGroup.Task`).
The `Task` passed to its task function has no special properties: its context is a normal `context.Context` instance.
However, when seen from the "outside", a `Task` does not behave like a normal context. Instead, its `Done()` and `Err()` are determined entirely by its function. The channel returned by `Done()` is closed when it returns or panics, and the error returned by `Err()` is the function's result or panic. Honoring the original context's cancellation is entirely up to the task function. This behavior allows external callers to observe a task's completion and result, which is crucial to task coordination in Gtg.
type TaskFunc ¶
Task functions may be invoked by `Start`, `Run`, `Task.Task`, and so on. They shouldn't be called manually, because the purpose of this package is to deduplicate tasks in the same group/graph.
Task functions may be statically defined or closures. All references to the same static function have the same identity, while closures created by the same function have different identities. Identity is used for deduplication.
func Choose ¶
Matches task names against function names (case-insensitive), selecting exactly one task function. Validates that all task names are "known", there are no duplicates among task names and functions, and that exactly one function can be selected. The returned error, if any, will list the "known" tasks derived from function names.
CLI scripts can use the `MustRunCmd` shortcut.
func Opt ¶
Short for "optional". Wraps a task function, making its success optional. The task will always run, but its error will simply be logged.
For a single dependency, this is no better than:
Log(Wait(task, fun))
The main use is for composition:
MustWait(task, Par(A, Opt(B), C))
This is a convenience feature for CLI scripts. Apps usually do their own logging, and would write their own version of this function.
func Par ¶
Short for "parallel" (although "concurrent" would be more precise). Creates a task function that will request all given tasks to be run concurrently.
As always, any task in the current group is run only once. A task that finished earlier will not be called again.
func Ser ¶
Short for "serial". Creates a task function that will wait on the given tasks in a sequence.
As always, any task in the current group is run only once. A task that finished earlier will not be called again. The actual order of task execution may not match the order in `Ser`.
Currently in Gtg, parallel takes priority over serial; making sure that no other task is trying to run everything in parallel is on the user.
type TaskGroup ¶
Describes a group of tasks. Able to deduplicate tasks, identifying them by the task function. The method `Task()` returns an existing task (possibly already finished) corresponding to the given function. If no such task exists, `Task()` creates it, launching the function on another goroutine, and returns the newly-created task.
Source Files
Directories
¶
| Path | Synopsis |
|---|---|
|
Usage: go run ./examples/diamond.go go run ./examples/diamond.go <task> Shaped like this: A / \ v v B C \ / v D Usage: go run ./examples/site.go go run ./examples/site.go <task>
|
Usage: go run ./examples/diamond.go go run ./examples/diamond.go <task> Shaped like this: A / \ v v B C \ / v D Usage: go run ./examples/site.go go run ./examples/site.go <task> |