Gofer: Run short-lived jobs easily.
Gofer is an opinionated, streamlined automation engine designed for the cloud-native era. It specializes in executing your custom scripts in a containerized environment, making it versatile for both developers and operations teams. Deploy Gofer effortlessly as a single static binary, and manage it using expressive, declarative configurations written in real programming languages. Once set up, Gofer takes care of scheduling and running your automation tasks—be it on Nomad, Kubernetes, or even Local Docker.
Its primary function is to execute short-term jobs like code linting, build automation, testing, port scanning, ETL operations, or any task you can containerize and trigger based on events.
- Simple Deployment: Install Gofer effortlessly with a single static binary and manage it through its intuitive command-line interface.
- Language Flexibility: Craft your pipelines in programming languages you're already comfortable with, such as Go or Rust—no more wrestling with unfamiliar YAML.
- Local Testing: Validate and run your pipelines locally, eliminating the guesswork of "commit and see" testing.
- Extensible Architecture: Easily extend Gofer's capabilities by writing your own plugins, backends, and more, in any language via gRPC.
- Built-In Storage: Comes with an integrated Object and Secret store for your convenience.
- DAG Support: Harness the power of Directed Acyclic Graphs (DAGs) for complex workflow automation.
- Robust Reliability: Automatic versioning, Blue/Green deployments, and canary releases ensure the stability and dependability of your pipelines.
Documentation & Getting Started
If you want to fully dive into Gofer, check out the documentation site!
Extended installation information is available through the documentation site.
Download a specific release:
Download the latest release:
Build from source:
You'll need to install protoc and its associated golang/grpc modules first
git clone https://github.com/clintjedwards/gofer && cd gofer
make build OUTPUT=/tmp/gofer
The Gofer binary comes with a CLI to manage the server as well as act as a client.
Gofer is setup such that the base run mode is the development mode. So simply running the binary without any additional flags allows easy authless development.
You'll need to install the following first:
To run Gofer dev mode:
To build protocol buffers:
Run from the Makefile
Gofer uses flags, env vars, and files to manage configuration (in order of most important). The Makefile already includes all the commands and flags you need to run in dev mode by simply running
In case you want to run without the make file simply run:
export GOFER_LOG_LEVEL=debug go build -o /tmp/$gofer /tmp/gofer service start --dev-mode
Gofer uses grpc and protobufs to communicate with both plugins and provide an external API. These protobuf
files are located in
/proto. To compile new protobufs once the original
.proto files have changed you can use the
make build-protos command.
Documentation is done with mdbook.
cargo install mdbook cargo install mdbook-linkcheck
Once you have mdbook you can simply run
make run-docs to give you an auto-reloading dev version of the documentation in a browser.
Regenerating Demo Gif
The Gif on the README page uses vhs; a very handy tool that allows you to write a configuration file which will pop out a gif on the other side.
In order to do this VHS has to run the commands so we must start the server first before we regenerate the gif.
rm -rf /tmp/gofer* # Start with a fresh database make run # Start the server in dev mode cd documentation/src/assets vhs < demo.tape # this will start running commands against the server and output the gif as demo.gif.
- Clint Edwards - Github
This software is provided as-is. It's a hobby project, done in my free time, and I don't get paid for doing it.
There is no documentation for this package.
Events contains the data models used in Gofer events.
|Events contains the data models used in Gofer events.|
Package api controls the bulk of the Gofer API logic.
|Package api controls the bulk of the Gofer API logic.|
Package app is the setup package for all things API related.
|Package app is the setup package for all things API related.|
Package cli controls the main user entry point into both the API and interacting with it.
|Package cli controls the main user entry point into both the API and interacting with it.|
Package cl contains global variables used across the cli package.
|Package cl contains global variables used across the cli package.|
Config controls the overall configuration of the application.
|Config controls the overall configuration of the application.|
Package models contains the core objects(otherwise called the domain models) needed by Gofer internally.
|Package models contains the core objects(otherwise called the domain models) needed by Gofer internally.|
Package scheduler defines the interface in which a scheduler must adhere to.
|Package scheduler defines the interface in which a scheduler must adhere to.|
Package storage contains the data storage interface in which Gofer stores all internal data.
|Package storage contains the data storage interface in which Gofer stores all internal data.|
Temporary sync map replacement with generics until go 1.18 bakes a bit and one gets released.
|Temporary sync map replacement with generics until go 1.18 bakes a bit and one gets released.|
Package dag is used to verify and map out directed acyclic graph implementations.
|Package dag is used to verify and map out directed acyclic graph implementations.|