funnel

command module
v0.12.3 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Nov 9, 2026 License: MIT Imports: 5 Imported by: 0

README

Tests Compliance Tests Status Godoc GitHub Release Docker

Funnel is a toolkit for distributed, batch task execution, including a server, worker, and a set of compute, storage, and database backends. Given a task description, Funnel will find a worker to execute the task, download inputs, run a series of (Docker) containers, upload outputs, capture logs, and track the whole process.

Funnel is an implementation of the GA4GH Task Execution Schemas, an effort to standardize the APIs used for task execution across many platforms.

Funnel provides an API server, multiple storage backends (local FS, S3, Google Bucket, etc.), multiple compute backends (local, HTCondor, Google Cloud, etc.), and a web dashboard.

https://calypr.org/tools/funnel/

Deployment provisioning, manifests, service definitions, upgrades, and infrastructure operations are maintained by operators. This repository provides the Funnel application, container images, and configuration documentation; it does not provide maintained deployment automation.

Documentation source lives in docs/. Hugo templates, site configuration, and downloadable assets live in website/. Run make website to generate the site and search index in build/website/, or make website-dev to preview it locally. docs/internal/ is reserved for local design notes and is excluded from the published site.

Documentation

The Go Gopher

There is no documentation for this package.

Directories

Path Synopsis
api
config
Package config defines Funnel configuration schemas shared with external plugins.
Package config defines Funnel configuration schemas shared with external plugins.
logger
Package logger defines logging configuration schemas shared with external plugins.
Package logger defines logging configuration schemas shared with external plugins.
plugin
Package plugin defines Funnel's authorization plugin contract and shared HashiCorp go-plugin transport adapters for hosts and plugin implementations.
Package plugin defines Funnel's authorization plugin contract and shared HashiCorp go-plugin transport adapters for hosts and plugin implementations.
tes
Package tes defines shared TES wire models, enum conveniences, and request filters.
Package tes defines shared TES wire models, enum conveniences, and request filters.
client
tes
Package tes provides a context-aware TES REST client using the shared github.com/calypr/funnel/api/tes wire models.
Package tes provides a context-aware TES REST client using the shared github.com/calypr/funnel/api/tes wire models.
Package examples bundles example tasks into the Funnel CLI.
Package examples bundles example tasks into the Funnel CLI.
internal
cli
Package cmd contains the Funnel CLI commands.
Package cmd contains the Funnel CLI commands.
cli/kubernetes
Package kubernetes contains CLI commands for managing Funnel's Kubernetes resources.
Package kubernetes contains CLI commands for managing Funnel's Kubernetes resources.
compute
Package compute defines shared compute backend contracts and configuration policy.
Package compute defines shared compute backend contracts and configuration policy.
compute/aws_batch
Package aws_batch contains code for accessing compute resources via AWS Batch.
Package aws_batch contains code for accessing compute resources via AWS Batch.
compute/gcp_batch
Package gcp_batch contains code for accessing compute resources via Google Batch.
Package gcp_batch contains code for accessing compute resources via Google Batch.
compute/hpc
Package hpc submits and tracks jobs through HPC scheduler commands.
Package hpc submits and tracks jobs through HPC scheduler commands.
compute/hpc/gridengine
Package gridengine contains code for accessing compute resources via Open Grid Engine.
Package gridengine contains code for accessing compute resources via Open Grid Engine.
compute/hpc/htcondor
Package htcondor contains code for accessing compute resources via HTCondor.
Package htcondor contains code for accessing compute resources via HTCondor.
compute/hpc/pbs
Package pbs contains code for accessing compute resources via PBS/Torque.
Package pbs contains code for accessing compute resources via PBS/Torque.
compute/hpc/slurm
Package slurm contains code for accessing compute resources via Slurm.
Package slurm contains code for accessing compute resources via Slurm.
compute/kubernetes
Package kubernetes contains code for accessing compute resources via the Kubernetes v1 Batch API.
Package kubernetes contains code for accessing compute resources via the Kubernetes v1 Batch API.
compute/kubernetes/storageplan
Package storageplan resolves native Kubernetes storage objects for one task.
Package storageplan resolves native Kubernetes storage objects for one task.
compute/local
Package local contains code for accessing compute resources via the local computer, for Funnel development and debugging.
Package local contains code for accessing compute resources via the local computer, for Funnel development and debugging.
compute/noop
Package noop contains a compute backend that does nothing, for testing purposes.
Package noop contains a compute backend that does nothing, for testing purposes.
compute/scheduler
Code generated by mockery v1.0.0.
Code generated by mockery v1.0.0.
config
Package config constructs, loads, and redacts Funnel application configuration.
Package config constructs, loads, and redacts Funnel application configuration.
database/mongodb/mongotest
Package mongotest provides a real single-node replica set for database tests.
Package mongotest provides a real single-node replica set for database tests.
events
Package events contains code for writing task events to various systems.
Package events contains code for writing task events to various systems.
httpauth
Package httpauth enforces credential transport rules for outbound API requests.
Package httpauth enforces credential transport rules for outbound API requests.
logger
Package logger contains Funnel's logging code.
Package logger contains Funnel's logging code.
pluginmanager
Plugin manager used by the main application to load and invoke plugins.
Plugin manager used by the main application to load and invoke plugins.
server
Package server owns HTTP/gRPC serving, service registration, and listener lifecycle.
Package server owns HTTP/gRPC serving, service registration, and listener lifecycle.
server/gateway
Package gateway contains the HTTP gateway's marshaling and response helpers.
Package gateway contains the HTTP gateway's marshaling and response helpers.
server/middleware
Package middleware provides HTTP and gRPC middleware for the server.
Package middleware provides HTTP and gRPC middleware for the server.
server/taskservice
Package taskservice implements TES task operations over readers, events, and compute backends.
Package taskservice implements TES task operations over readers, events, and compute backends.
shellwords
Package shellwords provides POSIX shell-style word splitting, replacing the kballard/go-shellquote and google/shlex dependencies with mvdan.cc/sh/v3.
Package shellwords provides POSIX shell-style word splitting, replacing the kballard/go-shellquote and google/shlex dependencies with mvdan.cc/sh/v3.
tesrpc
Package tesrpc is a reverse proxy.
Package tesrpc is a reverse proxy.
worker
Package worker contains code which executes a task.
Package worker contains code which executes a task.
tests
Package version reports the Funnel version.
Package version reports the Funnel version.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL