README
¶
Funnel is a toolkit for distributed, batch task execution, including a server, worker, and a set of compute, storage, and database backends. Given a task description, Funnel will find a worker to execute the task, download inputs, run a series of (Docker) containers, upload outputs, capture logs, and track the whole process.
Funnel is an implementation of the GA4GH Task Execution Schemas, an effort to standardize the APIs used for task execution across many platforms.
Funnel provides an API server, multiple storage backends (local FS, S3, Google Bucket, etc.), multiple compute backends (local, HTCondor, Google Cloud, etc.), and a web dashboard.
Documentation
¶
There is no documentation for this package.
Source Files
Directories
¶
| Path | Synopsis |
|---|---|
|
Package cmd contains the Funnel CLI commands.
|
Package cmd contains the Funnel CLI commands. |
|
Package compute contains code for accessing compute resources from many different cluster types, including AWS, Google Cloud, and HPC-style cluster scheduler.
|
Package compute contains code for accessing compute resources from many different cluster types, including AWS, Google Cloud, and HPC-style cluster scheduler. |
|
aws_batch
Package aws_batch contains code for accessing compute resources via AWS Batch.
|
Package aws_batch contains code for accessing compute resources via AWS Batch. |
|
gcp_batch
Package gcp_batch contains code for accessing compute resources via Google Batch.
|
Package gcp_batch contains code for accessing compute resources via Google Batch. |
|
gridengine
Package gridengine contains code for accessing compute resources via Open Grid Engine.
|
Package gridengine contains code for accessing compute resources via Open Grid Engine. |
|
htcondor
Package htcondor contains code for accessing compute resources via HTCondor.
|
Package htcondor contains code for accessing compute resources via HTCondor. |
|
kubernetes
Package kubernetes contains code for accessing compute resources via the Kubernetes v1 Batch API.
|
Package kubernetes contains code for accessing compute resources via the Kubernetes v1 Batch API. |
|
local
Package local contains code for accessing compute resources via the local computer, for Funnel development and debugging.
|
Package local contains code for accessing compute resources via the local computer, for Funnel development and debugging. |
|
noop
Package noop contains a compute backend that does nothing, for testing purposes.
|
Package noop contains a compute backend that does nothing, for testing purposes. |
|
pbs
Package pbs contains code for accessing compute resources via PBS/Torque.
|
Package pbs contains code for accessing compute resources via PBS/Torque. |
|
scheduler
Code generated by mockery v1.0.0.
|
Code generated by mockery v1.0.0. |
|
slurm
Package slurm contains code for accessing compute resources via Slurm.
|
Package slurm contains code for accessing compute resources via Slurm. |
|
Package config contains Funnel configuration structures and defaults.
|
Package config contains Funnel configuration structures and defaults. |
|
database
|
|
|
Package events contains code for writing task events to various systems.
|
Package events contains code for writing task events to various systems. |
|
Package examples bundles example tasks into the Funnel CLI.
|
Package examples bundles example tasks into the Funnel CLI. |
|
Package logger contains Funnel's logging code.
|
Package logger contains Funnel's logging code. |
|
Package server contains code for serving the Funnel API, and accessing database backends.
|
Package server contains code for serving the Funnel API, and accessing database backends. |
|
Package tes is a reverse proxy.
|
Package tes is a reverse proxy. |
|
Package util is the dumping ground for helper functions without a good home.
|
Package util is the dumping ground for helper functions without a good home. |
|
openapi2proto
command
|
|
|
Package version reports the Funnel version.
|
Package version reports the Funnel version. |
|
Package worker contains code which executes a task.
|
Package worker contains code which executes a task. |
Click to show internal directories.
Click to hide internal directories.