What is Argo Workflows?
Argo Workflows is an open source container-native workflow engine for orchestrating parallel jobs on Kubernetes. Argo Workflows is implemented as a Kubernetes CRD (Custom Resource Definition).
- Define workflows where each step in the workflow is a container.
- Model multi-step workflows as a sequence of tasks or capture the dependencies between tasks using a directed acyclic graph (DAG).
- Easily run compute intensive jobs for machine learning or data processing in a fraction of the time using Argo Workflows on Kubernetes.
- Run CI/CD pipelines natively on Kubernetes without configuring complex software development products.
Argo is a Cloud Native Computing Foundation (CNCF) hosted project.
Why Argo Workflows?
- Designed from the ground up for containers without the overhead and limitations of legacy VM and server-based environments.
- Cloud agnostic and can run on any Kubernetes cluster.
- Easily orchestrate highly parallel jobs on Kubernetes.
- Argo Workflows puts a cloud-scale supercomputer at your fingertips!
kubectl create namespace argo kubectl apply -n argo -f https://raw.githubusercontent.com/argoproj/argo/stable/manifests/install.yaml
Who uses Argo Workflows?
- UI to visualize and manage Workflows
- Artifact support (S3, Artifactory, Alibaba Cloud OSS, HTTP, Git, GCS, raw)
- Workflow templating to store commonly used Workflows in the cluster
- Archiving Workflows after executing for later access
- Scheduled workflows
- Server interface with REST API
- DAG or Steps based declaration of workflows
- Step level input & outputs (artifacts/parameters)
- Timeouts (step & workflow level)
- Retry (step & workflow level)
- Resubmit (memoized)
- Suspend & Resume
- K8s resource orchestration
- Exit Hooks (notifications, cleanup)
- Garbage collection of completed workflow
- Scheduling (affinity/tolerations/node selectors)
- Volumes (ephemeral/existing)
- Parallelism limits
- Daemoned steps
- DinD (docker-in-docker)
- Script steps
- Event emission
- Prometheus metrics
- Multiple executors
- Multiple pod and workflow garbage collection strategies
- Automatically calculated resource usage per step
- Pod Disruption Budget support
We host monthly community meetings where we and the community showcase demos and discuss the current and future state of the project. Feel free to join us! For Community Meeting information, minutes and recordings please see here.
Community Blogs and Presentations
- Argo Workflows and Pipelines - CI/CD, Machine Learning, and Other Kubernetes Workflows
- Argo Ansible role: Provisioning Argo Workflows on OpenShift
- Argo Workflows vs Apache Airflow
- CI/CD with Argo on Kubernetes
- Running Argo Workflows Across Multiple Kubernetes Clusters
- Open Source Model Management Roundup: Polyaxon, Argo, and Seldon
- Producing 200 OpenStreetMap extracts in 35 minutes using a scalable data workflow
- Argo integration review
- TGI Kubernetes with Joe Beda: Argo workflow system
Package clusterworkflowtemplate is a reverse proxy.
|Package clusterworkflowtemplate is a reverse proxy.|
Package cronworkflow is a reverse proxy.
|Package cronworkflow is a reverse proxy.|
Package event is a reverse proxy.
|Package event is a reverse proxy.|
Package eventsource is a reverse proxy.
|Package eventsource is a reverse proxy.|
Package info is a reverse proxy.
|Package info is a reverse proxy.|
Package sensor is a reverse proxy.
|Package sensor is a reverse proxy.|
Package workflow is a reverse proxy.
|Package workflow is a reverse proxy.|
Package workflowarchive is a reverse proxy.
|Package workflowarchive is a reverse proxy.|
Package workflowtemplate is a reverse proxy.
|Package workflowtemplate is a reverse proxy.|
Package v1alpha1 is the v1alpha1 version of the API.
|Package v1alpha1 is the v1alpha1 version of the API.|
This package has the automatically generated clientset.
|This package has the automatically generated clientset.|
This package has the automatically generated fake clientset.
|This package has the automatically generated fake clientset.|
This package contains the scheme of the automatically generated clientset.
|This package contains the scheme of the automatically generated clientset.|
This package has the automatically generated typed clients.
|This package has the automatically generated typed clients.|
Package fake has the automatically generated clients.
|Package fake has the automatically generated clients.|
Package cmd provides functionally common to various argo CLIs
|Package cmd provides functionally common to various argo CLIs|