dagu

README

A lightweight and powerful workflow engine for enterprise & small teams

Self-contained. Language agnostic. Lightweight.

Website | Quick Start | Discussions

• Overview - Orchestrate workflows without complexity
  • Use Cases
  • Quick Demos
  • Quick Start
    • 1. Install dagu
    • 2. Create your first workflow
    • 3. Run the workflow
    • 4. Check the status and view logs
    • 5. Explore the Web UI
  • Development
    • Building from Source
      • Prerequisites
      • 1. Clone the repository and build server
      • 2. Run the frontend development server
  • Documentation
  • Discussion
  • Recent Updates
    • v1.18 - 2025-07-29
  • Contributing
  • Acknowledgements
    • Contributors
    • Sponsors & Supporters
  • License

Overview - Orchestrate workflows without complexity

Dagu /dah-goo/ is a compact, portable workflow engine implemented in Go. It provides a declarative model for orchestrating command execution across diverse environments, including shell scripts, Python commands, containerized operations, or remote commands.

steps:
  - name: step1
    command: sleep 1 && echo "Hello, dagu!"
  - name: step2
    command: sleep 1 && echo "This is a second step"

By declaratively defining job processes, complex workflows can be visualized, making troubleshooting and recovery easier. Viewing logs and retrying jobs can be performed from the Web UI, eliminating the need to log into a server via SSH.

It is equipped with many features to meet the detailed requirements of enterprise environments. It operates even in environments without internet access. Being a statically compiled binary, it includes all dependencies, allowing it to run in any environment, including on-premise, cloud, and IoT devices. It is a lightweight workflow engine that meets enterprise requirements.

Note: For a list of features, please refer to the documentation.

Workflow jobs are defined as commands. Therefore, legacy scripts that have been in operation for a long time within a company or organization can be used as-is without modification. There is no need to learn a complex new language, and you can start using it right away.

dagu is designed for small teams of 1-3 people to easily manage complex workflows. It aims to be an ideal choice for teams that find large-scale, high-cost infrastructure like Airflow to be overkill and are looking for a simpler solution. It requires no database management and only needs a shared filesystem, allowing you to focus on your high-value work.

Use Cases

dagu is designed to orchestrate workflows across various domains, particularly those involving multi-step batch jobs and complex data dependencies. Example applications include:

• AI/ML & Data Science - Automating machine learning workflows, including data ingestion, feature engineering, model training, validation, and deployment.
• Geospatial & Environmental Analysis - Processing datasets from sources such as satellites (earth observation), aerial/terrestrial sensors, seismic surveys, and ground-based radar. Common uses include numerical weather prediction and natural resource management.
• Finance & Trading - Implementing time-sensitive workflows for stock market analysis, quantitative modeling, risk assessment, and report generation.
• Medical Imaging & Bioinformatics - Creating pipelines to process and analyze large volumes of medical scans (e.g., MRI, CT) or genomic data for clinical and research purposes.
• Data Engineering (ETL/ELT) - Building, scheduling, and monitoring pipelines for moving and transforming data between systems like databases, data warehouses, and data lakes.
• IoT & Edge Computing - Orchestrating workflows that collect, process, and analyze data from distributed IoT devices, sensors, and edge nodes.
• Media & Content Processing - Automating workflows for video transcoding, image processing, and content delivery, including tasks like format conversion, compression, and metadata extraction.

Quick Demos

CLI Demo: Create and run a simple DAG workflow from the command line.

Web UI Demo: Create and manage workflows using the web interface, with real-time monitoring and control.

Docs on CLI

Docs on Web UI

Quick Start

1. Install dagu

# Install via npm
npm install -g dagu

Note: see documentation for other methods.

2. Create your first workflow

cat > ./hello.yaml << 'EOF'
steps:
  - name: hello
    command: echo "Hello from dagu!"
  - name: world  
    command: echo "Running step 2"
EOF

3. Run the workflow

dagu start hello.yaml

4. Check the status and view logs

dagu status hello

5. Explore the Web UI

dagu start-all

Visit http://localhost:8080

Development

Building from Source

Prerequisites

• Go 1.24+
• Node.js
• pnpm

1. Clone the repository and build server

git clone https://github.com/dagu-org/dagu.git && cd dagu
make

This will start the dagu server at http://localhost:8080.

2. Run the frontend development server

cd ui
pnpm install
pnpm dev

Navigate to http://localhost:8081 to view the frontend.

Documentation

Full documentation is available at docs.dagu.cloud.

Discussion

For discussions, support, and sharing ideas, join our community on Discord.

Recent Updates

v1.18 - 2025-07-29

• OpenID Connect (OIDC) Support: OpenID Connect authentication for secure access (@Arvintian)
• Distributed Execution: Run steps across multiple machines with worker processes
• High Availability: Redundant scheduler instances with automatic failover
• Step-level Environment Variables: Define environment variables specific to individual steps (@admerzeau)
• Scheduler Health Check: HTTP health check endpoint for monitoring (default: http://localhost:8090/health) - automatically enabled with dagu scheduler (@jonasban)
• Enhanced Repeat Policy: Explicit until and while modes for clearer repeat logic (@thefishhat)
• Exponential Backoff: Added exponential backoff support for retry policies (@Sarvesh-11)
• Live Log Loading: Real-time log streaming in the Web UI (@tapir)
• Partial Success Status: Step-level partial success for sub-DAG executions (@ghansham)
• Multiple Email Recipients: Support for multiple recipients in email notifications
• UI Improvements: Manual sidebar toggle, better DAG sorting, and various enhancements (@ghansham)

Full changelog

Contributing

We welcome contributions of all kinds! If you have ideas, suggestions, or improvements, please open an issue or submit a pull request.

• Discord Community
• GitHub Issues

Acknowledgements

Contributors

Thanks to all the contributors who have helped make dagu better! Your contributions, whether through code, documentation, or feedback, are invaluable to the project.

Sponsors & Supporters

Thanks for supporting dagu’s development! Join our supporters: GitHub Sponsors

License

GNU GPLv3 - See LICENSE

---

If you find dagu useful, please ⭐ star this repository