argo-dataflow

module
v0.0.57 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Jun 25, 2021 License: Apache-2.0

README

Argo Dataflow

Build codecov

Summary

Argo Dataflow is intended as a cloud-native and language-agnostic platform for executing large parallel data-processing pipelines composed of many steps which are often small and homogenic.

Use Cases

  • Real-time "click" analytics
  • Anomaly detection
  • Fraud detection
  • Operational (including IoT) analytics

Screenshot

Screenshot

Example

pip install git+https://github.com/argoproj-labs/argo-dataflow#subdirectory=dsls/python
from argo_dataflow import cron, pipeline

if __name__ == '__main__':
    (pipeline('hello')
     .namespace('argo-dataflow-system')
     .step(
        (cron('*/3 * * * * *')
         .cat('main')
         .log())
    )
     .run())

Documentation

Read in order:

Beginner:

Intermediate:

Advanced

Architecture Diagram

Architecture

Directories

Path Synopsis
api
v1alpha1
Package v1alpha1 contains API Schema definitions for the dataflow v1alpha1 API group +kubebuilder:object:generate=true +groupName=dataflow.argoproj.io
Package v1alpha1 contains API Schema definitions for the dataflow v1alpha1 API group +kubebuilder:object:generate=true +groupName=dataflow.argoproj.io
hack
kafka Module
cat
map
shared

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL