Documentation
¶
Overview ¶
Package runners defines the common "--runner" flag. We have a single definition for it to avoid re-definition errors, between it and test packages or other integration harnesses.
Index ¶
Constants ¶
This section is empty.
Variables ¶
View Source
var Runner = flag.String("runner", "", "Pipeline runner.")
Runner is a flag to specify which Beam runner should be used to execute the pipeline.
Functions ¶
This section is empty.
Types ¶
This section is empty.
Source Files
Directories
¶
| Path | Synopsis |
|---|---|
|
Package dataflow contains the Dataflow runner for submitting pipelines to Google Cloud Dataflow.
|
Package dataflow contains the Dataflow runner for submitting pipelines to Google Cloud Dataflow. |
|
dataflowlib
Package dataflowlib translates a Beam pipeline model to the Dataflow API job model, for submission to Google Cloud Dataflow.
|
Package dataflowlib translates a Beam pipeline model to the Dataflow API job model, for submission to Google Cloud Dataflow. |
|
Package direct contains the direct runner for running single-bundle pipelines in the current process.
|
Package direct contains the direct runner for running single-bundle pipelines in the current process. |
|
Package dot is a Beam runner that "runs" a pipeline by producing a DOT graph of the execution plan.
|
Package dot is a Beam runner that "runs" a pipeline by producing a DOT graph of the execution plan. |
|
Package flink contains the Flink runner.
|
Package flink contains the Flink runner. |
|
Package prism contains a local runner for running pipelines in the current process.
|
Package prism contains a local runner for running pipelines in the current process. |
|
internal/config
Package config defines and handles the parsing and provision of configurations for the runner.
|
Package config defines and handles the parsing and provision of configurations for the runner. |
|
internal/engine
Package engine handles the operational components of a runner, to track elements, watermarks, timers, triggers etc
|
Package engine handles the operational components of a runner, to track elements, watermarks, timers, triggers etc |
|
internal/jobservices
Package jobservices handles services necessary WRT handling jobs from SDKs.
|
Package jobservices handles services necessary WRT handling jobs from SDKs. |
|
internal/urns
Package urns handles extracting urns from all the protos.
|
Package urns handles extracting urns from all the protos. |
|
internal/worker
Package worker handles interactions with SDK side workers, representing the worker services, communicating with those services, and SDK environments.
|
Package worker handles interactions with SDK side workers, representing the worker services, communicating with those services, and SDK environments. |
|
Package samza contains the Samza runner.
|
Package samza contains the Samza runner. |
|
Package spark contains the Spark runner.
|
Package spark contains the Spark runner. |
|
Package universal contains a general-purpose runner that can submit jobs to any portable Beam runner.
|
Package universal contains a general-purpose runner that can submit jobs to any portable Beam runner. |
|
extworker
Package extworker provides an external worker service and related utilities.
|
Package extworker provides an external worker service and related utilities. |
|
runnerlib
Package runnerlib contains utilities for submitting Go pipelines to a Beam model runner.
|
Package runnerlib contains utilities for submitting Go pipelines to a Beam model runner. |
|
Package vet is a Beam runner that "runs" a pipeline by producing generated code to avoid symbol table lookups and reflection in pipeline execution.
|
Package vet is a Beam runner that "runs" a pipeline by producing generated code to avoid symbol table lookups and reflection in pipeline execution. |
|
testpipeline
Package testpipeline exports small test pipelines for testing the vet runner.
|
Package testpipeline exports small test pipelines for testing the vet runner. |
Click to show internal directories.
Click to hide internal directories.