dataflow

package
Version: v2.34.0-RC1 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Oct 26, 2021 License: Apache-2.0, BSD-3-Clause, MIT Imports: 22 Imported by: 0

Documentation

Overview

Package dataflow contains the Dataflow runner for submitting pipelines to Google Cloud Dataflow.

This package infers Pipeline Options from flags automatically on job submission, for display in the Dataflow UI. Use the DontUseFlagAsPipelineOption function to prevent using a given flag as a PipelineOption.

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

func DontUseFlagAsPipelineOption

func DontUseFlagAsPipelineOption(s string)

DontUseFlagAsPipelineOption prevents a set flag from appearing as a PipelineOption in the Dataflow UI. Useful for sensitive, noisy, or irrelevant configuration.

func Execute

func Execute(ctx context.Context, p *beam.Pipeline) (beam.PipelineResult, error)

Execute runs the given pipeline on Google Cloud Dataflow. It uses the default application credentials to submit the job.

Types

This section is empty.

Source Files

Directories

Path Synopsis
Package dataflowlib translates a Beam pipeline model to the Dataflow API job model, for submission to Google Cloud Dataflow.
Package dataflowlib translates a Beam pipeline model to the Dataflow API job model, for submission to Google Cloud Dataflow.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL