Directories
¶
| Path | Synopsis |
|---|---|
|
Package beam is an implementation of the Apache Beam (https://beam.apache.org) programming model in Go.
|
Package beam is an implementation of the Apache Beam (https://beam.apache.org) programming model in Go. |
|
artifact
Package artifact contains utilities for staging and retrieving artifacts.
|
Package artifact contains utilities for staging and retrieving artifacts. |
|
artifact/gcsproxy
Package gcsproxy contains artifact staging and retrieval servers backed by GCS.
|
Package gcsproxy contains artifact staging and retrieval servers backed by GCS. |
|
core
Package core contains constants and other static data related to the SDK, such as the SDK Name and version.
|
Package core contains constants and other static data related to the SDK, such as the SDK Name and version. |
|
core/funcx
Package funcx contains functions and types used to perform type analysis of Beam functions.
|
Package funcx contains functions and types used to perform type analysis of Beam functions. |
|
core/graph
Package graph is the internal representation of the Beam execution plan.
|
Package graph is the internal representation of the Beam execution plan. |
|
core/graph/coder
Package coder contains coder representation and utilities.
|
Package coder contains coder representation and utilities. |
|
core/graph/coder/testutil
Package testutil contains helpers to test and validate custom Beam Schema coders.
|
Package testutil contains helpers to test and validate custom Beam Schema coders. |
|
core/graph/mtime
Package mtime contains a millisecond representation of time.
|
Package mtime contains a millisecond representation of time. |
|
core/graph/window
Package window contains window representation, windowing strategies and utilities.
|
Package window contains window representation, windowing strategies and utilities. |
|
core/metrics
Package metrics implements the Beam metrics API, described at http://s.apache.org/beam-metrics-api Metrics in the Beam model are uniquely identified by a namespace, a name, and the PTransform context in which they are used.
|
Package metrics implements the Beam metrics API, described at http://s.apache.org/beam-metrics-api Metrics in the Beam model are uniquely identified by a namespace, a name, and the PTransform context in which they are used. |
|
core/runtime
Package runtime contains runtime hooks and utilities for pipeline options and type registration.
|
Package runtime contains runtime hooks and utilities for pipeline options and type registration. |
|
core/runtime/coderx
Package coderx contains coders for primitive types that aren't included in the beam model.
|
Package coderx contains coders for primitive types that aren't included in the beam model. |
|
core/runtime/exec
Package exec contains runtime plan representation and execution.
|
Package exec contains runtime plan representation and execution. |
|
core/runtime/exec/optimized
Package optimized contains type-specialized shims for faster execution.
|
Package optimized contains type-specialized shims for faster execution. |
|
core/runtime/genx
Package genx is a convenience package to better support the code generator.
|
Package genx is a convenience package to better support the code generator. |
|
core/runtime/graphx
Package graphx provides facilities to help with the serialization of pipelines into a serializable graph structure suitable for the worker.
|
Package graphx provides facilities to help with the serialization of pipelines into a serializable graph structure suitable for the worker. |
|
core/runtime/graphx/schema
Package schema contains utility functions for relating Go types and Beam Schemas.
|
Package schema contains utility functions for relating Go types and Beam Schemas. |
|
core/runtime/harness
Package harness implements the SDK side of the Beam FnAPI.
|
Package harness implements the SDK side of the Beam FnAPI. |
|
core/runtime/harness/init
Package init contains the harness initialization code defined by the FnAPI.
|
Package init contains the harness initialization code defined by the FnAPI. |
|
core/runtime/pipelinex
Package pipelinex contains utilities for manipulating Beam proto pipelines.
|
Package pipelinex contains utilities for manipulating Beam proto pipelines. |
|
core/runtime/xlangx
Package xlangx contains various low-level utilities needed for adding cross-language transforms to the pipeline.
|
Package xlangx contains various low-level utilities needed for adding cross-language transforms to the pipeline. |
|
core/sdf
Package contains interfaces used specifically for splittable DoFns.
|
Package contains interfaces used specifically for splittable DoFns. |
|
core/typex
Package typex contains full type representation for PCollections and DoFns, and utilities for type checking.
|
Package typex contains full type representation for PCollections and DoFns, and utilities for type checking. |
|
core/util/dot
Package dot produces DOT graphs from Beam graph representations.
|
Package dot produces DOT graphs from Beam graph representations. |
|
core/util/hooks
Package hooks allows runners to tailor execution of the worker harness.
|
Package hooks allows runners to tailor execution of the worker harness. |
|
core/util/ioutilx
Package ioutilx contains additional io utilities.
|
Package ioutilx contains additional io utilities. |
|
core/util/jsonx
Package jsonx contains utilities for working with JSON encoded data.
|
Package jsonx contains utilities for working with JSON encoded data. |
|
core/util/protox
Package protox contains utilities for working with protobufs.
|
Package protox contains utilities for working with protobufs. |
|
core/util/reflectx
Package reflectx contains a set of reflection utilities and well-known types.
|
Package reflectx contains a set of reflection utilities and well-known types. |
|
core/util/stringx
Package stringx contains utilities for working with strings.
|
Package stringx contains utilities for working with strings. |
|
core/util/symtab
Package symtab allows reading low-level symbol information from the symbol table.
|
Package symtab allows reading low-level symbol information from the symbol table. |
|
io/avroio
Package avroio contains transforms for reading and writing avro files.
|
Package avroio contains transforms for reading and writing avro files. |
|
io/bigqueryio
Package bigqueryio provides transformations and utilities to interact with Google BigQuery.
|
Package bigqueryio provides transformations and utilities to interact with Google BigQuery. |
|
io/databaseio
Package databaseio provides transformations and utilities to interact with generic database database/sql API.
|
Package databaseio provides transformations and utilities to interact with generic database database/sql API. |
|
io/datastoreio
Package datastoreio provides transformations and utilities to interact with Google Datastore.
|
Package datastoreio provides transformations and utilities to interact with Google Datastore. |
|
io/filesystem
Package filesystem contains an extensible file system abstraction.
|
Package filesystem contains an extensible file system abstraction. |
|
io/filesystem/gcs
Package gcs contains a Google Cloud Storage (GCS) implementation of the Beam file system.
|
Package gcs contains a Google Cloud Storage (GCS) implementation of the Beam file system. |
|
io/filesystem/local
Package local contains a local file implementation of the Beam file system.
|
Package local contains a local file implementation of the Beam file system. |
|
io/filesystem/memfs
Package memfs contains a in-memory Beam filesystem.
|
Package memfs contains a in-memory Beam filesystem. |
|
io/pubsubio
Package pubsubio provides access to PubSub on Dataflow streaming.
|
Package pubsubio provides access to PubSub on Dataflow streaming. |
|
io/rtrackers/offsetrange
Package offsetrange defines a restriction and restriction tracker for offset ranges.
|
Package offsetrange defines a restriction and restriction tracker for offset ranges. |
|
io/synthetic
Package synthetic contains transforms for creating synthetic pipelines.
|
Package synthetic contains transforms for creating synthetic pipelines. |
|
io/textio
Package textio contains transforms for reading and writing text files.
|
Package textio contains transforms for reading and writing text files. |
|
io/xlang/kafkaio
Package kafkaio contains cross-language functionality for using Apache Kafka (http://kafka.apache.org/).
|
Package kafkaio contains cross-language functionality for using Apache Kafka (http://kafka.apache.org/). |
|
log
Package log contains a re-targetable context-aware logging system.
|
Package log contains a re-targetable context-aware logging system. |
|
model
Package model contains the portable Beam model contracts.
|
Package model contains the portable Beam model contracts. |
|
options/gcpopts
Package gcpopts contains shared options for Google Cloud Platform.
|
Package gcpopts contains shared options for Google Cloud Platform. |
|
options/jobopts
Package jobopts contains shared options for job submission.
|
Package jobopts contains shared options for job submission. |
|
provision
Package provision contains utilities for obtaining runtime provision, information -- such as pipeline options.
|
Package provision contains utilities for obtaining runtime provision, information -- such as pipeline options. |
|
runners/dataflow
Package dataflow contains the Dataflow runner for submitting pipelines to Google Cloud Dataflow.
|
Package dataflow contains the Dataflow runner for submitting pipelines to Google Cloud Dataflow. |
|
runners/dataflow/dataflowlib
Package dataflowlib translates a Beam pipeline model to the Dataflow API job model, for submission to Google Cloud Dataflow.
|
Package dataflowlib translates a Beam pipeline model to the Dataflow API job model, for submission to Google Cloud Dataflow. |
|
runners/direct
Package direct contains the direct runner for running single-bundle pipelines in the current process.
|
Package direct contains the direct runner for running single-bundle pipelines in the current process. |
|
runners/dot
Package dot is a Beam runner that "runs" a pipeline by producing a DOT graph of the execution plan.
|
Package dot is a Beam runner that "runs" a pipeline by producing a DOT graph of the execution plan. |
|
runners/flink
Package flink contains the Flink runner.
|
Package flink contains the Flink runner. |
|
runners/samza
Package samza contains the Samza runner.
|
Package samza contains the Samza runner. |
|
runners/spark
Package spark contains the Spark runner.
|
Package spark contains the Spark runner. |
|
runners/universal
Package universal contains a general-purpose runner that can submit jobs to any portable Beam runner.
|
Package universal contains a general-purpose runner that can submit jobs to any portable Beam runner. |
|
runners/universal/extworker
Package extworker provides an external worker service and related utilities.
|
Package extworker provides an external worker service and related utilities. |
|
runners/universal/runnerlib
Package runnerlib contains utilities for submitting Go pipelines to a Beam model runner.
|
Package runnerlib contains utilities for submitting Go pipelines to a Beam model runner. |
|
runners/vet
Package vet is a Beam runner that "runs" a pipeline by producing generated code to avoid symbol table lookups and reflection in pipeline execution.
|
Package vet is a Beam runner that "runs" a pipeline by producing generated code to avoid symbol table lookups and reflection in pipeline execution. |
|
runners/vet/testpipeline
Package testpipeline exports small test pipelines for testing the vet runner.
|
Package testpipeline exports small test pipelines for testing the vet runner. |
|
testing/passert
Package passert contains verification transformations for testing pipelines.
|
Package passert contains verification transformations for testing pipelines. |
|
testing/ptest
Package ptest contains utilities for pipeline unit testing.
|
Package ptest contains utilities for pipeline unit testing. |
|
transforms/filter
Package filter contains transformations for removing pipeline elements based on various conditions.
|
Package filter contains transformations for removing pipeline elements based on various conditions. |
|
transforms/stats
Package stats contains transforms for statistical processing.
|
Package stats contains transforms for statistical processing. |
|
transforms/top
Package top contains transformations for finding the smallest (or largest) N elements based on arbitrary orderings.
|
Package top contains transformations for finding the smallest (or largest) N elements based on arbitrary orderings. |
|
util/errorx
Package errorx contains utilities for handling errors.
|
Package errorx contains utilities for handling errors. |
|
util/execx
Package execx contains wrappers and utilities for the exec package.
|
Package execx contains wrappers and utilities for the exec package. |
|
util/gcsx
Package gcsx contains utilities for working with Google Cloud Storage (GCS).
|
Package gcsx contains utilities for working with Google Cloud Storage (GCS). |
|
util/grpcx
Package grpcx contains utilities for working with gRPC.
|
Package grpcx contains utilities for working with gRPC. |
|
util/pubsubx
Package pubsubx contains utilities for working with Google PubSub.
|
Package pubsubx contains utilities for working with Google PubSub. |
|
util/shimx
Package shimx specifies the templates for generating type assertion shims for Apache Beam Go SDK pipelines.
|
Package shimx specifies the templates for generating type assertion shims for Apache Beam Go SDK pipelines. |
|
util/starcgenx
Package starcgenx is a Static Analysis Type Assertion shim and Registration Code Generator which provides an extractor to extract types from a package, in order to generate approprate shimsr a package so code can be generated for it.
|
Package starcgenx is a Static Analysis Type Assertion shim and Registration Code Generator which provides an extractor to extract types from a package, in order to generate approprate shimsr a package so code can be generated for it. |
|
util/syscallx
Package syscallx provides system call utilities that attempt to hide platform differences.
|
Package syscallx provides system call utilities that attempt to hide platform differences. |
|
x/beamx
Package beamx is a convenience package for beam.
|
Package beamx is a convenience package for beam. |
|
x/debug
Package debug contains pipeline components that may help in debugging pipeline issues.
|
Package debug contains pipeline components that may help in debugging pipeline issues. |
|
x/hooks/perf
Package perf is to add performance measuring hooks to a runner, such as cpu, heap, or trace profiles.
|
Package perf is to add performance measuring hooks to a runner, such as cpu, heap, or trace profiles. |