beam
|
Package beam is an implementation of the Apache Beam (https://beam.apache.org) programming model in Go. |
beam/artifact
|
Package artifact contains utilities for staging and retrieving artifacts. |
beam/artifact/gcsproxy
|
Package gcsproxy contains artifact staging and retrieval servers backed by GCS. |
beam/core/funcx
|
Package funcx contains functions and types used to perform type analysis of Beam functions. |
beam/core/graph
|
Package graph is the internal representation of the Beam execution plan. |
beam/core/graph/coder
|
Package coder contains coder representation and utilities. |
beam/core/graph/mtime
|
Package mtime contains a millisecond representation of time. |
beam/core/graph/window
|
Package window contains window representation, windowing strategies and utilities. |
beam/core/metrics
|
Package metrics implements the Beam metrics API, described at http://s.apache.org/beam-metrics-api Metrics in the Beam model are uniquely identified by a namespace, a name, and the PTransform context in which they are used. |
beam/core/runtime
|
Package runtime contains runtime hooks and utilities for pipeline options and type registration. |
beam/core/runtime/coderx
|
Package coderx contains coders for primitive types that aren't included in the beam model. |
beam/core/runtime/exec
|
Package exec contains runtime plan representation and execution. |
beam/core/runtime/exec/optimized
|
Package optimized contains type-specialized shims for faster execution. |
beam/core/runtime/genx
|
Package genx is a convenience package to better support the code generator. |
beam/core/runtime/graphx
|
Package graphx provides facilities to help with the serialization of pipelines into a serializable graph structure suitable for the worker. |
beam/core/runtime/graphx/schema
|
Package schema contains utility functions for relating Go types and Beam Schemas. |
beam/core/runtime/graphx/v1
|
Package v1 is a generated protocol buffer package. |
beam/core/runtime/harness
|
Package harness implements the SDK side of the Beam FnAPI. |
beam/core/runtime/harness/init
|
Package init contains the harness initialization code defined by the FnAPI. |
beam/core/runtime/harness/session
|
Package session is a generated protocol buffer package. |
beam/core/runtime/metricsx
|
|
beam/core/runtime/pipelinex
|
Package pipelinex contains utilities for manipulating Beam proto pipelines. |
beam/core/runtime/xlangx
|
|
beam/core/sdf
|
Package contains interfaces used specifically for splittable DoFns. |
beam/core/typex
|
Package typex contains full type representation for PCollections and DoFns, and utilities for type checking. |
beam/core/util/dot
|
Package dot produces DOT graphs from Beam graph representations. |
beam/core/util/hooks
|
Package hooks allows runners to tailor execution of the worker harness. |
beam/core/util/ioutilx
|
Package ioutilx contains additional io utilities. |
beam/core/util/jsonx
|
Package jsonx contains utilities for working with JSON encoded data. |
beam/core/util/protox
|
Package protox contains utilities for working with protobufs. |
beam/core/util/reflectx
|
Package reflectx contains a set of reflection utilities and well-known types. |
beam/core/util/stringx
|
Package stringx contains utilities for working with strings. |
beam/core/util/symtab
|
Package symtab allows reading low-level symbol information from the symbol table. |
beam/internal/errors
|
Package errors contains functionality for creating and wrapping errors with improved formatting compared to the standard Go error functionality. |
beam/io/avroio
|
Package avroio contains transforms for reading and writing avro files. |
beam/io/bigqueryio
|
Package bigqueryio provides transformations and utilities to interact with Google BigQuery. |
beam/io/databaseio
|
Package databaseio provides transformations and utilities to interact with generic database database/sql API. |
beam/io/datastoreio
|
Package datastoreio provides transformations and utilities to interact with Google Datastore. |
beam/io/filesystem
|
Package filesystem contains an extensible file system abstraction. |
beam/io/filesystem/gcs
|
Package gcs contains a Google Cloud Storage (GCS) implementation of the Beam file system. |
beam/io/filesystem/local
|
Package local contains a local file implementation of the Beam file system. |
beam/io/filesystem/memfs
|
Package memfs contains a in-memory Beam filesystem. |
beam/io/pubsubio
|
Package pubsubio provides access to PubSub on Dataflow streaming. |
beam/io/pubsubio/v1
|
Package v1 is a generated protocol buffer package. |
beam/io/rtrackers/offsetrange
|
Package offsetrange defines a restriction and restriction tracker for offset ranges. |
beam/io/synthetic
|
Package synthetic contains transforms for creating synthetic pipelines. |
beam/io/textio
|
Package textio contains transforms for reading and writing text files. |
beam/log
|
Package log contains a re-targetable context-aware logging system. |
beam/model
|
Package model contains the portable Beam model contracts. |
beam/model/fnexecution_v1
|
|
beam/model/jobmanagement_v1
|
|
beam/model/pipeline_v1
|
|
beam/options/gcpopts
|
Package gcpopts contains shared options for Google Cloud Platform. |
beam/options/jobopts
|
Package jobopts contains shared options for job submission. |
beam/provision
|
Package provision contains utilities for obtaining runtime provision, information -- such as pipeline options. |
beam/runners/dataflow
|
Package dataflow contains the Dataflow runner for submitting pipelines to Google Cloud Dataflow. |
beam/runners/dataflow/dataflowlib
|
Package dataflowlib translates a Beam pipeline model to the Dataflow API job model, for submission to Google Cloud Dataflow. |
beam/runners/direct
|
Package direct contains the direct runner for running single-bundle pipelines in the current process. |
beam/runners/dot
|
Package dot is a Beam runner that "runs" a pipeline by producing a DOT graph of the execution plan. |
beam/runners/flink
|
Package flink contains the Flink runner. |
beam/runners/session
|
|
beam/runners/spark
|
Package spark contains the Spark runner. |
beam/runners/universal
|
Package universal contains a general-purpose runner that can submit jobs to any portable Beam runner. |
beam/runners/universal/extworker
|
Package extworker provides an external worker service and related utilities. |
beam/runners/universal/runnerlib
|
Package runnerlib contains utilities for submitting Go pipelines to a Beam model runner. |
beam/runners/vet
|
Package vet is a Beam runner that "runs" a pipeline by producing generated code to avoid symbol table lookups and reflection in pipeline execution. |
beam/runners/vet/testpipeline
|
Package testpipeline exports small test pipelines for testing the vet runner. |
beam/testing/passert
|
Package passert contains verification transformations for testing pipelines. |
beam/testing/ptest
|
Package ptest contains utilities for pipeline unit testing. |
beam/transforms/filter
|
Package filter contains transformations for removing pipeline elements based on various conditions. |
beam/transforms/stats
|
Package stats contains transforms for statistical processing. |
beam/transforms/top
|
Package top contains transformations for finding the smallest (or largest) N elements based on arbitrary orderings. |
beam/util/errorx
|
Package errorx contains utilities for handling errors. |
beam/util/execx
|
Package execx contains wrappers and utilities for the exec package. |
beam/util/gcsx
|
Package gcsx contains utilities for working with Google Cloud Storage (GCS). |
beam/util/grpcx
|
Package grpcx contains utilities for working with gRPC. |
beam/util/pubsubx
|
Package pubsubx contains utilities for working with Google PubSub. |
beam/util/shimx
|
Package shimx specifies the templates for generating type assertion shims for Apache Beam Go SDK pipelines. |
beam/util/starcgenx
|
Package starcgenx is a Static Analysis Type Assertion shim and Registration Code Generator which provides an extractor to extract types from a package, in order to generate approprate shimsr a package so code can be generated for it. |
beam/util/syscallx
|
Package syscallx provides system call utilities that attempt to hide platform differences. |
beam/x/beamx
|
Package beamx is a convenience package for beam. |
beam/x/debug
|
Package debug contains pipeline components that may help in debugging pipeline issues. |
beam/x/hooks/perf
|
Package perf is to add performance measuring hooks to a runner, such as cpu, heap, or trace profiles. |