pubsubio

Documentation

Overview

Package pubsubio provides access to Pub/Sub on Dataflow streaming.

This implementation only functions on the Dataflow runner.

See https://cloud.google.com/dataflow/docs/concepts/streaming-with-cloud-pubsub for details on using Pub/Sub with Dataflow.

Index

• func Read(s beam.Scope, project, topic string, opts *ReadOptions) beam.PCollection
• func Write(s beam.Scope, project, topic string, col beam.PCollection)
• type ReadOptions

Functions

func Read

func Read(s beam.Scope, project, topic string, opts *ReadOptions) beam.PCollection

Read reads an unbounded number of PubSubMessages from the given pubsub topic. It produces an unbounded PCollecton<*PubSubMessage>, if WithAttributes is set, or an unbounded PCollection<[]byte>.

func Write

func Write(s beam.Scope, project, topic string, col beam.PCollection)

Write writes PubSubMessages or []bytes to the given pubsub topic. Panics if the input pcollection type is not one of those two types.

When given []bytes, they are first wrapped in PubSubMessages.

Note: Doesn't function in batch pipelines.

Types

type ReadOptions

type ReadOptions struct {
	Subscription       string
	IDAttribute        string
	TimestampAttribute string
	WithAttributes     bool
}

ReadOptions represents options for reading from PubSub.