Documentation
¶
Overview ¶
Package primitives contains integration tests for primitives in beam.
Index ¶
- func CoGBK() *beam.Pipeline
- func CreateAndSplit(s beam.Scope, input []stringPair) beam.PCollection
- func Flatten() *beam.Pipeline
- func FlattenDup() *beam.Pipeline
- func Panes(s beam.Scope)
- func PanesFn(pn beam.PaneInfo, value float64, emit func(int))
- func ParDoKVSideInput() *beam.Pipeline
- func ParDoMultiMapSideInput() *beam.Pipeline
- func ParDoMultiOutput() *beam.Pipeline
- func ParDoPipelineOptions() *beam.Pipeline
- func ParDoSideInput() *beam.Pipeline
- func Reshuffle() *beam.Pipeline
- func ReshuffleKV() *beam.Pipeline
- func TestStreamBoolSequence() *beam.Pipeline
- func TestStreamByteSliceSequence() *beam.Pipeline
- func TestStreamFloat64Sequence() *beam.Pipeline
- func TestStreamInt64Sequence() *beam.Pipeline
- func TestStreamStrings() *beam.Pipeline
- func TestStreamTwoBoolSequences() *beam.Pipeline
- func TestStreamTwoFloat64Sequences() *beam.Pipeline
- func TestStreamTwoInt64Sequences() *beam.Pipeline
- func TriggerAfterEndOfWindow(s beam.Scope)
- func TriggerAfterProcessingTime(s beam.Scope)
- func TriggerAlways(s beam.Scope)
- func TriggerDefault(s beam.Scope)
- func TriggerElementCount(s beam.Scope)
- func TriggerRepeat(s beam.Scope)
- func ValidateWindowedSideInputs(s beam.Scope)
- func WindowSums(s beam.Scope, sumPerKey func(beam.Scope, beam.PCollection) beam.PCollection)
- func WindowSums_GBK(s beam.Scope)
- func WindowSums_Lifted(s beam.Scope)
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
func CreateAndSplit ¶ added in v2.35.0
func CreateAndSplit(s beam.Scope, input []stringPair) beam.PCollection
CreateAndSplit makes a KV PCollection from a list of stringPair types
func FlattenDup ¶
FlattenDups tests flatten with the same input multiple times.
func Panes ¶ added in v2.37.0
Panes constructs a teststream and applies a pardo to get the pane timings.
func ParDoKVSideInput ¶
ParDoKVSideInput computes the sum of ints using a KV side input.
func ParDoMultiMapSideInput ¶ added in v2.35.0
ParDoMultiMapSideInput checks that the multimap side input access pattern works correctly, properly producing the correct output with an asymmetric join.
func ParDoMultiOutput ¶
ParDoMultiOutput test a DoFn with multiple output.
func ParDoPipelineOptions ¶ added in v2.37.0
ParDoPipelineOptions creates a pipeline with flag options to validate that a DoFn can access them as PipelineOptions.
func ParDoSideInput ¶
ParDoSideInput computes the sum of ints using a side input.
func ReshuffleKV ¶
ReshuffleKV tests Reshuffle with KV PCollections.
func TestStreamBoolSequence ¶
TestStreamBoolSequence tests the TestStream primitive by inserting boolean elements then advancing the watermark past the point where they were inserted.
func TestStreamByteSliceSequence ¶
TestStreamByteSliceSequence tests the TestStream primitive by inserting byte slice elements then advancing the watermark to infinity and comparing the output..
func TestStreamFloat64Sequence ¶
TestStreamFloat64Sequence tests the TestStream primitive by inserting float64 elements then advancing the watermark past the point where they were inserted.
func TestStreamInt64Sequence ¶
TestStreamInt64Sequence tests the TestStream primitive by inserting int64 elements then advancing the watermark past the point where they were inserted.
func TestStreamStrings ¶
TestStreamSequence tests the TestStream primitive by inserting string elements then advancing the watermark past the point where they were inserted.
func TestStreamTwoBoolSequences ¶
TestStreamTwoBoolSequences tests the TestStream primitive by inserting two sets of boolean elements that arrive on-time into the TestStream
func TestStreamTwoFloat64Sequences ¶
TestStreamTwoFloat64Sequences tests the TestStream primitive by inserting two sets of float64 elements that arrive on-time into the TestStream
func TestStreamTwoInt64Sequences ¶
TestStreamTwoInt64Sequences tests the TestStream primitive by inserting two sets of int64 elements that arrive on-time into the TestStream
func TriggerAfterEndOfWindow ¶ added in v2.34.0
TriggerAfterEndOfWindow tests the AfterEndOfWindow Trigger. With AfterCount(2) as the early firing trigger and AfterCount(1) as late firing trigger. It fires two times, one with early firing when there are two elements while the third elements waits in. This third element is fired in the late firing.
func TriggerAfterProcessingTime ¶
TriggerAfterProcessingTime tests the AfterProcessingTime Trigger, it fires output panes once 't' processing time has passed Not yet supported by the flink runner: java.lang.UnsupportedOperationException: Advancing Processing time is not supported by the Flink Runner.
func TriggerAlways ¶
TriggerAlways tests the Always trigger, it is expected to receive every input value as the output.
func TriggerDefault ¶
TriggerDefault tests the default trigger which fires the pane after the end of the window
func TriggerElementCount ¶
TriggerElementCount tests the ElementCount Trigger, it waits for atleast N elements to be ready to fire an output pane
func TriggerRepeat ¶
TriggerRepeat tests the repeat trigger. As of now is it is configure to take only one trigger as a subtrigger. In the below test, it is expected to receive three output panes with two elements each.
func ValidateWindowedSideInputs ¶ added in v2.35.0
ValidateWindowedSideInputs checks that side inputs have accurate windowing information when used.
func WindowSums ¶
func WindowSums(s beam.Scope, sumPerKey func(beam.Scope, beam.PCollection) beam.PCollection)
WindowSums produces a pipeline that generates the numbers of a 3x3 magic square, and configures the pipeline so that PCollection. Sum is a closure to handle summing data over the window, in a few conditions.
func WindowSums_GBK ¶
func WindowSums_Lifted ¶
Types ¶
This section is empty.
Source Files