filter

package
v2.25.0+incompatible Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Oct 20, 2020 License: Apache-2.0 Imports: 8 Imported by: 0

Documentation

Overview

Package filter contains transformations for removing pipeline elements based on various conditions.

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

func Distinct

func Distinct(s beam.Scope, col beam.PCollection) beam.PCollection

Distinct removes all duplicates from a collection, under coder equality. It expects a PCollection<T> as input and returns a PCollection<T> with duplicates removed.

func Exclude

func Exclude(s beam.Scope, col beam.PCollection, fn interface{}) beam.PCollection

Exclude filters the elements of a PCollection<A> based on the given function, which must be of the form: A -> bool. Exclude removes all element for which the filter function returns true. It returns a PCollection of the same type as the input. For example:

words := beam.Create(s, "a", "b", "long", "alsolong")
long := filter.Exclude(s, words, func(s string) bool {
    return len(s) < 3
})

Here, "long" will contain "long" and "alsolong" at runtime.

func Include

func Include(s beam.Scope, col beam.PCollection, fn interface{}) beam.PCollection

Include filters the elements of a PCollection<A> based on the given function, which must be of the form: A -> bool. Include removes all element for which the filter function returns false. It returns a PCollection of the same type as the input. For example:

words := beam.Create(s, "a", "b", "long", "alsolong")
short := filter.Include(s, words, func(s string) bool {
    return len(s) < 3
})

Here, "short" will contain "a" and "b" at runtime.

Types

This section is empty.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL