Documentation

Overview

    Package filter contains transformations for removing pipeline elements based on various conditions.

    Index

    Constants

    This section is empty.

    Variables

    This section is empty.

    Functions

    func Distinct

    func Distinct(s beam.Scope, col beam.PCollection) beam.PCollection

      Distinct removes all duplicates from a collection, under coder equality. It expects a PCollection<T> as input and returns a PCollection<T> with duplicates removed.

      func Exclude

      func Exclude(s beam.Scope, col beam.PCollection, fn interface{}) beam.PCollection

        Exclude filters the elements of a PCollection<A> based on the given function, which must be of the form: A -> bool. Exclude removes all element for which the filter function returns true. It returns a PCollection of the same type as the input. For example:

        words := beam.Create(s, "a", "b", "long", "alsolong")
        long := filter.Exclude(s, words, func(s string) bool {
            return len(s) < 3
        })
        

        Here, "long" will contain "long" and "alsolong" at runtime.

        func Include

        func Include(s beam.Scope, col beam.PCollection, fn interface{}) beam.PCollection

          Include filters the elements of a PCollection<A> based on the given function, which must be of the form: A -> bool. Include removes all element for which the filter function returns false. It returns a PCollection of the same type as the input. For example:

          words := beam.Create(s, "a", "b", "long", "alsolong")
          short := filter.Include(s, words, func(s string) bool {
              return len(s) < 3
          })
          

          Here, "short" will contain "a" and "b" at runtime.

          Types

          This section is empty.