dataframe

Documentation

Overview

Package dataframe is a wrapper for DataframeTransform defined in Apache Beam Python SDK. An exapnsion service for python external transforms can be started by running $ python -m apache_beam.runners.portability.expansion_service_main -p $PORT_FOR_EXPANSION_SERVICE

Index

• func Transform(s beam.Scope, fn string, col beam.PCollection, outT reflect.Type, ...) beam.PCollection
• func WithExpansionAddr(expansionAddr string) configOption
• func WithIndexes() configOption

Functions

func Transform

func Transform(s beam.Scope, fn string, col beam.PCollection, outT reflect.Type, opts ...configOption) beam.PCollection

Transform is a multi-language wrapper for a Python DataframeTransform with a given lambda function. lambda function is a required parameter. Additional option for including indexes in dataframe can be provided by using dataframe.WithIndexes().

func WithExpansionAddr

func WithExpansionAddr(expansionAddr string) configOption

WithExpansionAddr sets an URL for a Python expansion service.

func WithIndexes

func WithIndexes() configOption

WithIndexes sets include_indexes option for DataframeTransform.