textio

package
v2.27.0-RC2+incompatible Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Dec 28, 2020 License: Apache-2.0 Imports: 12 Imported by: 0

Documentation

Overview

Package textio contains transforms for reading and writing text files.

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

func Immediate

func Immediate(s beam.Scope, filename string) (beam.PCollection, error)

Immediate reads a local file at pipeline construction-time and embeds the data into a I/O-free pipeline source. Should be used for small files only.

func Read

func Read(s beam.Scope, glob string) beam.PCollection

Read reads a set of file and returns the lines as a PCollection<string>. The newlines are not part of the lines.

func ReadAll

func ReadAll(s beam.Scope, col beam.PCollection) beam.PCollection

ReadAll expands and reads the filename given as globs by the incoming PCollection<string>. It returns the lines of all files as a single PCollection<string>. The newlines are not part of the lines.

func ReadAllSdf

func ReadAllSdf(s beam.Scope, col beam.PCollection) beam.PCollection

ReadAllSdf is a variation of ReadAll implemented via SplittableDoFn. This should result in increased performance with runners that support splitting.

func ReadSdf

func ReadSdf(s beam.Scope, glob string) beam.PCollection

ReadSdf is a variation of Read implemented via SplittableDoFn. This should result in increased performance with runners that support splitting.

func Write

func Write(s beam.Scope, filename string, col beam.PCollection)

Write writes a PCollection<string> to a file as separate lines. The writer add a newline after each element.

Types

This section is empty.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL