Documentation

Overview

    Package textio contains transforms for reading and writing text files.

    Index

    Constants

    This section is empty.

    Variables

    This section is empty.

    Functions

    func Immediate

    func Immediate(s beam.Scope, filename string) (beam.PCollection, error)

      Immediate reads a local file at pipeline construction-time and embeds the data into a I/O-free pipeline source. Should be used for small files only.

      func Read

      func Read(s beam.Scope, glob string) beam.PCollection

        Read reads a set of file and returns the lines as a PCollection<string>. The newlines are not part of the lines.

        func ReadAll

        func ReadAll(s beam.Scope, col beam.PCollection) beam.PCollection

          ReadAll expands and reads the filename given as globs by the incoming PCollection<string>. It returns the lines of all files as a single PCollection<string>. The newlines are not part of the lines.

          func ReadAllSdf

          func ReadAllSdf(s beam.Scope, col beam.PCollection) beam.PCollection

            ReadAllSdf is a variation of ReadAll implemented via SplittableDoFn. This should result in increased performance with runners that support splitting.

            func ReadSdf

            func ReadSdf(s beam.Scope, glob string) beam.PCollection

              ReadSdf is a variation of Read implemented via SplittableDoFn. This should result in increased performance with runners that support splitting.

              func Write

              func Write(s beam.Scope, filename string, col beam.PCollection)

                Write writes a PCollection<string> to a file as separate lines. The writer add a newline after each element.

                Types

                This section is empty.

                Source Files