shimx

Documentation

Overview

Package shimx specifies the templates for generating type assertion shims for Apache Beam Go SDK pipelines.

In particular, the shims are used by the Beam Go SDK to avoid reflection at runtime, which is the default mode of operation. The shims are specialized for the code in question, using type assertion to convert arguments as required, and invoke the user code.

Similar shims are required for emitters, and iterators in order to propagate values out of, and in to user functions respectively without reflection overhead.

Registering user types is required to support user types as PCollection types, while registering functions is required to avoid possibly expensive function resolution at worker start up, which defaults to using DWARF Symbol tables.

The generator largely relies on basic types and strings to ensure that it's usable by dynamic processes via reflection, or by any static analysis approach that is used in the future.

Index

• Variables
• func File(w io.Writer, top *Top)
• func FuncName(inNames, outNames []string) string
• func Name(t string) string
• type Emitter
• type Func
• type Input
• type Top
• type Wrap

Variables

var (
	ExecImport     = "github.com/apache/beam/sdks/go/pkg/beam/core/runtime/exec"
	TypexImport    = "github.com/apache/beam/sdks/go/pkg/beam/core/typex"
	ReflectxImport = "github.com/apache/beam/sdks/go/pkg/beam/core/util/reflectx"
	RuntimeImport  = "github.com/apache/beam/sdks/go/pkg/beam/core/runtime"
	SchemaImport   = "github.com/apache/beam/sdks/go/pkg/beam/core/runtime/graphx/schema"
)

Beam imports that the generated code requires.

Functions

func File

func File(w io.Writer, top *Top)

File writes go code to the given writer.

func FuncName

func FuncName(inNames, outNames []string) string

FuncName returns a compilable Go identifier for a function, given valid type names as generated by Name. See https://golang.org/ref/spec#Identifiers for details.

func Name

func Name(t string) string

Name creates a capitalized identifier from a type string. The identifier follows the rules of go identifiers and should be compileable. See https://golang.org/ref/spec#Identifiers for details.

Types

type Emitter

type Emitter struct {
	Name, Type string // The user name of the function, the type of the emit.
	Time       bool   // if this uses event time.
	Key, Val   string // The type of the emits.
}

Emitter represents an emitter shim to be generated.

type Func

type Func struct {
	Name, Type string
	In, Out    []string
}

Func represents a type assertion shim for function invocation to be generated.

type Input

type Input struct {
	Name, Type string // The user name of the function, the type of the iterator (including the bool).
	Time       bool   // if this uses event time.
	Key, Val   string // The type of the inputs, pointers removed.
}

Input represents an iterator shim to be generated.

type Top

type Top struct {
	FileName, ToolName, Package string

	Imports   []string // the full import paths
	Functions []string // the plain names of the functions to be registered.
	Types     []string // the plain names of the types to be registered.
	Wraps     []Wrap
	Emitters  []Emitter
	Inputs    []Input
	Shims     []Func
}

Top is the top level inputs into the template file for generating shims.

type Wrap

type Wrap struct {
	Name, Type string
	Methods    []Func
}

Wrap represents a type assertion shim for Structural DoFn method invocation to be generated.