Documentation ¶
Overview ¶
Package pipelinex contains utilities for manipulating Beam proto pipelines. The utilities generally uses shallow copies and do not mutate their inputs.
Index ¶
- func Bounded(p *pipepb.Pipeline) bool
- func ContainerImages(p *pipepb.Pipeline) []string
- func Normalize(p *pipepb.Pipeline) (*pipepb.Pipeline, error)
- func ShallowCloneFunctionSpec(p *pipepb.FunctionSpec) *pipepb.FunctionSpec
- func ShallowClonePTransform(t *pipepb.PTransform) *pipepb.PTransform
- func ShallowCloneParDoPayload(p *pipepb.ParDoPayload) *pipepb.ParDoPayload
- func ShallowCloneSideInput(p *pipepb.SideInput) *pipepb.SideInput
- func TopologicalSort(xforms map[string]*pipepb.PTransform, ids []string) []string
- func TrimCoders(coders map[string]*pipepb.Coder, ids ...string) map[string]*pipepb.Coder
- func Update(p *pipepb.Pipeline, values *pipepb.Components) (*pipepb.Pipeline, error)
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
func Bounded ¶
Bounded returns true iff all PCollections are bounded.
func ContainerImages ¶
ContainerImages returns the set of container images used in the given pipeline.
func Normalize ¶
Normalize recomputes derivative information in the pipeline, such as roots and input/output for composite transforms. It also ensures that unique names are so and topologically sorts each subtransform list.
func ShallowCloneFunctionSpec ¶
func ShallowCloneFunctionSpec(p *pipepb.FunctionSpec) *pipepb.FunctionSpec
ShallowCloneFunctionSpec makes a shallow copy of the given FunctionSpec.
func ShallowClonePTransform ¶
func ShallowClonePTransform(t *pipepb.PTransform) *pipepb.PTransform
ShallowClonePTransform makes a shallow copy of the given PTransform.
func ShallowCloneParDoPayload ¶
func ShallowCloneParDoPayload(p *pipepb.ParDoPayload) *pipepb.ParDoPayload
ShallowCloneParDoPayload makes a shallow copy of the given ParDoPayload.
func ShallowCloneSideInput ¶
ShallowCloneSideInput makes a shallow copy of the given SideInput.
func TopologicalSort ¶
func TopologicalSort(xforms map[string]*pipepb.PTransform, ids []string) []string
TopologicalSort returns a topologically sorted list of the given ids, generally from the same scope/composite. Assumes acyclic graph.
func TrimCoders ¶
TrimCoders returns the transitive closure of the given coders ids.
Types ¶
This section is empty.