resource

Documentation

Overview

Package resource supports Beam resource hints to specify scoped hints or annotations to pipelines.

See https://beam.apache.org/documentation/runtime/resource-hints/ for more information.

Index

• type Hint
  • func Accelerator(v string) Hint
  • func MinRAMBytes(v uint64) Hint
  • func ParseMinRAM(v string) Hint
• type Hints
  • func NewHints(hs ...Hint) Hints
  • func (hs Hints) Equal(other Hints) bool
  • func (hs Hints) MergeWithOuter(outer Hints) Hints
  • func (hs Hints) Payloads() map[string][]byte

Types

type Hint

type Hint interface {
	// URN returns the name for this hint.
	URN() string
	// Payload returns the serialized version of this payload.
	Payload() []byte
	// MergeWithOuter an outer scope hint.
	MergeWithOuter(outer Hint) Hint
}

Hint contains all the information about a given resource hint.

func Accelerator

func Accelerator(v string) Hint

Accelerator hints that this scope should be put in a machine with a given accelerator.

Hints for accelerators will have formats that are runner specific. For example, the following is valid accelerator syntax for the Dataflow runner:

accelerator="type:<type>;count:<n>;<options>"

Hints are advisory only and runners may not respect them.

See https://beam.apache.org/documentation/runtime/resource-hints/ for more information about resource hints.

func MinRAMBytes

func MinRAMBytes(v uint64) Hint

MinRAMBytes hints that this scope should be put in a machine with at least this many bytes of memory.

Hints are advisory only and runners may not respect them.

See https://beam.apache.org/documentation/runtime/resource-hints/ for more information about resource hints.

func ParseMinRAM

func ParseMinRAM(v string) Hint

ParseMinRAM converts various byte units, including MB, GB, MiB, and GiB into a hint. An invalid byte size format will cause ParseMinRAM to panic.

Hints are advisory only and runners may not respect them.

See https://beam.apache.org/documentation/runtime/resource-hints/ for more information about resource hints.

type Hints

type Hints struct {
	// contains filtered or unexported fields
}

Hints contains a list of hints for a given scope.

func NewHints

func NewHints(hs ...Hint) Hints

NewHints produces a hints map from a list of hints. If there are multiple hints with the same URN, the last one in the list is used.

func (Hints) Equal

func (hs Hints) Equal(other Hints) bool

Equal checks if two sets of hints are identical. A hint is identical to another if their payloads are the same for a given URN.

func (Hints) MergeWithOuter

func (hs Hints) MergeWithOuter(outer Hints) Hints

MergeWithOuter produces a new list of Hints from this Hints, and the Hints from the outer scope. Semantics are defined per hint urn, but by default the "inner" hint will be prefered over the outer hint if both scopes have the same urn.

func (Hints) Payloads

func (hs Hints) Payloads() map[string][]byte

Payloads retuns a map from all hint URNs to the serialized byte representation of their payloads.