v2.5.0+incompatible Latest Latest

This package is not in the latest version of its module.

Go to latest
Published: Jun 16, 2018 License: Apache-2.0, BSD-3-Clause, MIT Imports: 8 Imported by: 0



Package metrics implements the Beam metrics API, described at

Metrics in the Beam model are uniquely identified by a namespace, a name, and the PTransform context in which they are used. Further, they are reported as a delta against the bundle being processed, so that overcounting doesn't occur if a bundle needs to be retried. Each metric is scoped to their bundle, and ptransform.

Cells (or metric cells) are defined for each Beam model metric type, and the serve as concurrency safe storage of a given metric's values. Proxys are exported values representing the metric, for use in user ptransform code. They don't retain their cells, since they don't have the context to be able to store them for export back to the pipeline runner.

Metric cells aren't initialized until their first mutation, which follows from the Beam model design, where metrics are only sent for a bundle if they have changed. This is particularly convenient for distributions which means their min and max fields can be set to the first value on creation rather than have some marker of uninitialized state, which would otherwise need to be checked for on every update.

Metric values are implemented as lightweight proxies of the user provided namespace and name. This allows them to be declared globally, and used in any ParDo. Further, as per the design, they can be declared dynamically at runtime.

To handle reporting deltas on the metrics by bundle, metrics are keyed by bundleID,PTransformID,namespace, and name, so metrics that are identical except for bundles are treated as distinct, effectively providing per bundle deltas, since a new value cell is used per bundle.



This section is empty.


This section is empty.


func Clear

func Clear()

Clear resets all storage associated with metrics for tests. Calling this in pipeline code leads to inaccurate metrics.

func ClearBundleData

func ClearBundleData(b string)

ClearBundleData removes stored references associated with a given bundle, so it can be garbage collected.

func DumpToLog

func DumpToLog(ctx context.Context)

DumpToLog is a debugging function that outputs all metrics available locally to beam.Log.

func DumpToOut

func DumpToOut()

DumpToOut is a debugging function that outputs all metrics available locally to std out.

func SetBundleID

func SetBundleID(ctx context.Context, id string) context.Context

SetBundleID sets the id of the current Bundle.

func SetPTransformID

func SetPTransformID(ctx context.Context, id string) context.Context

SetPTransformID sets the id of the current PTransform.

func ToProto

func ToProto(b, pt string) []*fnexecution_v1.Metrics_User

ToProto exports all collected metrics for the given BundleID and PTransform ID pair.


type Counter

type Counter struct {
	// contains filtered or unexported fields

Counter is a simple counter for incrementing and decrementing a value.

func NewCounter

func NewCounter(ns, n string) Counter

NewCounter returns the Counter with the given namespace and name.

func (Counter) Dec

func (m Counter) Dec(ctx context.Context, v int64)

Dec decrements the counter within the given PTransform context by v.

func (Counter) Inc

func (m Counter) Inc(ctx context.Context, v int64)

Inc increments the counter within the given PTransform context by v.

func (Counter) String

func (m Counter) String() string

type Distribution

type Distribution struct {
	// contains filtered or unexported fields

Distribution is a simple distribution of values.

func NewDistribution

func NewDistribution(ns, n string) Distribution

NewDistribution returns the Distribution with the given namespace and name.

func (Distribution) String

func (m Distribution) String() string

func (Distribution) Update

func (m Distribution) Update(ctx context.Context, v int64)

Update updates the distribution within the given PTransform context with v.

type Gauge

type Gauge struct {
	// contains filtered or unexported fields

Gauge is a time, value pair metric.

func NewGauge

func NewGauge(ns, n string) Gauge

NewGauge returns the Gauge with the given namespace and name.

func (Gauge) Set

func (m Gauge) Set(ctx context.Context, v int64)

Set sets the gauge to the given value, and associates it with the current time on the clock.

func (Gauge) String

func (m Gauge) String() string

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL