v1.5.0 Latest Latest

This package is not in the latest version of its module.

Go to latest
Published: Apr 7, 2022 License: Apache-2.0 Imports: 31 Imported by: 266



Package instrument implements functions to make instrumenting code, including metrics and logging, easier.



View Source
const (
	// InvariantViolatedMetricName is the name of the metric emitted upon
	// invocation of `EmitInvariantViolation`.
	InvariantViolatedMetricName = "invariant_violated"

	// InvariantViolatedLogFieldName is the name of the log field to be
	// used when generating errors/log statements pertaining to the violation
	// of an invariant.
	InvariantViolatedLogFieldName = "violation"

	// InvariantViolatedLogFieldValue is the value of the log field to be
	// used when generating errors/log statements pertaining to the violation
	// of an invariant.
	InvariantViolatedLogFieldValue = InvariantViolatedMetricName

	// ShouldPanicEnvironmentVariableName is the name of the environment variable
	// that must be set to "true" in order for the invariant violated functions
	// to panic after logging / emitting metrics. Should only be set in test
	// environments.
	ShouldPanicEnvironmentVariableName = "PANIC_ON_INVARIANT_VIOLATED"


View Source
var (
	// Revision is the VCS revision associated with this build. Overridden using ldflags
	// at compile time. Example:
	// $ go build -ldflags "-X" ...
	// Adapted from:
	Revision = "unknown"

	// Branch is the VCS branch associated with this build.
	Branch = "unknown"

	// Version is the version associated with this build.
	Version = "unknown"

	// BuildDate is the date this build was created.
	BuildDate = "unknown"

	// BuildTimeUnix is the seconds since epoch representing the date this build was created.
	BuildTimeUnix = "0"

	// LogBuildInfoAtStartup controls whether we log build information at startup. If its
	// set to a non-empty string, we log the build information at process startup.
	LogBuildInfoAtStartup string

	// LogBuildInfoToStdout controls whether we log build information to stdout or stderr.
	// If it is set to a non-empty string then the build info will be logged to stdout,
	// otherwise it will be logged to stderr (assuming LogBuildInfoAtStartup is also
	// non-empty).
	LogBuildInfoToStdout string


func DefaultHistogramTimerHistogramBuckets added in v0.15.0

func DefaultHistogramTimerHistogramBuckets() tally.Buckets

DefaultHistogramTimerHistogramBuckets returns a set of default histogram timer histogram buckets, from 2ms up to 1hr.

func DefaultSummaryQuantileObjectives added in v0.15.0

func DefaultSummaryQuantileObjectives() map[float64]float64

DefaultSummaryQuantileObjectives is a set of default summary quantile objectives and allowed error thresholds.

func EmitAndLogInvariantViolation

func EmitAndLogInvariantViolation(opts Options, f func(l *zap.Logger))

EmitAndLogInvariantViolation calls EmitInvariantViolation and then calls the provided function with a supplied logger that is pre-configured with an invariant violated field. Optionally panics if the ShouldPanicEnvironmentVariableName is set to "true".

func EmitInvariantViolation

func EmitInvariantViolation(opts Options)

EmitInvariantViolation emits a metric to indicate a system invariant has been violated. Users of this method are expected to monitor/alert off this metric to ensure they're notified when such an event occurs. Further, they should log further information to aid diagnostics of the system invariant violated at the callsite of the violation. Optionally panics if the ShouldPanicEnvironmentVariableName is set to "true".

func InvariantErrorf

func InvariantErrorf(format string, a ...interface{}) error

InvariantErrorf constructs a new error, prefixed with a string indicating that an invariant violation occurred. Optionally panics if the ShouldPanicEnvironmentVariableName is set to "true".

func LogBuildInfo

func LogBuildInfo()

LogBuildInfo logs the build information using the default logger.

func LogBuildInfoWithLogger added in v1.4.2

func LogBuildInfoWithLogger(logger *log.Logger)

LogBuildInfoWithLogger logs the build information using the provided logger.

func MustCreateSampledTimer

func MustCreateSampledTimer(base tally.Timer, rate float64) tally.Timer

MustCreateSampledTimer creates a new sampled timer, and panics if an error is encountered.

func NewPrometheusProcessCollector added in v0.9.3

func NewPrometheusProcessCollector(opts ProcessCollectorOpts) prometheus.Collector

NewPrometheusProcessCollector returns a collector which exports the current state of process metrics including CPU, memory and file descriptor usage as well as the process start time. The detailed behavior is defined by the provided ProcessCollectorOpts. The zero value of ProcessCollectorOpts creates a collector for the current process with an empty namespace string and no error reporting.

Currently, the collector depends on a Linux-style proc filesystem and therefore only exports metrics for Linux.

NB(r): This version of the Prometheus process collector allows skipping emitting open FDs due to excessive load reporting open FDs with processes with a large number of open FDs.

func NewSampledTimer

func NewSampledTimer(base tally.Timer, rate float64) (tally.Timer, error)

NewSampledTimer creates a new sampled timer.

func NewTestDebugLogger added in v0.15.0

func NewTestDebugLogger(t *testing.T) *zap.Logger

NewTestDebugLogger returns a new debug logger for tests.

func NewTimer added in v0.15.0

func NewTimer(scope tally.Scope, name string, opts TimerOptions) tally.Timer

NewTimer returns a new timer that is backed by a timer or a histogram based on the timer options.

func SetShouldPanicEnvironmentVariable added in v1.2.0

func SetShouldPanicEnvironmentVariable(value bool) func()

SetShouldPanicEnvironmentVariable sets the env variable and returns a func to reset to the previous value. Useful for tests to use a defer statement when they need to test a specific value.

func SparseHistogramTimerHistogramBuckets added in v1.0.1

func SparseHistogramTimerHistogramBuckets() tally.Buckets

SparseHistogramTimerHistogramBuckets returns a small spare set of histogram timer histogram buckets, from 1ms up to 8m.


type BatchMethodMetrics

type BatchMethodMetrics struct {
	RetryableErrors    tally.Counter
	NonRetryableErrors tally.Counter
	Errors             tally.Counter
	Success            tally.Counter
	Latency            tally.Timer

BatchMethodMetrics is a bundle of common metrics for methods with batch semantics.

func NewBatchMethodMetrics

func NewBatchMethodMetrics(
	scope tally.Scope,
	methodName string,
	opts TimerOptions,
) BatchMethodMetrics

NewBatchMethodMetrics creates new batch method metrics.

func (*BatchMethodMetrics) ReportLatency

func (m *BatchMethodMetrics) ReportLatency(d time.Duration)

ReportLatency reports latency.

func (*BatchMethodMetrics) ReportNonRetryableErrors

func (m *BatchMethodMetrics) ReportNonRetryableErrors(n int)

ReportNonRetryableErrors reports non-retryable errors.

func (*BatchMethodMetrics) ReportRetryableErrors

func (m *BatchMethodMetrics) ReportRetryableErrors(n int)

ReportRetryableErrors reports retryable errors.

func (*BatchMethodMetrics) ReportSuccess

func (m *BatchMethodMetrics) ReportSuccess(n int)

ReportSuccess reports successess.

type ExtendedMetricsType

type ExtendedMetricsType int

ExtendedMetricsType is a type of extended metrics to report.

const (
	// NoExtendedMetrics describes no extended metrics.
	NoExtendedMetrics ExtendedMetricsType = iota

	// SimpleExtendedMetrics describes just a simple level of extended metrics:
	// - number of active goroutines
	// - number of configured gomaxprocs

	// ModerateExtendedMetrics describes a moderately verbose level of extended metrics:
	// - number of active goroutines
	// - number of configured gomaxprocs
	// - number of file descriptors

	// DetailedExtendedMetrics describes a detailed level of extended metrics:
	// - number of active goroutines
	// - number of configured gomaxprocs
	// - number of file descriptors
	// - memory allocated running count
	// - memory used by heap
	// - memory used by heap that is idle
	// - memory used by heap that is in use
	// - memory used by stack
	// - number of garbage collections
	// - GC pause times

	// DetailedGoRuntimeMetrics reports all detailed metrics, sans FD metrics to save CPU
	// if in-use file descriptors are measured by an external system, like cAdvisor.

	// DefaultExtendedMetricsType is the default extended metrics level.
	DefaultExtendedMetricsType = SimpleExtendedMetrics

func (ExtendedMetricsType) String

func (t ExtendedMetricsType) String() string

func (*ExtendedMetricsType) UnmarshalYAML

func (t *ExtendedMetricsType) UnmarshalYAML(unmarshal func(interface{}) error) error

UnmarshalYAML unmarshals an ExtendedMetricsType into a valid type from string.

type HistogramObjective added in v0.15.0

type HistogramObjective struct {
	Upper float64 `yaml:"upper"`

HistogramObjective is a Prometheus histogram bucket. See:

type HistogramTimerOptions added in v0.15.0

type HistogramTimerOptions struct {
	HistogramBuckets tally.Buckets

HistogramTimerOptions is a set of histogram timer options.

type MethodMetrics

type MethodMetrics struct {
	Errors         tally.Counter
	Success        tally.Counter
	ErrorsLatency  tally.Timer
	SuccessLatency tally.Timer

MethodMetrics is a bundle of common metrics with a uniform naming scheme.

func NewMethodMetrics

func NewMethodMetrics(scope tally.Scope, methodName string, opts TimerOptions) MethodMetrics

NewMethodMetrics returns a new Method metrics for the given method name.

func (*MethodMetrics) ReportError

func (m *MethodMetrics) ReportError(d time.Duration)

ReportError reports an error.

func (*MethodMetrics) ReportSuccess

func (m *MethodMetrics) ReportSuccess(d time.Duration)

ReportSuccess reports a success.

func (*MethodMetrics) ReportSuccessOrError

func (m *MethodMetrics) ReportSuccessOrError(e error, d time.Duration)

ReportSuccessOrError increments Error/Success counter dependending on the error.

type MetricSanitizationType

type MetricSanitizationType int

MetricSanitizationType is a type of sanitizer to use for metrics.

const (
	// NoMetricSanitization performs no metric sanitization.
	NoMetricSanitization MetricSanitizationType = iota

	// M3MetricSanitization performs M3 metric sanitization.

	// PrometheusMetricSanitization performs Prometheus metric sanitization.

func (*MetricSanitizationType) NewOptions

func (t *MetricSanitizationType) NewOptions() *tally.SanitizeOptions

NewOptions returns a new set of sanitization options for the sanitization type.

func (MetricSanitizationType) String

func (t MetricSanitizationType) String() string

func (*MetricSanitizationType) UnmarshalYAML

func (t *MetricSanitizationType) UnmarshalYAML(unmarshal func(interface{}) error) error

UnmarshalYAML unmarshals a MetricSanitizationType into a valid type from string.

type MetricsConfiguration

type MetricsConfiguration struct {
	// Root scope configuration.
	RootScope *ScopeConfiguration `yaml:"scope"`

	// M3 reporter configuration.
	M3Reporter *m3.Configuration `yaml:"m3"`

	// Prometheus reporter configuration.
	PrometheusReporter *PrometheusConfiguration `yaml:"prometheus"`

	// Metrics sampling rate.
	SamplingRate float64 `yaml:"samplingRate" validate:"nonzero,min=0.0,max=1.0"`

	// Extended metrics type.
	ExtendedMetrics *ExtendedMetricsType `yaml:"extended"`

	// Metric sanitization type.
	Sanitization *MetricSanitizationType `yaml:"sanitization"`

MetricsConfiguration configures options for emitting metrics.

func (*MetricsConfiguration) NewRootScope

func (mc *MetricsConfiguration) NewRootScope() (tally.Scope, io.Closer, error)

NewRootScope creates a new tally.Scope based on a tally.CachedStatsReporter based on the the the config.

func (*MetricsConfiguration) NewRootScopeAndReporters added in v0.14.0

NewRootScopeAndReporters creates a new tally.Scope based on a tally.CachedStatsReporter based on the the the config along with the reporters used.

func (*MetricsConfiguration) NewRootScopeReporter

func (mc *MetricsConfiguration) NewRootScopeReporter(
	r tally.CachedStatsReporter,
) (tally.Scope, io.Closer)

NewRootScopeReporter creates a new tally.Scope based on a given tally.CachedStatsReporter and given root scope config. In most cases NewRootScope should be used, but for cases such as hooking into the reporter to manually flush it.

func (*MetricsConfiguration) ReportInterval

func (mc *MetricsConfiguration) ReportInterval() time.Duration

ReportInterval returns the metrics reporting interval.

func (*MetricsConfiguration) SampleRate

func (mc *MetricsConfiguration) SampleRate() float64

SampleRate returns the metrics sampling rate.

type MetricsConfigurationM3Reporter added in v0.14.0

type MetricsConfigurationM3Reporter struct {
	Reporter m3.Reporter

MetricsConfigurationM3Reporter is the M3 reporter if constructed.

type MetricsConfigurationPrometheusReporter added in v0.14.0

type MetricsConfigurationPrometheusReporter struct {
	Reporter prometheus.Reporter
	Registry *prom.Registry

MetricsConfigurationPrometheusReporter is the Prometheus reporter if constructed.

type MetricsConfigurationReporters added in v0.14.0

type MetricsConfigurationReporters struct {
	AllReporters       []tally.CachedStatsReporter
	M3Reporter         *MetricsConfigurationM3Reporter
	PrometheusReporter *MetricsConfigurationPrometheusReporter

MetricsConfigurationReporters is the reporters constructed.

type NewRootScopeAndReportersOptions added in v0.15.0

type NewRootScopeAndReportersOptions struct {
	PrometheusHandlerListener    net.Listener
	PrometheusDefaultServeMux    *http.ServeMux
	PrometheusExternalRegistries []PrometheusExternalRegistry
	PrometheusOnError            func(e error)
	// CommonLabels will be appended to every metric gathered.
	CommonLabels map[string]string

NewRootScopeAndReportersOptions is a set of options.

type NoOpProfiler added in v1.2.0

type NoOpProfiler struct{}

NoOpProfiler is default disabled profiler.

func NewNoOpProfiler added in v1.2.0

func NewNoOpProfiler() *NoOpProfiler

NewNoOpProfiler creates a new no-op profiler.

func (NoOpProfiler) StartCPUProfile added in v1.2.0

func (f NoOpProfiler) StartCPUProfile(name string) error

StartCPUProfile starts named cpu profile.

func (NoOpProfiler) StopCPUProfile added in v1.2.0

func (f NoOpProfiler) StopCPUProfile() error

StopCPUProfile stops started cpu profile.

func (NoOpProfiler) WriteHeapProfile added in v1.2.0

func (f NoOpProfiler) WriteHeapProfile(name string) error

WriteHeapProfile writes named heap profile.

type Option added in v1.2.0

type Option func(*Options)

Option applys optional settings to the instrument options.

func WithLoggerTagged added in v1.2.0

func WithLoggerTagged(k, v string) Option

WithLoggerTagged tags the logger in the instrument options.

func WithScopeAndLoggerTagged added in v1.2.0

func WithScopeAndLoggerTagged(k, v string) Option

WithScopeAndLoggerTagged tags both the metric scope and logger in the instrument options.

func WithScopeTagged added in v1.2.0

func WithScopeTagged(k, v string) Option

WithScopeTagged tags the metric scope in the instrument options.

type Options

type Options interface {
	// SetLogger sets the zap logger
	SetLogger(value *zap.Logger) Options

	// Logger returns the zap logger
	Logger() *zap.Logger

	// SetMetricsScope sets the metrics scope.
	SetMetricsScope(value tally.Scope) Options

	// MetricsScope returns the metrics scope.
	MetricsScope() tally.Scope

	// Tracer returns the tracer.
	Tracer() opentracing.Tracer

	// SetTracer sets the tracer.
	SetTracer(tracer opentracing.Tracer) Options

	// SetTimerOptions sets the metrics timer options to used
	// when building timers from timer options.
	SetTimerOptions(value TimerOptions) Options

	// TimerOptions returns the metrics timer options to used
	// when building timers from timer options.
	TimerOptions() TimerOptions

	// SetReportInterval sets the time between reporting metrics within the system.
	SetReportInterval(time.Duration) Options

	// ReportInterval returns the time between reporting metrics within the system.
	ReportInterval() time.Duration

	// SetCustomBuildTags sets custom tags to be added to build report metrics in
	// addition to the defaults.
	SetCustomBuildTags(tags map[string]string) Options

	// CustomBuildTags returns the custom build tags.
	CustomBuildTags() map[string]string

	// SetProfiler sets the profiler.
	SetProfiler(value Profiler) Options

	// Profiler returns the Profiler.
	Profiler() Profiler

Options represents the options for instrumentation.

func NewOptions

func NewOptions() Options

NewOptions creates new instrument options.

func NewTestOptions added in v0.15.0

func NewTestOptions(t *testing.T) Options

NewTestOptions returns a set of instrument options useful for tests. This includes: - Logger built using development config with debug level set.

func WithOptions added in v1.2.0

func WithOptions(iOpts Options, opts ...Option) Options

WithOptions creates a new instrument options with options.

type ProcessCollectorOpts added in v0.9.3

type ProcessCollectorOpts struct {
	// PidFn returns the PID of the process the collector collects metrics
	// for. It is called upon each collection. By default, the PID of the
	// current process is used, as determined on construction time by
	// calling os.Getpid().
	PidFn func() (int, error)
	// If non-empty, each of the collected metrics is prefixed by the
	// provided string and an underscore ("_").
	Namespace string
	// DisableOpenFDs allows disabling the reporting of open FDs due to
	// the cost that is required to report the number of file descriptors.
	DisableOpenFDs bool
	// If true, any error encountered during collection is reported as an
	// invalid metric (see NewInvalidMetric). Otherwise, errors are ignored
	// and the collected metrics will be incomplete. (Possibly, no metrics
	// will be collected at all.) While that's usually not desired, it is
	// appropriate for the common "mix-in" of process metrics, where process
	// metrics are nice to have, but failing to collect them should not
	// disrupt the collection of the remaining metrics.
	ReportErrors bool

ProcessCollectorOpts defines the behavior of a process metrics collector created with NewProcessCollector.

type Profiler added in v1.2.0

type Profiler interface {
	// StartCPUProfile starts the named cpu profile.
	StartCPUProfile(name string) error

	// StopCPUProfile stops started cpu profile.
	StopCPUProfile() error

	// WriteHeapProfile writes heap profile.
	WriteHeapProfile(name string) error

Profiler represents profiler for profiling long-running tasks.

type PrometheusConfiguration added in v0.15.0

type PrometheusConfiguration struct {
	// HandlerPath if specified will be used instead of using the default
	// HTTP handler path "/metrics".
	HandlerPath string `yaml:"handlerPath"`

	// ListenAddress if specified will be used instead of just registering the
	// handler on the default HTTP serve mux without listening.
	ListenAddress string `yaml:"listenAddress"`

	// TimerType is the default Prometheus type to use for Tally timers.
	TimerType string `yaml:"timerType"`

	// DefaultHistogramBuckets if specified will set the default histogram
	// buckets to be used by the reporter.
	DefaultHistogramBuckets []prometheus.HistogramObjective `yaml:"defaultHistogramBuckets"`

	// DefaultSummaryObjectives if specified will set the default summary
	// objectives to be used by the reporter.
	DefaultSummaryObjectives []prometheus.SummaryObjective `yaml:"defaultSummaryObjectives"`

	// OnError specifies what to do when an error either with listening
	// on the specified listen address or registering a metric with the
	// Prometheus. By default the registerer will panic.
	OnError string `yaml:"onError"`

PrometheusConfiguration is a configuration for a Prometheus reporter.

func (PrometheusConfiguration) NewReporter added in v0.15.0

NewReporter creates a new M3 Prometheus reporter from this configuration.

type PrometheusConfigurationOptions added in v0.15.0

type PrometheusConfigurationOptions struct {
	// Registry if not nil will specify the specific registry to use
	// for registering metrics.
	Registry *prom.Registry
	// ExternalRegistries if set (with combination of a specified Registry)
	// will also
	ExternalRegistries []PrometheusExternalRegistry
	// HandlerListener is the listener to register the server handler on.
	HandlerListener net.Listener
	// DefaultServeMux is the ServeMux to use if no HandlerListener or
	// ListenAddress on PrometheusConfiguration is specified.
	DefaultServeMux *http.ServeMux
	// HandlerOpts is the reporter HTTP handler options, not specifying will
	// use defaults.
	HandlerOpts promhttp.HandlerOpts
	// OnError allows for customization of what to do when a metric
	// registration error fails, the default is to panic.
	OnError func(e error)
	// CommonLabels will be appended to every metric gathered.
	CommonLabels map[string]string

PrometheusConfigurationOptions allows some programatic options, such as using a specific registry and what error callback to register.

type PrometheusExternalRegistry added in v0.15.0

type PrometheusExternalRegistry struct {
	// Registry is the external prometheus registry to list.
	Registry *extprom.Registry
	// SubScope will add a prefix to all metric names exported by
	// this registry.
	SubScope string

PrometheusExternalRegistry is an external Prometheus registry to also expose as part of the handler.

type Reporter

type Reporter interface {
	// Start starts the reporter.
	Start() error
	// Stop stops the reporter.
	Stop() error

Reporter reports metrics about a component.

func NewBuildReporter

func NewBuildReporter(
	opts Options,
) Reporter

NewBuildReporter returns a new build version reporter.

func NewExtendedMetricsReporter

func NewExtendedMetricsReporter(
	scope tally.Scope,
	reportInterval time.Duration,
	metricsType ExtendedMetricsType,
) Reporter

NewExtendedMetricsReporter creates a new extended metrics reporter that reports runtime and process metrics.

func NewProcessReporter

func NewProcessReporter(
	scope tally.Scope,
	reportInterval time.Duration,
) Reporter

NewProcessReporter returns a new reporter that reports process metrics, currently just the process file descriptor count.

func StartReportingExtendedMetrics

func StartReportingExtendedMetrics(
	scope tally.Scope,
	reportInterval time.Duration,
	metricsType ExtendedMetricsType,
) (Reporter, error)

StartReportingExtendedMetrics creates a extend metrics reporter and starts the reporter returning it so it may be stopped if successfully started.

type ScopeConfiguration

type ScopeConfiguration struct {
	// Prefix of metrics in this scope.
	Prefix string `yaml:"prefix"`

	// Metrics reporting interval.
	ReportingInterval time.Duration `yaml:"reportingInterval"`

	// Common tags shared by metrics reported.
	CommonTags map[string]string `yaml:"tags"`

ScopeConfiguration configures a metric scope.

type StringListEmitter added in v0.11.0

type StringListEmitter struct {
	// contains filtered or unexported fields

StringListEmitter emits a gauges indicating the order of a list of strings.

func NewStringListEmitter added in v0.11.0

func NewStringListEmitter(scope tally.Scope, name string) *StringListEmitter

NewStringListEmitter returns a StringListEmitter.

func (*StringListEmitter) Close added in v0.11.0

func (sle *StringListEmitter) Close() error

Close stops emitting the gauge.

func (*StringListEmitter) Start added in v0.11.0

func (sle *StringListEmitter) Start(sl []string) error

Start starts a goroutine that continuously emits the value of the gauges

func (*StringListEmitter) UpdateStringList added in v0.11.0

func (sle *StringListEmitter) UpdateStringList(sl []string) error

UpdateStringList updates the gauges according to the passed list of strings. It will first set the old gauges to 0, then emit new metrics with different values for the "type" label.

type SummaryObjective added in v0.15.0

type SummaryObjective struct {
	Percentile   float64 `yaml:"percentile"`
	AllowedError float64 `yaml:"allowedError"`

SummaryObjective is a Prometheus summary objective. See:

type TimerOptions added in v0.15.0

type TimerOptions struct {
	Type               TimerType
	StandardSampleRate float64
	HistogramBuckets   tally.Buckets

TimerOptions is a set of timer options when creating a timer.

func NewHistogramTimerOptions added in v0.15.0

func NewHistogramTimerOptions(opts HistogramTimerOptions) TimerOptions

NewHistogramTimerOptions returns a set of histogram timer options and if no histogram buckets are set it will use the default histogram buckets defined.

func NewStandardTimerOptions added in v0.15.0

func NewStandardTimerOptions() TimerOptions

NewStandardTimerOptions returns a set of standard timer options for standard timer types.

func (TimerOptions) NewTimer added in v0.15.0

func (o TimerOptions) NewTimer(scope tally.Scope, name string) tally.Timer

NewTimer creates a new timer based on the timer options.

type TimerType added in v0.15.0

type TimerType uint

TimerType is a type of timer, standard or histogram timer.

const (
	// StandardTimerType is a standard timer type back by a regular timer.
	StandardTimerType TimerType = iota
	// HistogramTimerType is a histogram timer backed by a histogram.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL