README

go-metrics

This library provides a metrics package which can be used to instrument code, expose application metrics, and profile runtime performance in a flexible manner.

Current API: GoDoc

Sinks

The metrics package makes use of a MetricSink interface to support delivery to any type of backend. Currently the following sinks are provided:

  • StatsiteSink : Sinks to a statsite instance (TCP)
  • StatsdSink: Sinks to a StatsD / statsite instance (UDP)
  • PrometheusSink: Sinks to a Prometheus metrics endpoint (exposed via HTTP for scrapes)
  • InmemSink : Provides in-memory aggregation, can be used to export stats
  • FanoutSink : Sinks to multiple sinks. Enables writing to multiple statsite instances for example.
  • BlackholeSink : Sinks to nowhere

In addition to the sinks, the InmemSignal can be used to catch a signal, and dump a formatted output of recent metrics. For example, when a process gets a SIGUSR1, it can dump to stderr recent performance metrics for debugging.

Labels

Most metrics do have an equivalent ending with WithLabels, such methods allow to push metrics with labels and use some features of underlying Sinks (ex: translated into Prometheus labels).

Since some of these labels may increase greatly cardinality of metrics, the library allow to filter labels using a blacklist/whitelist filtering system which is global to all metrics.

  • If Config.AllowedLabels is not nil, then only labels specified in this value will be sent to underlying Sink, otherwise, all labels are sent by default.
  • If Config.BlockedLabels is not nil, any label specified in this value will not be sent to underlying Sinks.

By default, both Config.AllowedLabels and Config.BlockedLabels are nil, meaning that no tags are filetered at all, but it allow to a user to globally block some tags with high cardinality at application level.

Examples

Here is an example of using the package:

func SlowMethod() {
    // Profiling the runtime of a method
    defer metrics.MeasureSince([]string{"SlowMethod"}, time.Now())
}

// Configure a statsite sink as the global metrics sink
sink, _ := metrics.NewStatsiteSink("statsite:8125")
metrics.NewGlobal(metrics.DefaultConfig("service-name"), sink)

// Emit a Key/Value pair
metrics.EmitKey([]string{"questions", "meaning of life"}, 42)

Here is an example of setting up a signal handler:

// Setup the inmem sink and signal handler
inm := metrics.NewInmemSink(10*time.Second, time.Minute)
sig := metrics.DefaultInmemSignal(inm)
metrics.NewGlobal(metrics.DefaultConfig("service-name"), inm)

// Run some code
inm.SetGauge([]string{"foo"}, 42)
inm.EmitKey([]string{"bar"}, 30)

inm.IncrCounter([]string{"baz"}, 42)
inm.IncrCounter([]string{"baz"}, 1)
inm.IncrCounter([]string{"baz"}, 80)

inm.AddSample([]string{"method", "wow"}, 42)
inm.AddSample([]string{"method", "wow"}, 100)
inm.AddSample([]string{"method", "wow"}, 22)

....

When a signal comes in, output like the following will be dumped to stderr:

[2014-01-28 14:57:33.04 -0800 PST][G] 'foo': 42.000
[2014-01-28 14:57:33.04 -0800 PST][P] 'bar': 30.000
[2014-01-28 14:57:33.04 -0800 PST][C] 'baz': Count: 3 Min: 1.000 Mean: 41.000 Max: 80.000 Stddev: 39.509
[2014-01-28 14:57:33.04 -0800 PST][S] 'method.wow': Count: 3 Min: 22.000 Mean: 54.667 Max: 100.000 Stddev: 40.513
Expand ▾ Collapse ▴

Documentation

Index

Constants

View Source
const (
	// DefaultSignal is used with DefaultInmemSignal
	DefaultSignal = syscall.SIGUSR1
)

Variables

This section is empty.

Functions

func AddSample

func AddSample(key []string, val float32)

func AddSampleWithLabels

func AddSampleWithLabels(key []string, val float32, labels []Label)

func EmitKey

func EmitKey(key []string, val float32)

func IncrCounter

func IncrCounter(key []string, val float32)

func IncrCounterWithLabels

func IncrCounterWithLabels(key []string, val float32, labels []Label)

func MeasureSince

func MeasureSince(key []string, start time.Time)

func MeasureSinceWithLabels

func MeasureSinceWithLabels(key []string, start time.Time, labels []Label)

func SetGauge

func SetGauge(key []string, val float32)

    Proxy all the methods to the globalMetrics instance

    func SetGaugeWithLabels

    func SetGaugeWithLabels(key []string, val float32, labels []Label)

    func UpdateFilter

    func UpdateFilter(allow, block []string)

    func UpdateFilterAndLabels

    func UpdateFilterAndLabels(allow, block, allowedLabels, blockedLabels []string)

      UpdateFilterAndLabels set allow/block prefixes of metrics while allowedLabels and blockedLabels - when not nil - allow filtering of labels in order to block/allow globally labels (especially useful when having large number of values for a given label). See README.md for more information about usage.

      Types

      type AggregateSample

      type AggregateSample struct {
      	Count       int       // The count of emitted pairs
      	Rate        float64   // The values rate per time unit (usually 1 second)
      	Sum         float64   // The sum of values
      	SumSq       float64   `json:"-"` // The sum of squared values
      	Min         float64   // Minimum value
      	Max         float64   // Maximum value
      	LastUpdated time.Time `json:"-"` // When value was last updated
      }

        AggregateSample is used to hold aggregate metrics about a sample

        func (*AggregateSample) Ingest

        func (a *AggregateSample) Ingest(v float64, rateDenom float64)

          Ingest is used to update a sample

          func (*AggregateSample) Mean

          func (a *AggregateSample) Mean() float64

            Computes a mean of the values

            func (*AggregateSample) Stddev

            func (a *AggregateSample) Stddev() float64

              Computes a Stddev of the values

              func (*AggregateSample) String

              func (a *AggregateSample) String() string

              type BlackholeSink

              type BlackholeSink struct{}

                BlackholeSink is used to just blackhole messages

                func (*BlackholeSink) AddSample

                func (*BlackholeSink) AddSample(key []string, val float32)

                func (*BlackholeSink) AddSampleWithLabels

                func (*BlackholeSink) AddSampleWithLabels(key []string, val float32, labels []Label)

                func (*BlackholeSink) EmitKey

                func (*BlackholeSink) EmitKey(key []string, val float32)

                func (*BlackholeSink) IncrCounter

                func (*BlackholeSink) IncrCounter(key []string, val float32)

                func (*BlackholeSink) IncrCounterWithLabels

                func (*BlackholeSink) IncrCounterWithLabels(key []string, val float32, labels []Label)

                func (*BlackholeSink) SetGauge

                func (*BlackholeSink) SetGauge(key []string, val float32)

                func (*BlackholeSink) SetGaugeWithLabels

                func (*BlackholeSink) SetGaugeWithLabels(key []string, val float32, labels []Label)

                type Config

                type Config struct {
                	ServiceName          string        // Prefixed with keys to separate services
                	HostName             string        // Hostname to use. If not provided and EnableHostname, it will be os.Hostname
                	EnableHostname       bool          // Enable prefixing gauge values with hostname
                	EnableHostnameLabel  bool          // Enable adding hostname to labels
                	EnableServiceLabel   bool          // Enable adding service to labels
                	EnableRuntimeMetrics bool          // Enables profiling of runtime metrics (GC, Goroutines, Memory)
                	EnableTypePrefix     bool          // Prefixes key with a type ("counter", "gauge", "timer")
                	TimerGranularity     time.Duration // Granularity of timers.
                	ProfileInterval      time.Duration // Interval to profile runtime metrics
                
                	AllowedPrefixes []string // A list of metric prefixes to allow, with '.' as the separator
                	BlockedPrefixes []string // A list of metric prefixes to block, with '.' as the separator
                	AllowedLabels   []string // A list of metric labels to allow, with '.' as the separator
                	BlockedLabels   []string // A list of metric labels to block, with '.' as the separator
                	FilterDefault   bool     // Whether to allow metrics by default
                }

                  Config is used to configure metrics settings

                  func DefaultConfig

                  func DefaultConfig(serviceName string) *Config

                    DefaultConfig provides a sane default configuration

                    type FanoutSink

                    type FanoutSink []MetricSink

                      FanoutSink is used to sink to fanout values to multiple sinks

                      func (FanoutSink) AddSample

                      func (fh FanoutSink) AddSample(key []string, val float32)

                      func (FanoutSink) AddSampleWithLabels

                      func (fh FanoutSink) AddSampleWithLabels(key []string, val float32, labels []Label)

                      func (FanoutSink) EmitKey

                      func (fh FanoutSink) EmitKey(key []string, val float32)

                      func (FanoutSink) IncrCounter

                      func (fh FanoutSink) IncrCounter(key []string, val float32)

                      func (FanoutSink) IncrCounterWithLabels

                      func (fh FanoutSink) IncrCounterWithLabels(key []string, val float32, labels []Label)

                      func (FanoutSink) SetGauge

                      func (fh FanoutSink) SetGauge(key []string, val float32)

                      func (FanoutSink) SetGaugeWithLabels

                      func (fh FanoutSink) SetGaugeWithLabels(key []string, val float32, labels []Label)

                      type GaugeValue

                      type GaugeValue struct {
                      	Name  string
                      	Hash  string `json:"-"`
                      	Value float32
                      
                      	Labels        []Label           `json:"-"`
                      	DisplayLabels map[string]string `json:"Labels"`
                      }

                      type InmemSignal

                      type InmemSignal struct {
                      	// contains filtered or unexported fields
                      }

                        InmemSignal is used to listen for a given signal, and when received, to dump the current metrics from the InmemSink to an io.Writer

                        func DefaultInmemSignal

                        func DefaultInmemSignal(inmem *InmemSink) *InmemSignal

                          DefaultInmemSignal returns a new InmemSignal that responds to SIGUSR1 and writes output to stderr. Windows uses SIGBREAK

                          func NewInmemSignal

                          func NewInmemSignal(inmem *InmemSink, sig syscall.Signal, w io.Writer) *InmemSignal

                            NewInmemSignal creates a new InmemSignal which listens for a given signal, and dumps the current metrics out to a writer

                            func (*InmemSignal) Stop

                            func (i *InmemSignal) Stop()

                              Stop is used to stop the InmemSignal from listening

                              type InmemSink

                              type InmemSink struct {
                              	// contains filtered or unexported fields
                              }

                                InmemSink provides a MetricSink that does in-memory aggregation without sending metrics over a network. It can be embedded within an application to provide profiling information.

                                func NewInmemSink

                                func NewInmemSink(interval, retain time.Duration) *InmemSink

                                  NewInmemSink is used to construct a new in-memory sink. Uses an aggregation interval and maximum retention period.

                                  func (*InmemSink) AddSample

                                  func (i *InmemSink) AddSample(key []string, val float32)

                                  func (*InmemSink) AddSampleWithLabels

                                  func (i *InmemSink) AddSampleWithLabels(key []string, val float32, labels []Label)

                                  func (*InmemSink) Data

                                  func (i *InmemSink) Data() []*IntervalMetrics

                                    Data is used to retrieve all the aggregated metrics Intervals may be in use, and a read lock should be acquired

                                    func (*InmemSink) DisplayMetrics

                                    func (i *InmemSink) DisplayMetrics(resp http.ResponseWriter, req *http.Request) (interface{}, error)

                                      DisplayMetrics returns a summary of the metrics from the most recent finished interval.

                                      func (*InmemSink) EmitKey

                                      func (i *InmemSink) EmitKey(key []string, val float32)

                                      func (*InmemSink) IncrCounter

                                      func (i *InmemSink) IncrCounter(key []string, val float32)

                                      func (*InmemSink) IncrCounterWithLabels

                                      func (i *InmemSink) IncrCounterWithLabels(key []string, val float32, labels []Label)

                                      func (*InmemSink) SetGauge

                                      func (i *InmemSink) SetGauge(key []string, val float32)

                                      func (*InmemSink) SetGaugeWithLabels

                                      func (i *InmemSink) SetGaugeWithLabels(key []string, val float32, labels []Label)

                                      type IntervalMetrics

                                      type IntervalMetrics struct {
                                      	sync.RWMutex
                                      
                                      	// The start time of the interval
                                      	Interval time.Time
                                      
                                      	// Gauges maps the key to the last set value
                                      	Gauges map[string]GaugeValue
                                      
                                      	// Points maps the string to the list of emitted values
                                      	// from EmitKey
                                      	Points map[string][]float32
                                      
                                      	// Counters maps the string key to a sum of the counter
                                      	// values
                                      	Counters map[string]SampledValue
                                      
                                      	// Samples maps the key to an AggregateSample,
                                      	// which has the rolled up view of a sample
                                      	Samples map[string]SampledValue
                                      }

                                        IntervalMetrics stores the aggregated metrics for a specific interval

                                        func NewIntervalMetrics

                                        func NewIntervalMetrics(intv time.Time) *IntervalMetrics

                                          NewIntervalMetrics creates a new IntervalMetrics for a given interval

                                          type Label

                                          type Label struct {
                                          	Name  string
                                          	Value string
                                          }

                                          type MetricSink

                                          type MetricSink interface {
                                          	// A Gauge should retain the last value it is set to
                                          	SetGauge(key []string, val float32)
                                          	SetGaugeWithLabels(key []string, val float32, labels []Label)
                                          
                                          	// Should emit a Key/Value pair for each call
                                          	EmitKey(key []string, val float32)
                                          
                                          	// Counters should accumulate values
                                          	IncrCounter(key []string, val float32)
                                          	IncrCounterWithLabels(key []string, val float32, labels []Label)
                                          
                                          	// Samples are for timing information, where quantiles are used
                                          	AddSample(key []string, val float32)
                                          	AddSampleWithLabels(key []string, val float32, labels []Label)
                                          }

                                            The MetricSink interface is used to transmit metrics information to an external system

                                            func NewInmemSinkFromURL

                                            func NewInmemSinkFromURL(u *url.URL) (MetricSink, error)

                                              NewInmemSinkFromURL creates an InmemSink from a URL. It is used (and tested) from NewMetricSinkFromURL.

                                              func NewMetricSinkFromURL

                                              func NewMetricSinkFromURL(urlStr string) (MetricSink, error)

                                                NewMetricSinkFromURL allows a generic URL input to configure any of the supported sinks. The scheme of the URL identifies the type of the sink, the and query parameters are used to set options.

                                                "statsd://" - Initializes a StatsdSink. The host and port are passed through as the "addr" of the sink

                                                "statsite://" - Initializes a StatsiteSink. The host and port become the "addr" of the sink

                                                "inmem://" - Initializes an InmemSink. The host and port are ignored. The "interval" and "duration" query parameters must be specified with valid durations, see NewInmemSink for details.

                                                func NewStatsdSinkFromURL

                                                func NewStatsdSinkFromURL(u *url.URL) (MetricSink, error)

                                                  NewStatsdSinkFromURL creates an StatsdSink from a URL. It is used (and tested) from NewMetricSinkFromURL.

                                                  func NewStatsiteSinkFromURL

                                                  func NewStatsiteSinkFromURL(u *url.URL) (MetricSink, error)

                                                    NewStatsiteSinkFromURL creates an StatsiteSink from a URL. It is used (and tested) from NewMetricSinkFromURL.

                                                    type Metrics

                                                    type Metrics struct {
                                                    	Config
                                                    	// contains filtered or unexported fields
                                                    }

                                                      Metrics represents an instance of a metrics sink that can be used to emit

                                                      func Default

                                                      func Default() *Metrics

                                                        Default returns the shared global metrics instance.

                                                        func New

                                                        func New(conf *Config, sink MetricSink) (*Metrics, error)

                                                          New is used to create a new instance of Metrics

                                                          func NewGlobal

                                                          func NewGlobal(conf *Config, sink MetricSink) (*Metrics, error)

                                                            NewGlobal is the same as New, but it assigns the metrics object to be used globally as well as returning it.

                                                            func (*Metrics) AddSample

                                                            func (m *Metrics) AddSample(key []string, val float32)

                                                            func (*Metrics) AddSampleWithLabels

                                                            func (m *Metrics) AddSampleWithLabels(key []string, val float32, labels []Label)

                                                            func (*Metrics) EmitKey

                                                            func (m *Metrics) EmitKey(key []string, val float32)

                                                            func (*Metrics) EmitRuntimeStats

                                                            func (m *Metrics) EmitRuntimeStats()

                                                              Emits various runtime statsitics

                                                              func (*Metrics) IncrCounter

                                                              func (m *Metrics) IncrCounter(key []string, val float32)

                                                              func (*Metrics) IncrCounterWithLabels

                                                              func (m *Metrics) IncrCounterWithLabels(key []string, val float32, labels []Label)

                                                              func (*Metrics) MeasureSince

                                                              func (m *Metrics) MeasureSince(key []string, start time.Time)

                                                              func (*Metrics) MeasureSinceWithLabels

                                                              func (m *Metrics) MeasureSinceWithLabels(key []string, start time.Time, labels []Label)

                                                              func (*Metrics) SetGauge

                                                              func (m *Metrics) SetGauge(key []string, val float32)

                                                              func (*Metrics) SetGaugeWithLabels

                                                              func (m *Metrics) SetGaugeWithLabels(key []string, val float32, labels []Label)

                                                              func (*Metrics) UpdateFilter

                                                              func (m *Metrics) UpdateFilter(allow, block []string)

                                                                UpdateFilter overwrites the existing filter with the given rules.

                                                                func (*Metrics) UpdateFilterAndLabels

                                                                func (m *Metrics) UpdateFilterAndLabels(allow, block, allowedLabels, blockedLabels []string)

                                                                  UpdateFilterAndLabels overwrites the existing filter with the given rules.

                                                                  type MetricsSummary

                                                                  type MetricsSummary struct {
                                                                  	Timestamp string
                                                                  	Gauges    []GaugeValue
                                                                  	Points    []PointValue
                                                                  	Counters  []SampledValue
                                                                  	Samples   []SampledValue
                                                                  }

                                                                    MetricsSummary holds a roll-up of metrics info for a given interval

                                                                    type PointValue

                                                                    type PointValue struct {
                                                                    	Name   string
                                                                    	Points []float32
                                                                    }

                                                                    type SampledValue

                                                                    type SampledValue struct {
                                                                    	Name string
                                                                    	Hash string `json:"-"`
                                                                    	*AggregateSample
                                                                    	Mean   float64
                                                                    	Stddev float64
                                                                    
                                                                    	Labels        []Label           `json:"-"`
                                                                    	DisplayLabels map[string]string `json:"Labels"`
                                                                    }

                                                                    type StatsdSink

                                                                    type StatsdSink struct {
                                                                    	// contains filtered or unexported fields
                                                                    }

                                                                      StatsdSink provides a MetricSink that can be used with a statsite or statsd metrics server. It uses only UDP packets, while StatsiteSink uses TCP.

                                                                      func NewStatsdSink

                                                                      func NewStatsdSink(addr string) (*StatsdSink, error)

                                                                        NewStatsdSink is used to create a new StatsdSink

                                                                        func (*StatsdSink) AddSample

                                                                        func (s *StatsdSink) AddSample(key []string, val float32)

                                                                        func (*StatsdSink) AddSampleWithLabels

                                                                        func (s *StatsdSink) AddSampleWithLabels(key []string, val float32, labels []Label)

                                                                        func (*StatsdSink) EmitKey

                                                                        func (s *StatsdSink) EmitKey(key []string, val float32)

                                                                        func (*StatsdSink) IncrCounter

                                                                        func (s *StatsdSink) IncrCounter(key []string, val float32)

                                                                        func (*StatsdSink) IncrCounterWithLabels

                                                                        func (s *StatsdSink) IncrCounterWithLabels(key []string, val float32, labels []Label)

                                                                        func (*StatsdSink) SetGauge

                                                                        func (s *StatsdSink) SetGauge(key []string, val float32)

                                                                        func (*StatsdSink) SetGaugeWithLabels

                                                                        func (s *StatsdSink) SetGaugeWithLabels(key []string, val float32, labels []Label)

                                                                        func (*StatsdSink) Shutdown

                                                                        func (s *StatsdSink) Shutdown()

                                                                          Close is used to stop flushing to statsd

                                                                          type StatsiteSink

                                                                          type StatsiteSink struct {
                                                                          	// contains filtered or unexported fields
                                                                          }

                                                                            StatsiteSink provides a MetricSink that can be used with a statsite metrics server

                                                                            func NewStatsiteSink

                                                                            func NewStatsiteSink(addr string) (*StatsiteSink, error)

                                                                              NewStatsiteSink is used to create a new StatsiteSink

                                                                              func (*StatsiteSink) AddSample

                                                                              func (s *StatsiteSink) AddSample(key []string, val float32)

                                                                              func (*StatsiteSink) AddSampleWithLabels

                                                                              func (s *StatsiteSink) AddSampleWithLabels(key []string, val float32, labels []Label)

                                                                              func (*StatsiteSink) EmitKey

                                                                              func (s *StatsiteSink) EmitKey(key []string, val float32)

                                                                              func (*StatsiteSink) IncrCounter

                                                                              func (s *StatsiteSink) IncrCounter(key []string, val float32)

                                                                              func (*StatsiteSink) IncrCounterWithLabels

                                                                              func (s *StatsiteSink) IncrCounterWithLabels(key []string, val float32, labels []Label)

                                                                              func (*StatsiteSink) SetGauge

                                                                              func (s *StatsiteSink) SetGauge(key []string, val float32)

                                                                              func (*StatsiteSink) SetGaugeWithLabels

                                                                              func (s *StatsiteSink) SetGaugeWithLabels(key []string, val float32, labels []Label)

                                                                              func (*StatsiteSink) Shutdown

                                                                              func (s *StatsiteSink) Shutdown()

                                                                                Close is used to stop flushing to statsite

                                                                                Directories

                                                                                Path Synopsis