statsdreceiver

README

StatsD Receiver

StatsD receiver for ingesting StatsD messages(https://github.com/statsd/statsd/blob/master/docs/metric_types.md) into the OpenTelemetry Collector. Note: This receiver does not support horizontally scaled collector deployments. It is intended to run in agent mode, where a single collector instance receives StatsD input.

Status
Stability	beta: metrics
Distributions	contrib
Issues
Code coverage
Code Owners	@jmacd, @dmitryax

Configuration

The Following settings are optional:

• endpoint: Address and port to listen on.

  • For udp and tcp based transport, this config will default to localhost:8125
  • For unixgram transport, this config will default to /var/run/statsd-receiver.sock

• transport (default = udp): Protocol used by the StatsD server. Currently supported transports can be found in this file.

• socket_permissions (default = 0622): When transport is set to unixgram, can be used to customize permissions of the binded socket.

• aggregation_interval: 70s(default value is 60s): The aggregation time that the receiver aggregates the metrics (similar to the flush interval in StatsD server)

• enable_metric_type: true(default value is false): Enable the statsd receiver to be able to emit the metric type(gauge, counter, timer(in the future), histogram(in the future)) as a label.

• enable_ip_only_aggregation (default value is false): Enables metric aggregation on Client+IP only. Normally, aggregation is performed on Client+IP+Port. This setting is useful when the client sends metrics from a random ports or the receiver should aggregate metrics from the same client but different ports.

• ignore_host (default value is false): Completely ignores source IP for aggregation. All metrics with the same name, type, and tags aggregate together regardless of the sender's IP address. This is useful when you want to aggregate metrics from multiple sources into a single metric. Takes precedence over enable_ip_only_aggregation.

• enable_simple_tags: true(default value is false): Enable parsing tags that do not have a value, e.g. #mykey instead of #mykey:myvalue. DogStatsD supports such tagging.

• is_monotonic_counter (default value is false): Set all counter-type metrics the statsd receiver received as monotonic.

• timer_histogram_mapping:(default value is below): Specify what OTLP type to convert received timing/histogram data to.

"statsd_type" specifies received Statsd data type. Possible values for this setting are "timing", "timer", "histogram" and "distribution".

"observer_type" specifies OTLP data type to convert to. We support "gauge", "summary", and "histogram". For "gauge", it does not perform any aggregation. For "summary, the statsD receiver will aggregate to one OTLP summary metric for one metric description (the same metric name with the same tags). By default, it will send percentile 0, 10, 50, 90, 95, 100 to the downstream. The "histogram" setting selects an auto-scaling exponential histogram configured with only a maximum size, as shown in the example below unless it matches the configured explicit_buckets matcher pattern. TODO: Add a new option to use a smoothed summary like Prometheus: https://github.com/open-telemetry/opentelemetry-collector-contrib/pull/3261

Example:

receivers:
  statsd:
  statsd/2:
    endpoint: "localhost:8127"
    aggregation_interval: 70s
    enable_metric_type: true
    is_monotonic_counter: false
    timer_histogram_mapping:
      - statsd_type: "histogram"
        observer_type: "gauge"
      - statsd_type: "timing"
        observer_type: "histogram"
        histogram: 
          max_size: 100
          explicit_buckets:
            - matcher_pattern: "foo.*"
              buckets: [1, 10, 100]
            - matcher_pattern: "bar.*"
              buckets: [0.1, 0.5, 1]
      - statsd_type: "distribution"
        observer_type: "summary"
        summary: 
          percentiles: [0, 10, 50, 90, 95, 100]

The full list of settings exposed for this receiver are documented in config.go with detailed sample configurations in testdata/config.yaml.

Aggregation

Aggregation is done in statsD receiver. The default aggregation interval is 60s. The receiver only aggregates the metrics with the same metric name, metric type, label keys and label values. After each aggregation interval, the receiver will send all metrics (after aggregation) in this aggregation interval to the following workflow.

It supports: Counter(transferred to int):

• statsdTestMetric1:3000|c|#mykey:myvalue statsdTestMetric1:4000|c|#mykey:myvalue (get the value after incrementation: 7000)
• statsdTestMetric1:3000|c|#mykey:myvalue statsdTestMetric1:20|c|@0.25|#mykey:myvalue (get the value after incrementation with sample rate: 3000+20/0.25=3080)

When the receiver receives valid sample rate (greater than 0 and less than 1), we covert the count value to float, divide by the sample rate and then covert back to integer.

The official doc does not support negative counter, we follow this pattern at this time. There are some requests for negative counters, we need to take a look if we want to support later. For example: https://github.com/influxdata/telegraf/issues/1898 https://thenewstack.io/collecting-metrics-using-statsd-a-standard-for-real-time-monitoring/ https://docs.datadoghq.com/developers/metrics/dogstatsd_metrics_submission/#count

Gauge(transferred to double):

• statsdTestMetric1:500|g|#mykey:myvalue statsdTestMetric1:400|g|#mykey:myvalue (get the latest value: 400)
• statsdTestMetric1:500|g|#mykey:myvalue statsdTestMetric1:+2|g|#mykey:myvalue statsdTestMetric1:-1|g|#mykey:myvalue (get the value after calculation: 501)

Metrics

General format is:

<name>:<value>|<type>|@<sample-rate>|#<tag1-key>:<tag1-value>,<tag2-k/v>

Counter

<name>:<value>|c|@<sample-rate>|#<tag1-key>:<tag1-value>

It supports sample rate. When a sample rate is provided (between 0 and 1), the receiver divides the counter value by the sample rate during aggregation to estimate the actual count. For example, a counter value of 20 with sample rate 0.25 results in an estimated count of 80 (20 ÷ 0.25).

Note: Currently, the sample rate is applied during aggregation and the adjusted value is exported. If OTLP adds native support for sample rate as a metric parameter in the future, the implementation may be updated to preserve the original sample rate information.

Gauge

<name>:<value>|g|@<sample-rate>|#<tag1-key>:<tag1-value>

Timer

<name>:<value>|ms|@<sample-rate>|#<tag1-key>:<tag1-value> <name>:<value>|h|@<sample-rate>|#<tag1-key>:<tag1-value>

It supports sample rate.

Testing

Full sample collector config

receivers:
  statsd:
    endpoint: "localhost:8125" # default
    aggregation_interval: 60s  # default
    enable_metric_type: false   # default
    is_monotonic_counter: false # default
    timer_histogram_mapping:
      - statsd_type: "histogram"
        observer_type: "histogram"
        histogram:
          max_size: 50
          explicit_buckets:
            - matcher_pattern: "foo.*"
              buckets: [1, 10, 100]
            - matcher_pattern: "bar.*"
              buckets: [0.1, 0.5, 1]
      - statsd_type: "distribution"
        observer_type: "histogram"
        histogram: 
          max_size: 50    
      - statsd_type: "timing"
        observer_type: "summary"

exporters:
  file:
    path: ./test.json

service:
  pipelines:
    metrics:
     receivers: [statsd]
     exporters: [file]

Send StatsD message into the receiver

A simple way to send a metric to localhost:8125:

echo "test.metric:42|c|#myKey:myVal" | nc -w 1 -u -4 localhost 8125;
echo "test.metric:42|c|#myKey:myVal" | nc -w 1 -u -6 localhost 8125;

Which sends a UDP packet using both IPV4 and IPV6, which is needed because the receiver's UDP server only accepts one or the other.

Documentation

Overview

Package statsdreceiver implements a collector receiver that listens on UDP port 8125 by default for incoming StatsD messages and parses them into OTLP equivalent metric representations.

Index

• func NewFactory() receiver.Factory
• type Config
  • func (c *Config) Validate() error

Functions

func NewFactory

func NewFactory() receiver.Factory

NewFactory creates a factory for the StatsD receiver.

Types

type Config

type Config struct {
	NetAddr                 confignet.AddrConfig `mapstructure:",squash"`
	AggregationInterval     time.Duration        `mapstructure:"aggregation_interval"`
	EnableIPOnlyAggregation bool                 `mapstructure:"enable_ip_only_aggregation"`
	IgnoreHost              bool                 `mapstructure:"ignore_host"`
	EnableMetricType        bool                 `mapstructure:"enable_metric_type"`
	EnableSimpleTags        bool                 `mapstructure:"enable_simple_tags"`
	IsMonotonicCounter      bool                 `mapstructure:"is_monotonic_counter"`
	// CounterType specifies how counter values are represented in exported metrics.
	// Valid values: "int" (default), "float", "stochastic_int".
	CounterType           protocol.CounterType             `mapstructure:"counter_type"`
	TimerHistogramMapping []protocol.TimerHistogramMapping `mapstructure:"timer_histogram_mapping"`
	// Will only be used when transport set to 'unixgram'.
	SocketPermissions os.FileMode `mapstructure:"socket_permissions"`
}

Config defines configuration for StatsD receiver.

func (*Config) Validate

func (c *Config) Validate() error