cumulativetodeltaprocessor

README

Cumulative to Delta Processor

The cumulative to delta processor (cumulativetodeltaprocessor) converts monotonic sum, histogram, and exponential histogram metrics from cumulative to delta aggregation temporality. Non-monotonic sums are excluded. Delta metrics are excluded from any conversion and forwarded without changes.

Status
Stability	beta: metrics
Distributions	contrib, k8s
Warnings	Statefulness
Issues
Code coverage
Code Owners	@TylerHelmuth

Configuration

Configuration is specified through a list of metrics. The processor uses metric names to identify a set of cumulative metrics and converts them from cumulative to delta.

The following settings can be optionally configured:

• include: List of metrics names (case-insensitive), patterns or metric types to convert to delta. Valid values for metric types are: sum, histogram, exponentialhistogram.
• exclude: List of metrics names (case-insensitive), patterns or metric types to not convert to delta. If a metric name matches both include and exclude, exclude takes precedence. Valid values for metric types are: sum, histogram, exponentialhistogram.
• max_staleness: The total time a state entry will live past the time it was last seen. Set to 0 to retain state indefinitely. Default: 1 hour
• initial_value: Handling of the first observed point for a given metric identity. When the collector (re)starts, there's no record of how much of a given cumulative counter has already been converted to delta values.
  • auto (default): Send if and only if the startime is set AND the starttime happens after the component started AND the starttime is different from the timestamp. Suitable for gateway deployments, this heuristic is like drop, but keeps values for newly started counters (which could not have had previous observed values).
  • keep: Send the observed value as the delta value. Suitable for when the incoming metrics have not been observed before, e.g. running the collector as a sidecar, the collector lifecycle is tied to the metric source.
  • drop: Keep the observed value but don't send. Suitable for gateway deployments, guarantees that all delta counts it produces haven't been observed before, but loses the values between thir first 2 observations.

If neither include nor exclude are supplied, no filtering is applied.

Examples

processors:
    # processor name: cumulative_to_delta
    cumulative_to_delta:

        # list the exact cumulative sum or histogram metrics to convert to delta
        include:
            metrics:
                - <metric_1_name>
                - <metric_2_name>
                .
                .
                - <metric_n_name>
            match_type: strict

processors:
    # processor name: cumulative_to_delta
    cumulative_to_delta:

        # Convert all sum metrics
        include:
            metric_types:
              - sum

processors:
    # processor name: cumulative_to_delta
    cumulative_to_delta:

        # Convert cumulative sum or histogram metrics to delta
        # if and only if 'metric' is in the name
        include:
            metrics:
                - ".*metric.*"
            match_type: regexp

processors:
    # processor name: cumulative_to_delta
    cumulative_to_delta:

        # Convert cumulative sum metrics to delta
        # if and only if 'metric' is in the name
        include:
            metrics:
                - ".*metric.*"
            match_type: regexp
            metric_types:
              - sum

processors:
    # processor name: cumulative_to_delta
    cumulative_to_delta:

        # Convert cumulative sum or histogram metrics to delta
        # if and only if 'metric' is not in the name
        exclude:
            metrics:
                - ".*metric.*"
            match_type: regexp

processors:
    # processor name: cumulative_to_delta
    cumulative_to_delta:

        # Convert cumulative sum metrics with 'metric' in their name,
        # but exclude histogram metrics
        include:
            metrics:
                - ".*metric.*"
            match_type: regexp
        exclude:
          metric_types:
            - histogram

processors:
    # processor name: cumulative_to_delta
    cumulative_to_delta:
        # If include/exclude are not specified
        # convert all cumulative sum or histogram metrics to delta

Troubleshooting

When Telemetry is enabled, this component can export several metrics. All metrics are disabled by default and must be opted-in via the collector's service.telemetry.metrics.level configuration.

Warnings

• Statefulness: The cumulative_to_delta processor's calculates delta by remembering the previous value of a metric. For this reason, the calculation is only accurate if the metric is continuously sent to the same instance of the collector. As a result, the cumulative_to_delta processor may not work as expected if used in a deployment of multiple collectors. When using this processor it is best for the data source to being sending data to a single collector.

Documentation

Overview

package cumulativetodeltaprocessor implements a processor which converts cumulative sum metrics to cumulative delta.

Index

• func NewFactory() processor.Factory
• type Config
  • func (config *Config) Validate() error
• type MatchMetrics

Functions

func NewFactory

func NewFactory() processor.Factory

NewFactory returns a new factory for the Metrics Generation processor.

Types

type Config

type Config struct {
	// MaxStaleness is the total time a state entry will live past the time it was last seen. Set to 0 to retain state indefinitely.
	MaxStaleness time.Duration `mapstructure:"max_staleness"`

	// InitialValue determines how to handle the first datapoint for a given metric. Valid values:
	//
	//   - auto: (default) send the first point iff the startime is set AND the starttime happens after the component started AND the starttime is different from the timestamp
	//   - keep: always send the first point
	//   - drop: don't send the first point, but store it for subsequent delta calculations
	InitialValue tracking.InitialValue `mapstructure:"initial_value"`

	// Include specifies a filter on the metrics that should be converted.
	// Exclude specifies a filter on the metrics that should not be converted.
	// If neither `include` nor `exclude` are set, all metrics will be converted.
	// Cannot be used with deprecated Metrics config option.
	Include MatchMetrics `mapstructure:"include"`
	Exclude MatchMetrics `mapstructure:"exclude"`
}

Config defines the configuration for the processor.

func (*Config) Validate

func (config *Config) Validate() error

Validate checks whether the input configuration has all of the required fields for the processor. An error is returned if there are any invalid inputs.

type MatchMetrics

type MatchMetrics struct {
	Config filterset.Config `mapstructure:",squash"`

	Metrics []string `mapstructure:"metrics"`

	MetricTypes []string `mapstructure:"metric_types"`
}