processor

README

Processors

Note This documentation is still in progress. For any questions, please reach out in the OpenTelemetry Gitter or refer to the issues page.

Supported processors (sorted alphabetically):

• Attributes Processor
• Node Batcher Processor
• Probabilistic Sampler Processor
• Queued Processor
• Span Processor
• Tail Sampling Processor

Ordering Processors

The order processors are specified in a pipeline is important as this is the order in which each processor is applied to traces.

Attributes Processor

The attributes processor modifies attributes of a span.

It takes a list of actions which are performed in order specified in the config. The supported actions are:

• insert: Inserts a new attribute in spans where the key does not already exist.
• update: Updates an attribute in spans where the key does exist.
• upsert: Performs insert or update. Inserts a new attribute in spans where the key does not already exist and updates an attribute in spans where the key does exist.
• delete: Deletes an attribute from a span.

For the actions insert, update and upsert,

• key is required
• one of value or from_attribute is required
• action is required.

  # Key specifies the attribute to act upon.
- key: <key>
  action: {insert, update, upsert}
  # Value specifies the value to populate for the key.
  # The type is inferred from the configuration.
  value: <value>

  # Key specifies the attribute to act upon.
- key: <key>
  action: {insert, update, upsert}
  # FromAttribute specifies the attribute from the span to use to populate
  # the value. If the attribute doesn't exist, no action is performed.
  from_attribute: <other key>

For the delete action,

• key is required
• action: delete is required.

# Key specifies the attribute to act upon.
- key: <key>
  action: delete

Please refer to config.go for the config spec.

Include/Exclude Spans

It is optional to provide a set of properties of a span to match against to determine if the span should be included or excluded from the processor. By default, all spans are processed by the processor.

To configure this option, under include and/or exclude:

• at least one of or both services and attributes is required.

Note: If both include and exclude are specified, the include properties are checked before the exclude properties.

attributes:
    # include and/or exclude can be specified. However, the include properties
    # are always checked before the exclude properties.
    {include, exclude}:
      # At least one of services or attributes must be specified. It is supported
      # to have both specified, but both `services` and `attributes` must evaluate
      # to true for a match to occur.
    
      # Services specify the list of service name to match against.
      # A match occurs if the span service name is in this list.
      # Note: This is an optional field.
      services: [<key1>, ..., <keyN>]
      # Attributes specifies the list of attributes to match against.
      # All of these attributes must match exactly for a match to occur.
      # Note: This is an optional field.
      attributes:
          # Key specifies the attribute to match against.
        - key: <key>
          # Value specifies the exact value to match against.
          # If not specified, a match occurs if the key is present in the attributes.
          value: {value} 

Example

The list of actions can be composed to create rich scenarios, such as back filling attribute, copying values to a new key, redacting sensitive information. The following is a sample configuration.

processors:
  attributes/example:
    actions:
      - key: db.table
        action: delete
      - key: redacted_span
        value: true
        action: upsert
      - key: copy_key
        from_attribute: key_original
        action: update
      - key: account_id
        value: 2245
      - key: account_password
        action: delete

Refer to config.yaml for detailed examples on using the processor.

Node Batcher Processor

<FILL ME IN - I'M LONELY!>

Probabilistic Sampler Processor

<FILL ME IN - I'M LONELY!>

Queued Processor

<FILL ME IN - I'M LONELY!>

Span Processor

The span processor modifies top level settings of a span. Currently, only renaming a span is supported.

Name a span

It takes a list of from_attributes and an optional separator string. The attribute value for the keys are used to create a new name in the order specified in the configuration. If a separator is specified, it will separate values.

If renaming is dependent on attributes being modified by the attributes processor, ensure the span processor is specified after the attributes processor in the pipeline specification.

For more information, refer to config.go

span:
  name:
    # from_attributes represents the attribute keys to pull the values from to generate the
    # new span name.
    from_attributes: [<key1>, <key2>, ...]
    # Separator is the string used to concatenate various parts of the span name.
    separator: <value>

Example configuration

For more examples with detailed comments, refer to config.yaml

span:
  name:
    from_attributes: ["db.svc", "operation"]
    separator: "::"

Tail Sampling Processor

<FILL ME IN - I'M LONELY!>

Documentation

Overview

Package processor contains interfaces that compose trace/metrics consumers.

Index

• Variables
• func Build(factories ...Factory) (map[string]Factory, error)
• func MetricTagKeys(level telemetry.Level) []tag.Key
• func MetricViews(level telemetry.Level) []*view.View
• func ServiceNameForNode(node *commonpb.Node) string
• func StatsTagsForBatch(processorName, serviceName, spanFormat string) []tag.Mutator
• type Factory
• type MetricsProcessor
  • func NewMetricsFanOutConnector(mcs []consumer.MetricsConsumer) MetricsProcessor
• type Processor
• type TraceProcessor
  • func NewTraceFanOutConnector(tcs []consumer.TraceConsumer) TraceProcessor

Variables

var (
	TagSourceFormatKey, _ = tag.NewKey("format")
	TagServiceNameKey, _  = tag.NewKey("service")
	TagExporterNameKey, _ = tag.NewKey("exporter")

	StatReceivedSpanCount = stats.Int64(
		"spans_received",
		"counts the number of spans received",
		stats.UnitDimensionless)
	StatDroppedSpanCount = stats.Int64(
		"spans_dropped",
		"counts the number of spans dropped",
		stats.UnitDimensionless)
	StatBadBatchDroppedSpanCount = stats.Int64(
		"bad_batch_spans_dropped",
		"counts the number of spans dropped due to being in bad batches",
		stats.UnitDimensionless)
)

Keys and stats for telemetry.

Functions

func Build

func Build(factories ...Factory) (map[string]Factory, error)

Build takes a list of processor factories and returns a map of type map[string]Factory with factory type as keys. It returns a non-nil error when more than one factories have the same type.

func MetricTagKeys

func MetricTagKeys(level telemetry.Level) []tag.Key

MetricTagKeys returns the metric tag keys according to the given telemetry level.

func MetricViews

func MetricViews(level telemetry.Level) []*view.View

MetricViews return the metrics views according to given telemetry level.

func ServiceNameForNode

func ServiceNameForNode(node *commonpb.Node) string

ServiceNameForNode gets the service name for a specified node. Used for metrics.

func StatsTagsForBatch

func StatsTagsForBatch(processorName, serviceName, spanFormat string) []tag.Mutator

StatsTagsForBatch gets the stat tags based on the specified processorName, serviceName, and spanFormat.

Types

type Factory

type Factory interface {
	// Type gets the type of the Processor created by this factory.
	Type() string

	// CreateDefaultConfig creates the default configuration for the Processor.
	CreateDefaultConfig() configmodels.Processor

	// CreateTraceProcessor creates a trace processor based on this config.
	// If the processor type does not support tracing or if the config is not valid
	// error will be returned instead.
	CreateTraceProcessor(logger *zap.Logger, nextConsumer consumer.TraceConsumer,
		cfg configmodels.Processor) (TraceProcessor, error)

	// CreateMetricsProcessor creates a metrics processor based on this config.
	// If the processor type does not support metrics or if the config is not valid
	// error will be returned instead.
	CreateMetricsProcessor(logger *zap.Logger, nextConsumer consumer.MetricsConsumer,
		cfg configmodels.Processor) (MetricsProcessor, error)
}

Factory is factory interface for processors.

type MetricsProcessor

type MetricsProcessor interface {
	consumer.MetricsConsumer
}

MetricsProcessor composes MetricsConsumer with some additional processor-specific functions.

func NewMetricsFanOutConnector

func NewMetricsFanOutConnector(mcs []consumer.MetricsConsumer) MetricsProcessor

NewMetricsFanOutConnector wraps multiple metrics consumers in a single one.

type Processor

type Processor interface {
	consumer.DataConsumer
}

Processor is a data consumer.

type TraceProcessor

type TraceProcessor interface {
	consumer.TraceConsumer
}

TraceProcessor composes TraceConsumer with some additional processor-specific functions.

func NewTraceFanOutConnector

func NewTraceFanOutConnector(tcs []consumer.TraceConsumer) TraceProcessor

NewTraceFanOutConnector wraps multiple trace consumers in a single one.