receiver

README

Note This documentation is still in progress. For any questions, please reach out in the OpenTelemetry Gitter or refer to the issues page.

Receivers

A receiver is how data gets into OpenTelemetry Collector. Generally, a receiver accepts data in a specified format and can support traces and/or metrics. The format of the traces and metrics supported are receiver specific.

Supported receivers (sorted alphabetically):

• Jaeger Receiver
• OpenCensus Receiver
• Prometheus Receiver
• VM Metrics Receiver
• Zipkin Receiver

Configuring Receiver(s)

Receivers are configured via YAML under the top-level receivers tag. There must be at least one enabled receiver for this configuration to be considered valid.

The following is a sample configuration for the examplereceiver.

receivers:
  # Receiver 1.
  # <receiver type>:
  examplereceiver:
    # <setting one>: <value one>
    endpoint: 1.2.3.4:8080
    # ...
  # Receiver 2.
  # <receiver type>/<name>:
  examplereceiver/settings:
    # <setting one>: <value one>
    disabled: false
    # <setting two>: <value two>
    endpoint: localhost:9211

A receiver instance is referenced by its full name in other parts of the config, such as in pipelines. A full name consists of the receiver type, '/' and the name appended to the receiver type in the configuration. All receiver full names must be unique. For the example above:

• Receiver 1 has full name examplereceiver.
• Receiver 2 has full name examplereceiver/settings.

All receivers expose a setting to disable it, by default receivers are enabled. At least one receiver must be enabled per pipeline to be a valid configuration.

OpenCensus Receiver

Traces and metrics are supported.

This receiver receives trace and metrics from OpenCensus instrumented applications. It translates them into the internal format sent to processors and exporters in the pipeline.

To get started, all that is required to enable the OpenCensus receiver is to include it in the receiver definitions. This will enable the default values as specified here. The following is an example:

receivers:
  opencensus:

The full list of settings exposed for this receiver are documented here with detailed sample configurations here.

Communicating over TLS

This receiver supports communication using Transport Layer Security (TLS). TLS can be configured by specifying a tls-crendentials object in the receiver configuration for receivers that support it.

receivers:
  opencensus:
    tls_credentials:
      key_file: /key.pem # path to private key
      cert_file: /cert.pem # path to certificate

Writing with HTTP/JSON

The OpenCensus receiver for the agent can receive trace export calls via HTTP/JSON in addition to gRPC. The HTTP/JSON address is the same as gRPC as the protocol is recognized and processed accordingly.

To write traces with HTTP/JSON, POST to [address]/v1/trace. The JSON message format parallels the gRPC protobuf format, see this OpenApi spec for it.

The HTTP/JSON endpoint can also optionally CORS, which is enabled by specifying a list of allowed CORS origins in the cors_allowed_origins field:

receivers:
  opencensus:
    endpoint: "localhost:55678"
    cors_allowed_origins:
    - http://test.com
    # Origins can have wildcards with *, use * by itself to match any origin.
    - https://*.example.com  

Jaeger Receiver

Only traces are supported.

This receiver receives traces in the Jaeger format. It translates them into the internal format and sends it to processors and exporters.

It supports the Jaeger Collector and Agent protocols:

• gRPC
• Thrift HTTP
• Thrift TChannel
• Thrift Compact
• Thrift Binary

By default, the Jaeger receiver will not serve any protocol. A protocol must be named for the jaeger receiver to start. The following demonstrates how to start the Jaeger receiver with only gRPC enabled on the default port.

receivers:
  jaeger:
    protocols:
      grpc:

It is possible to configure the protocols on different ports, refer to config.yaml for detailed config examples.

Communicating over TLS

This receiver supports communication using Transport Layer Security (TLS), but only using the gRPC protocol. It can be configured by specifying a tls-crendentials object in the gRPC receiver configuration.

receivers:
  jaeger:
    protocols:
      grpc:
        tls_credentials:
          key_file: /key.pem # path to private key
          cert_file: /cert.pem # path to certificate
        endpoint: "localhost:9876"

Remote Sampling

The Jaeger receiver also supports fetching sampling configuration from a remote collector. It works by proxying client requests for remote sampling configuration to the configured collector.

+---------------+ +--------------+ +-----------------+ | | get | | proxy | | | client +--- sampling ---->+ agent +------------->+ collector | | | strategy | | | | +---------------+ +--------------+ +-----------------+

Remote sampling can be enabled by specifying the following lines in the jaeger receiver config:

receivers:
  jaeger:
    protocols:
      grpc:
      .
      .
    remote_sampling:
      fetch_endpoint: "jaeger-collector:1234"

Prometheus Receiver

Only metrics are supported.

This receiver is a drop-in replacement for getting Prometheus to scrape your services. Just like you would write in a YAML configuration file before starting Prometheus, such as with:

prometheus --config.file=prom.yaml

you can copy and paste that same configuration under section

receivers:
  prometheus:
    config:

such as:

receivers:
    prometheus:
      config:
        scrape_configs:
          - job_name: 'opencensus_service'
            scrape_interval: 5s
            static_configs:
              - targets: ['localhost:8889']

          - job_name: 'jdbc_apps'
            scrape_interval: 3s
            static_configs:
              - targets: ['localhost:9777']

Include Filter

Include Filter provides ability to filter scraping metrics per target. If a filter is specified for a target then only those metrics which exactly matches one of the metrics specified in the Include Filter list will be scraped. Rest of the metrics from the targets will be dropped.

Syntax

• Endpoint should be double quoted.
• Metrics should be specified in form of a list.

Example

receivers:
    prometheus:
      include_filter: {
        "localhost:9777" : [http/server/server_latency, custom_metric1],
        "localhost:9778" : [http/client/roundtrip_latency],                
      }
      config:
        scrape_configs:
          ...

VM Metrics Receiver

Only metrics are supported.

<Add more information - I'm lonely.>

Zipkin Receiver

Only traces are supported.

This receiver receives spans from Zipkin (V1 and V2) HTTP uploads and translates them into the internal span types that are then sent to the collector/exporters.

Its address can be configured in the YAML configuration file under section "receivers", subsection "zipkin" and field "address". The syntax of the field "address" is [address|host]:<port-number>.

For example:

receivers:
  zipkin:
    address: "localhost:9411"

Common Configuration Errors

Documentation

Overview

Example (EndToEnd)

package main

import (
	"context"
	"log"
	"time"

	"contrib.go.opencensus.io/exporter/ocagent"
	"go.opencensus.io/trace"
	"go.uber.org/zap"

	"github.com/open-telemetry/opentelemetry-collector/config/configmodels"
	"github.com/open-telemetry/opentelemetry-collector/exporter/loggingexporter"
	"github.com/open-telemetry/opentelemetry-collector/receiver"
	"github.com/open-telemetry/opentelemetry-collector/receiver/opencensusreceiver"
)

func main() {
	// This is what the cmd/ocagent code would look like this.
	// A trace receiver as per the trace receiver
	// configs that have been parsed.
	lte, err := loggingexporter.NewTraceExporter(&configmodels.ExporterSettings{}, zap.NewNop())
	if err != nil {
		log.Fatalf("Failed to create logging exporter: %v", err)
	}

	tr, err := opencensusreceiver.New("localhost:55678", lte, nil)
	if err != nil {
		log.Fatalf("Failed to create trace receiver: %v", err)
	}

	// The agent will combine all trace receivers like this.
	trl := []receiver.TraceReceiver{tr}

	// Once we have the span receiver which will connect to the
	// various exporter pipeline i.e. *tracepb.Span->OpenCensus.SpanData
	for _, tr := range trl {
		if err := tr.Start(nil); err != nil {
			log.Fatalf("Failed to start trace receiver: %v", err)
		}
	}

	// Before exiting, stop all the trace receivers
	defer func() {
		for _, tr := range trl {
			_ = tr.Shutdown()
		}
	}()
	log.Println("Done starting the trace receiver")
	// We are done with the agent-core

	// Now this code would exist in the client application e.g. client code.
	// Create the agent exporter
	oce, err := ocagent.NewExporter(ocagent.WithInsecure())
	if err != nil {
		log.Fatalf("Failed to create ocagent exporter: %v", err)
	}
	defer oce.Stop()

	// Register it as a trace exporter
	trace.RegisterExporter(oce)
	// For demo purposes we are always sampling
	trace.ApplyConfig(trace.Config{DefaultSampler: trace.AlwaysSample()})

	log.Println("Starting loop")
	ctx, span := trace.StartSpan(context.Background(), "ClientLibrarySpan")
	for i := 0; i < 10; i++ {
		_, span := trace.StartSpan(ctx, "ChildSpan")
		span.Annotatef([]trace.Attribute{
			trace.StringAttribute("type", "Child"),
			trace.Int64Attribute("i", int64(i)),
		}, "This is an annotation")
		<-time.After(100 * time.Millisecond)
		span.End()
		oce.Flush()
	}
	span.End()

	<-time.After(400 * time.Millisecond)
	oce.Flush()
	<-time.After(5 * time.Second)
}

Index

• func Build(factories ...Factory) (map[string]Factory, error)
• type CustomUnmarshaler
• type Factory
• type MetricsReceiver
• type Receiver
• type SecureReceiverSettings
• type TLSCredentials
  • func (tlsCreds *TLSCredentials) ToGrpcServerOption() (opt grpc.ServerOption, err error)
• type TraceReceiver

Examples

• Package (EndToEnd)

Functions

func Build

func Build(factories ...Factory) (map[string]Factory, error)

Build takes a list of receiver factories and returns a map of type map[string]Factory with factory type as keys. It returns a non-nil error when more than one factories have the same type.

Types

type CustomUnmarshaler

type CustomUnmarshaler func(v *viper.Viper, viperKey string, sourceViperSection *viper.Viper, intoCfg interface{}) error

CustomUnmarshaler is a function that un-marshals a viper data into a config struct in a custom way. v *viper.Viper

A viper instance at the "receivers" node in the config yaml.  v.Sub(viperKey) is
the raw config this function should load.

viperKey string

The name of this config.  i.e. "jaeger/custom".  v.Sub(viperKey) is the raw config
this function should load.

sourceViperSection *viper.Viper

The value of v.Sub(viperKey) with all environment substitution complete.

intoCfg interface{}

An empty interface wrapping a pointer to the config struct to unmarshal into.

type Factory

type Factory interface {
	// Type gets the type of the Receiver created by this factory.
	Type() string

	// CreateDefaultConfig creates the default configuration for the Receiver.
	// This method can be called multiple times depending on the pipeline
	// configuration and should not cause side-effects that prevent the creation
	// of multiple instances of the Receiver.
	// The object returned by this method needs to pass the checks implemented by
	// 'configcheck.ValidateConfig'. It is recommended to have such check in the
	// tests of any implementation of the Factory interface.
	CreateDefaultConfig() configmodels.Receiver

	// CustomUnmarshaler returns a custom unmarshaler for the configuration or nil if
	// there is no need for custom unmarshaling. This is typically used if viper.UnmarshalExact()
	// is not sufficient to unmarshal correctly.
	CustomUnmarshaler() CustomUnmarshaler

	// CreateTraceReceiver creates a trace receiver based on this config.
	// If the receiver type does not support tracing or if the config is not valid
	// error will be returned instead.
	CreateTraceReceiver(ctx context.Context, logger *zap.Logger, cfg configmodels.Receiver,
		nextConsumer consumer.TraceConsumer) (TraceReceiver, error)

	// CreateMetricsReceiver creates a metrics receiver based on this config.
	// If the receiver type does not support metrics or if the config is not valid
	// error will be returned instead.
	CreateMetricsReceiver(logger *zap.Logger, cfg configmodels.Receiver,
		consumer consumer.MetricsConsumer) (MetricsReceiver, error)
}

Factory is factory interface for receivers.

type MetricsReceiver

type MetricsReceiver interface {
	Receiver

	// MetricsSource returns the name of the metrics data source.
	MetricsSource() string
}

A MetricsReceiver is an "arbitrary data"-to-"metric proto" converter. Its purpose is to translate data from the wild into metric proto accompanied by a *commonpb.Node to uniquely identify where that data comes from. MetricsReceiver feeds a consumer.MetricsConsumer with data.

For example it could be Prometheus data source which translates Prometheus metrics into *metricpb.Metric-s.

type Receiver

type Receiver interface {
	component.Component
}

Receiver defines functions that trace and metric receivers must implement.

type SecureReceiverSettings

type SecureReceiverSettings struct {
	configmodels.ReceiverSettings `mapstructure:",squash"` // squash ensures fields are correctly decoded in embedded struct
	// Configures the receiver to use TLS.
	// The default value is nil, which will cause the receiver to not use TLS.
	TLSCredentials *TLSCredentials `mapstructure:"tls_credentials, omitempty"`
}

SecureReceiverSettings defines common settings for receivers that use Transport Layer Security (TLS)

type TLSCredentials

type TLSCredentials struct {
	// CertFile is the file path containing the TLS certificate.
	CertFile string `mapstructure:"cert_file"`

	// KeyFile is the file path containing the TLS key.
	KeyFile string `mapstructure:"key_file"`
}

TLSCredentials contains path information for a certificate and key to be used for TLS

func (*TLSCredentials) ToGrpcServerOption

func (tlsCreds *TLSCredentials) ToGrpcServerOption() (opt grpc.ServerOption, err error)

ToGrpcServerOption creates a gRPC ServerOption from TLSCredentials. If TLSCredentials is nil, returns empty option.

type TraceReceiver

type TraceReceiver interface {
	Receiver

	// TraceSource returns the name of the trace data source.
	TraceSource() string
}

A TraceReceiver is an "arbitrary data"-to-"trace proto span" converter. Its purpose is to translate data from the wild into trace proto accompanied by a *commonpb.Node to uniquely identify where that data comes from. TraceReceiver feeds a consumer.TraceConsumer with data.

For example it could be Zipkin data source which translates Zipkin spans into *tracepb.Span-s.