receiver

README

Note This documentation is still in progress. For any questions, please reach out in the OpenTelemetry Gitter or refer to the issues page.

Receivers

A receiver is how data gets into OpenTelemetry Service. Generally, a receiver accepts data in a specified format and can support traces and/or metrics. The format of the traces and metrics supported are receiver specific.

Supported receivers (sorted alphabetically):

• Jaeger Receiver
• OpenCensus Receiver
• Prometheus Receiver
• VM Metrics Receiver
• Zipkin Receiver

Configuring Receiver(s)

Receivers are configured via YAML under the top-level receivers tag. There must be at least one enabled receiver for this configuration to be considered valid.

The following is a sample configuration for the examplereceiver.

receivers:
  # Receiver 1.
  # <receiver type>:
  examplereceiver:
    # <setting one>: <value one>
    endpoint: 1.2.3.4:8080
    # ...
  # Receiver 2.
  # <receiver type>/<name>:
  examplereceiver/settings:
    # <setting one>: <value one>
    disabled: false
    # <setting two>: <value two>
    endpoint: 127.0.0.1:9211

A receiver instance is referenced by its full name in other parts of the config, such as in pipelines. A full name consists of the receiver type, '/' and the name appended to the receiver type in the configuration. All receiver full names must be unique. For the example above:

• Receiver 1 has full name examplereceiver.
• Receiver 2 has full name examplereceiver/settings.

All receivers expose a setting to disable it, by default receivers are enabled. At least one receiver must be enabled per pipeline to be a valid configuration.

OpenCensus Receiver

Traces and metrics are supported.

This receiver receives trace and metrics from OpenCensus instrumented applications. It translates them into the internal format sent to processors and exporters in the pipeline.

To get started, all that is required to enable the OpenCensus receiver is to include it in the receiver definitions. This will enable the default values as specified here. The following is an example:

receivers:
  opencensus:

The full list of settings exposed for this receiver are documented here with detailed sample configurations here.

Communicating over TLS

This receiver supports communication using Transport Layer Security (TLS). TLS can be configured by specifying a tls-crendentials object in the receiver configuration for receivers that support it.

receivers:
  opencensus:
    tls-credentials:
      key-file: /key.pem # path to private key
      cert-file: /cert.pem # path to certificate

Writing with HTTP/JSON

The OpenCensus receiver for the agent can receive trace export calls via HTTP/JSON in addition to gRPC. The HTTP/JSON address is the same as gRPC as the protocol is recognized and processed accordingly.

To write traces with HTTP/JSON, POST to [address]/v1/trace. The JSON message format parallels the gRPC protobuf format, see this OpenApi spec for it.

The HTTP/JSON endpoint can also optionally CORS, which is enabled by specifying a list of allowed CORS origins in the cors-allowed-origins field:

receivers:
  opencensus:
    endpoint: "localhost:55678"
    cors-allowed-origins:
    - http://test.com
    # Origins can have wildcards with *, use * by itself to match any origin.
    - https://*.example.com  

Jaeger Receiver

Only traces are supported.

This receiver receives traces in the Jaeger format. It translates them into the internal format and sends it to processors and exporters.

It supports multiple protocols:

• Thrift HTTP
• Thrift TChannel
• gRPC

By default, the Jaeger receiver supports all three protocols on the default ports specified in factory.go. The following demonstrates how to specify the default Jaeger receiver.

receivers:
  jaeger:

It is possible to configure the protocols on different ports, refer to config.yaml for detailed config examples.

// TODO Issue https://github.com/open-telemetry/opentelemetry-service/issues/158 // The Jaeger receiver enables all protocols even when one is specified or a // subset is enabled. The documentation should be updated when that fix occurs.

Communicating over TLS

This receiver supports communication using Transport Layer Security (TLS), but only using the gRPC protocol. It can be configured by specifying a tls-crendentials object in the gRPC receiver configuration.

receivers:
  jaeger:
    protocols:
      grpc:
        tls-credentials:
          key-file: /key.pem # path to private key
          cert-file: /cert.pem # path to certificate
        endpoint: "127.0.0.1:9876"

Prometheus Receiver

Only metrics are supported.

This receiver is a drop-in replacement for getting Prometheus to scrape your services. Just like you would write in a YAML configuration file before starting Prometheus, such as with:

prometheus --config.file=prom.yaml

you can copy and paste that same configuration under section

receivers:
  prometheus:
    config:

such as:

receivers:
    prometheus:
      config:
        scrape_configs:
          - job_name: 'opencensus_service'
            scrape_interval: 5s
            static_configs:
              - targets: ['localhost:8889']

          - job_name: 'jdbc_apps'
            scrape_interval: 3s
            static_configs:
              - targets: ['localhost:9777']

Include Filter

Include Filter provides ability to filter scraping metrics per target. If a filter is specified for a target then only those metrics which exactly matches one of the metrics specified in the Include Filter list will be scraped. Rest of the metrics from the targets will be dropped.

Syntax

• Endpoint should be double quoted.
• Metrics should be specified in form of a list.

Example

receivers:
    prometheus:
      include_filter: {
        "localhost:9777" : [http/server/server_latency, custom_metric1],
        "localhost:9778" : [http/client/roundtrip_latency],                
      }
      config:
        scrape_configs:
          ...

VM Metrics Receiver

Only metrics are supported.

<Add more information - I'm lonely.>

Zipkin Receiver

Only traces are supported.

This receiver receives spans from Zipkin (V1 and V2) HTTP uploads and translates them into the internal span types that are then sent to the collector/exporters.

Its address can be configured in the YAML configuration file under section "receivers", subsection "zipkin" and field "address". The syntax of the field "address" is [address|host]:<port-number>.

For example:

receivers:
  zipkin:
    address: "127.0.0.1:9411"

Common Configuration Errors

Documentation

Overview

Example (EndToEnd)

package main

import (
	"context"
	"log"
	"time"

	"contrib.go.opencensus.io/exporter/ocagent"
	"go.opencensus.io/trace"
	"go.uber.org/zap"

	"github.com/open-telemetry/opentelemetry-service/config/configmodels"
	"github.com/open-telemetry/opentelemetry-service/exporter/loggingexporter"
	"github.com/open-telemetry/opentelemetry-service/receiver"
	"github.com/open-telemetry/opentelemetry-service/receiver/opencensusreceiver"
)

func main() {
	// This is what the cmd/ocagent code would look like this.
	// A trace receiver as per the trace receiver
	// configs that have been parsed.
	lte, err := loggingexporter.NewTraceExporter(&configmodels.ExporterSettings{}, zap.NewNop())
	if err != nil {
		log.Fatalf("Failed to create logging exporter: %v", err)
	}

	tr, err := opencensusreceiver.New("localhost:55678", lte, nil)
	if err != nil {
		log.Fatalf("Failed to create trace receiver: %v", err)
	}

	// The agent will combine all trace receivers like this.
	trl := []receiver.TraceReceiver{tr}

	// Once we have the span receiver which will connect to the
	// various exporter pipeline i.e. *tracepb.Span->OpenCensus.SpanData
	for _, tr := range trl {
		if err := tr.StartTraceReception(nil); err != nil {
			log.Fatalf("Failed to start trace receiver: %v", err)
		}
	}

	// Before exiting, stop all the trace receivers
	defer func() {
		for _, tr := range trl {
			_ = tr.StopTraceReception()
		}
	}()
	log.Println("Done starting the trace receiver")
	// We are done with the agent-core

	// Now this code would exist in the client application e.g. client code.
	// Create the agent exporter
	oce, err := ocagent.NewExporter(ocagent.WithInsecure())
	if err != nil {
		log.Fatalf("Failed to create ocagent exporter: %v", err)
	}
	defer oce.Stop()

	// Register it as a trace exporter
	trace.RegisterExporter(oce)
	// For demo purposes we are always sampling
	trace.ApplyConfig(trace.Config{DefaultSampler: trace.AlwaysSample()})

	log.Println("Starting loop")
	ctx, span := trace.StartSpan(context.Background(), "ClientLibrarySpan")
	for i := 0; i < 10; i++ {
		_, span := trace.StartSpan(ctx, "ChildSpan")
		span.Annotatef([]trace.Attribute{
			trace.StringAttribute("type", "Child"),
			trace.Int64Attribute("i", int64(i)),
		}, "This is an annotation")
		<-time.After(100 * time.Millisecond)
		span.End()
		oce.Flush()
	}
	span.End()

	<-time.After(400 * time.Millisecond)
	oce.Flush()
	<-time.After(5 * time.Second)
}

Index

• func Build(factories ...Factory) (map[string]Factory, error)
• type CustomUnmarshaler
• type Factory
• type Host
• type MetricsReceiver
• type SecureReceiverSettings
• type TLSCredentials
  • func (tlsCreds *TLSCredentials) ToGrpcServerOption() (opt grpc.ServerOption, err error)
• type TraceReceiver

Examples

• Package (EndToEnd)

Functions

func Build

func Build(factories ...Factory) (map[string]Factory, error)

Build takes a list of receiver factories and returns a map of type map[string]Factory with factory type as keys. It returns a non-nil error when more than one factories have the same type.

Types

type CustomUnmarshaler

type CustomUnmarshaler func(v *viper.Viper, viperKey string, intoCfg interface{}) error

CustomUnmarshaler is a function that un-marshals a viper data into a config struct in a custom way.

type Factory

type Factory interface {
	// Type gets the type of the Receiver created by this factory.
	Type() string

	// CreateDefaultConfig creates the default configuration for the Receiver.
	CreateDefaultConfig() configmodels.Receiver

	// CustomUnmarshaler returns a custom unmarshaler for the configuration or nil if
	// there is no need for custom unmarshaling. This is typically used if viper.Unmarshal()
	// is not sufficient to unmarshal correctly.
	CustomUnmarshaler() CustomUnmarshaler

	// CreateTraceReceiver creates a trace receiver based on this config.
	// If the receiver type does not support tracing or if the config is not valid
	// error will be returned instead.
	CreateTraceReceiver(ctx context.Context, logger *zap.Logger, cfg configmodels.Receiver,
		nextConsumer consumer.TraceConsumer) (TraceReceiver, error)

	// CreateMetricsReceiver creates a metrics receiver based on this config.
	// If the receiver type does not support metrics or if the config is not valid
	// error will be returned instead.
	CreateMetricsReceiver(logger *zap.Logger, cfg configmodels.Receiver,
		consumer consumer.MetricsConsumer) (MetricsReceiver, error)
}

Factory is factory interface for receivers.

type Host

type Host interface {
	// Context returns a context provided by the host to be used on the receiver
	// operations.
	Context() context.Context

	// ReportFatalError is used to report to the host that the receiver encountered
	// a fatal error (i.e.: an error that the instance can't recover from) after
	// its start function has already returned.
	ReportFatalError(err error)
}

Host represents the entity where the receiver is being hosted. It is used to allow communication between the receiver and its host.

type MetricsReceiver

type MetricsReceiver interface {
	// MetricsSource returns the name of the metrics data source.
	MetricsSource() string

	// StartMetricsReception tells the receiver to start its processing.
	// By convention the consumer of the data received is set at creation time.
	StartMetricsReception(host Host) error

	// StopMetricsReception tells the receiver that should stop reception,
	// giving it a chance to perform any necessary clean-up.
	StopMetricsReception() error
}

A MetricsReceiver is an "arbitrary data"-to-"metric proto" converter. Its purpose is to translate data from the wild into metric proto accompanied by a *commonpb.Node to uniquely identify where that data comes from. MetricsReceiver feeds a consumer.MetricsConsumer with data.

For example it could be Prometheus data source which translates Prometheus metrics into *metricpb.Metric-s.

type SecureReceiverSettings

type SecureReceiverSettings struct {
	configmodels.ReceiverSettings `mapstructure:",squash"` // squash ensures fields are correctly decoded in embedded struct
	// Configures the receiver to use TLS.
	// The default value is nil, which will cause the receiver to not use TLS.
	TLSCredentials *TLSCredentials `mapstructure:"tls-credentials, omitempty"`
}

SecureReceiverSettings defines common settings for receivers that use Transport Layer Security (TLS)

type TLSCredentials

type TLSCredentials struct {
	// CertFile is the file path containing the TLS certificate.
	CertFile string `mapstructure:"cert-file"`

	// KeyFile is the file path containing the TLS key.
	KeyFile string `mapstructure:"key-file"`
}

TLSCredentials contains path information for a certificate and key to be used for TLS

func (*TLSCredentials) ToGrpcServerOption

func (tlsCreds *TLSCredentials) ToGrpcServerOption() (opt grpc.ServerOption, err error)

ToGrpcServerOption creates a gRPC ServerOption from TLSCredentials. If TLSCredentials is nil, returns empty option.

type TraceReceiver

type TraceReceiver interface {
	// TraceSource returns the name of the trace data source.
	TraceSource() string

	// StartTraceReception tells the receiver to start its processing.
	// By convention the consumer of the data received is set at creation time.
	StartTraceReception(host Host) error

	// StopTraceReception tells the receiver that should stop reception,
	// giving it a chance to perform any necessary clean-up.
	StopTraceReception() error
}

A TraceReceiver is an "arbitrary data"-to-"trace proto span" converter. Its purpose is to translate data from the wild into trace proto accompanied by a *commonpb.Node to uniquely identify where that data comes from. TraceReceiver feeds a consumer.TraceConsumer with data.

For example it could be Zipkin data source which translates Zipkin spans into *tracepb.Span-s.