service

README

OpenTelemetry Collector Service

How to provide configuration?

The --config flag accepts either a file path or values in the form of a config URI "<scheme>:<opaque_data>". Currently, the OpenTelemetry Collector supports the following providers scheme:

• file - Reads configuration from a file. E.g. file:path/to/config.yaml.
• env - Reads configuration from an environment variable. E.g. env:MY_CONFIG_IN_AN_ENVVAR.
• yaml - Reads configuration from yaml bytes. E.g. yaml:exporters::debug::verbosity: detailed.
• http - Reads configuration from a HTTP URI. E.g. http://www.example.com

For more technical details about how configuration is resolved you can read the configuration resolving design.

Single Config Source

1. Simple local file:

   ./otelcorecol --config=examples/local/otel-config.yaml

2. Simple local file using the new URI format:

   ./otelcorecol --config=file:examples/local/otel-config.yaml

3. Config provided via an environment variable:

   ./otelcorecol --config=env:MY_CONFIG_IN_AN_ENVVAR

Multiple Config Sources

1. Merge a otel-config.yaml file with the content of an environment variable MY_OTHER_CONFIG and use the merged result as the config:

   ./otelcorecol --config=file:examples/local/otel-config.yaml --config=env:MY_OTHER_CONFIG

2. Merge a config.yaml file with the content of a yaml bytes configuration (overwrites the exporters::debug::verbosity config) and use the content as the config:

   ./otelcorecol --config=file:examples/local/otel-config.yaml --config="yaml:exporters::debug::verbosity: normal"

Embedding other configuration providers

One configuration provider can also make references to other config providers, like the following:

receivers:
  otlp:
    protocols:
      grpc:

exporters: ${file:otlp-exporter.yaml}

service:
  extensions: [ ]
  pipelines:
    traces:
      receivers:  [ otlp ]
      processors: [  ]
      exporters:  [ otlp ]

How to override config properties?

The --set flag allows to set arbitrary config property. The --set values are merged into the final configuration after all the sources specified by the --config are resolved and merged.

The Format and Limitations of --set

Simple property

The --set option takes always one key/value pair, and it is used like this: --set key=value. The YAML equivalent of that is:

key: value

Complex nested keys

Use dot (.) in the pair's name as key separator to reference nested map values. For example, --set outer.inner=value is translated into this:

outer:
  inner: value

Multiple values

To set multiple values specify multiple --set flags, so --set a=b --set c=d becomes:

a: b
c: d

Array values

Arrays can be expressed by enclosing values in []. For example, --set "key=[a, b, c]" translates to:

key:
  - a
  - b
  - c

Map values

Maps can be expressed by enclosing values in {}. For example, "--set "key={a: c}" translates to:

key:
  a: c

Limitations

1. Does not support setting a key that contains a dot ..
2. Does not support setting a key that contains a equal sign =.
3. The configuration key separator inside the value part of the property is "::". For example --set "name={a::b: c}" is equivalent with --set name.a.b=c.

How to check components available in a distribution

Use the sub command build-info. Below is an example:

   ./otelcorecol components

Sample output:

buildinfo:
   command: otelcorecol
   description: Local OpenTelemetry Collector binary, testing only.
   version: 0.62.1-dev
receivers:
   - otlp
processors:
   - memory_limiter
   - batch
exporters:
   - otlp
   - otlphttp
   - debug
extensions:
   - zpages

How to validate configuration file and return all errors without running collector

   ./otelcorecol validate --config=file:examples/local/otel-config.yaml

How to examine the final configuration after merging and resolving from various sources?

   ./otelcorecol print-initial-config --config=file:file.yaml --config=http:http://remote:8080/config --config=file:file2.yaml

Documentation

Index

• func Validate(ctx context.Context, set Settings, cfg Config) error
• type Config
• type ModuleInfo
• type ModuleInfos
• type Service
  • func New(ctx context.Context, set Settings, cfg Config) (*Service, error)
  • func (srv *Service) Logger() *zap.Logger
  • func (srv *Service) Shutdown(ctx context.Context) error
  • func (srv *Service) Start(ctx context.Context) error
• type Settings

Functions

func Validate

func Validate(ctx context.Context, set Settings, cfg Config) error

Validate verifies the graph by calling the internal graph.Build.

Types

type Config

type Config struct {
	// Telemetry is the configuration for collector's own telemetry.
	Telemetry telemetry.Config `mapstructure:"telemetry"`

	// Extensions are the ordered list of extensions configured for the service.
	Extensions extensions.Config `mapstructure:"extensions,omitempty"`

	// Pipelines are the set of data pipelines configured for the service.
	Pipelines pipelines.Config `mapstructure:"pipelines"`
	// contains filtered or unexported fields
}

Config defines the configurable components of the Service.

type ModuleInfo

type ModuleInfo = moduleinfo.ModuleInfo

ModuleInfo describes the Go module for a particular component.

type ModuleInfos

type ModuleInfos = moduleinfo.ModuleInfos

ModuleInfos describes the go module for all components.

type Service

type Service struct {
	// contains filtered or unexported fields
}

Service represents the implementation of a component.Host.

func New

func New(ctx context.Context, set Settings, cfg Config) (*Service, error)

New creates a new Service, its telemetry, and Components.

func (*Service) Logger

func (srv *Service) Logger() *zap.Logger

Logger returns the logger created for this service. This is a temporary API that may be removed soon after investigating how the collector should record different events.

func (*Service) Shutdown

func (srv *Service) Shutdown(ctx context.Context) error

Shutdown the service. Shutdown will do the following steps in order: 1. Notify extensions that the pipeline is shutting down. 2. Shutdown all pipelines. 3. Shutdown all extensions. 4. Shutdown telemetry.

func (*Service) Start

func (srv *Service) Start(ctx context.Context) error

Start starts the extensions and pipelines. If Start fails Shutdown should be called to ensure a clean state. Start does the following steps in order: 1. Start all extensions. 2. Notify extensions about Collector configuration 3. Start all pipelines. 4. Notify extensions that the pipeline is ready.

type Settings

type Settings struct {
	// BuildInfo provides collector start information.
	BuildInfo component.BuildInfo

	// CollectorConf contains the Collector's current configuration
	CollectorConf *confmap.Conf

	// Receivers configuration to its builder.
	ReceiversConfigs   map[component.ID]component.Config
	ReceiversFactories map[component.Type]receiver.Factory

	// Processors configuration to its builder.
	ProcessorsConfigs   map[component.ID]component.Config
	ProcessorsFactories map[component.Type]processor.Factory

	// exporters configuration to its builder.
	ExportersConfigs   map[component.ID]component.Config
	ExportersFactories map[component.Type]exporter.Factory

	// Connectors configuration to its builder.
	ConnectorsConfigs   map[component.ID]component.Config
	ConnectorsFactories map[component.Type]connector.Factory

	// Extensions builder for extensions.
	Extensions builders.Extension

	// Extensions configuration to its builder.
	ExtensionsConfigs   map[component.ID]component.Config
	ExtensionsFactories map[component.Type]extension.Factory

	// ModuleInfo describes the go module for each component.
	ModuleInfos ModuleInfos

	// AsyncErrorChannel is the channel that is used to report fatal errors.
	AsyncErrorChannel chan error

	// LoggingOptions provides a way to change behavior of zap logging.
	LoggingOptions []zap.Option
}

Settings holds configuration for building a new Service.