kafkaexporter

README

Kafka Exporter

Kafka exporter exports traces to Kafka. This exporter uses a synchronous producer that blocks and does not batch messages, therefore it should be used with batch and queued retry processors for higher throughput and resiliency.

Message payloads are serialized OTLP ExportTraceServiceRequest.

The following settings are required:

• protocol_version (no default): Kafka protocol version e.g. 2.0.0

The following settings can be optionally configured:

• brokers (default = localhost:9092): The list of kafka brokers
• topic (default = otlp_spans): The name of the kafka topic to export to
• metadata.full (default = true): Whether to maintain a full set of metadata. When disabled the client does not make the initial request to broker at the startup.
• metadata.retry.max (default = 3): The number of retries to get metadata
• metadata.retry.backoff (default = 250ms): How long to wait between metadata retries

Example configuration:

exporters:
  kafka:
    brokers:
      - localhost:9092
    protocol_version: 2.0.0

Documentation

Index

• func NewFactory() component.ExporterFactory
• type Config
• type Metadata
• type MetadataRetry

Functions

func NewFactory

func NewFactory() component.ExporterFactory

NewFactory creates Kafka exporter factory.

Types

type Config

type Config struct {
	configmodels.ExporterSettings `mapstructure:",squash"`
	// The list of kafka brokers (default localhost:9092)
	Brokers []string `mapstructure:"brokers"`
	// Kafka protocol version
	ProtocolVersion string `mapstructure:"protocol_version"`
	// The name of the kafka topic to export to (default "otlp_spans")
	Topic string `mapstructure:"topic"`

	// Metadata is the namespace for metadata management properties used by the
	// Client, and shared by the Producer/Consumer.
	Metadata Metadata `mapstructure:"metadata"`
}

Config defines configuration for Kafka exporter.

type Metadata

type Metadata struct {
	// Whether to maintain a full set of metadata for all topics, or just
	// the minimal set that has been necessary so far. The full set is simpler
	// and usually more convenient, but can take up a substantial amount of
	// memory if you have many topics and partitions. Defaults to true.
	Full bool `mapstructure:"full"`

	// Retry configuration for metadata.
	// This configuration is useful to avoid race conditions when broker
	// is starting at the same time as collector.
	Retry MetadataRetry `mapstructure:"retry"`
}

Metadata defines configuration for retrieving metadata from the broker.

type MetadataRetry

type MetadataRetry struct {
	// The total number of times to retry a metadata request when the
	// cluster is in the middle of a leader election or at startup (default 3).
	Max int `mapstructure:"max"`
	// How long to wait for leader election to occur before retrying
	// (default 250ms). Similar to the JVM's `retry.backoff.ms`.
	Backoff time.Duration `mapstructure:"backoff"`
}

MetadataRetry defines retry configuration for Metadata.