awsfirehosereceiver

README

AWS Kinesis Data Firehose Receiver

The AWS Kinesis Data Firehose Receiver is for ingesting delivery stream messages and parsing the records received based on the configured record type.

Status
Stability	alpha: metrics, logs
Distributions	contrib
Issues
Code coverage
Code Owners	@Aneurysm9, @axw

Configuration

Example:

receivers:
  awsfirehose:
    endpoint: 0.0.0.0:4433
    encoding: awscloudwatchmetricstreams_encoding
    access_key: "some_access_key"
    tls:
      cert_file: server.crt
      key_file: server.key

extensions:
  awscloudwatchmetricstreams_encoding:
    format: json

The configuration includes the OpenTelemetry collector's server confighttp, which allows for a variety of settings. Only the most relevant ones will be discussed here, but all are available.

The Amazon Data Firehose Delivery Streams currently only support HTTPS endpoints using port 443. This can be potentially circumvented using a Load Balancer.

endpoint:

The address:port to bind the listener to.

default: localhost:4433

See our security best practices doc to understand how to set the endpoint in different environments.

tls:

See documentation for more details.

A cert_file and key_file are required.

encoding:

The ID of an encoding extension for decoding logs or metrics. This configuration also supports the built-in encodings listed in the Encodings section.

If no encoding is specified, then the receiver will default to a signal-specific encoding: cwmetrics for metrics, and cwlogs for logs. In the future, the default encodings will be removed, and it will be required to specify an encoding extension.

record_type:

Deprecated, use encoding instead. record_type will be removed in a future release; it is an alias for encoding.

access_key (Optional):

The access key to be checked on each request received. This can be set when creating or updating the delivery stream. See documentation for details.

Encodings

Any encoding extension can be used with this receiver. There are additionally built-in encodings, which are deprecated. In the future it will be required to specify an encoding extension, and the built-in encodings will be removed.

cwmetrics (deprecated)

The built-in cwmetrics encoding is deprecated and will be removed in a future version. Use the awscloudwatchmetricstreams_encoding extension with format: json instead.

extensions:
  awscloudwatchmetricstreams_encoding:
    format: json
receivers:
  awsfirehose:
    encoding: awscloudwatchmetricstreams_encoding

cwlogs (deprecated)

The built-in cwlogs encoding is deprecated and will be removed in a future version. Use the aws_logs_encoding extension with format: cloudwatch instead.

The receiver detects and decompresses gzip-compressed Firehose records before invoking any configured logs encoding.

extensions:
  aws_logs_encoding:
    format: cloudwatch
receivers:
  awsfirehose:
    encoding: aws_logs_encoding

otlp_v1 (deprecated)

The built-in otlp_v1 encoding is deprecated and will be removed in a future version. Use the awscloudwatchmetricstreams_encoding extension with format: opentelemetry1.0 instead.

extensions:
  awscloudwatchmetricstreams_encoding:
    format: opentelemetry1.0
receivers:
  awsfirehose:
    encoding: awscloudwatchmetricstreams_encoding

Documentation

Index

• func NewFactory() receiver.Factory
• type Config
  • func (c *Config) Validate() error

Functions

func NewFactory

func NewFactory() receiver.Factory

NewFactory creates a receiver factory for awsfirehose. Currently, only available in metrics pipelines.

Types

type Config

type Config struct {
	// ServerConfig is used to set up the Firehose delivery
	// endpoint. The Firehose delivery stream expects an HTTPS
	// endpoint, so TLSs must be used to enable that.
	confighttp.ServerConfig `mapstructure:",squash"`
	// Encoding identifies the encoding of records received from
	// Firehose.
	//
	// Defaults to telemetry-specific encodings: "cwlog" for logs,
	// and "cwmetrics" for metrics. This default behavior is
	// deprecated as of v0.149.0, and in the future it will be
	// required to specify the encoding explicitly.
	Encoding string `mapstructure:"encoding"`
	// RecordType is an alias for Encoding for backwards compatibility.
	// It is an error to specify both encoding and record_type.
	//
	// Deprecated: [v0.121.0] use Encoding instead.
	RecordType string `mapstructure:"record_type"`
	// AccessKey is checked against the one received with each request.
	// This can be set when creating or updating the Firehose delivery
	// stream.
	AccessKey configopaque.String `mapstructure:"access_key"`
}

func (*Config) Validate

func (c *Config) Validate() error

Validate checks that the endpoint and record type exist and are valid.