awscloudwatchlogsexporter

README

AWS CloudWatch Logs Exporter

Status
Stability	alpha: logs
Distributions	contrib
Issues
Code coverage
Code Owners	@yaten2302 | Seeking more code owners!
Emeritus	@bryan-aguilar, @boostchicken, @rapphil

AWS CloudWatch Logs Exporter sends logs data to AWS CloudWatch Logs. AWS credentials are retrieved from the default credential chain.

NOTE: OpenTelemetry Logging support is experimental, hence this exporter is subject to change.

Configuration

The following settings are required:

• log_group_name: The group name of the CloudWatch Logs. If it does not exist it will be created automatically. It supports several placeholder names. One valid example is /aws/metrics/{ClusterName}. It will search for ClusterName (or aws.ecs.cluster.name) resource attribute in the metrics data and replace with the actual cluster name. If none of them are found in the resource attribute map, {ClusterName} will be replaced by undefined. Similar way, for the {TaskId}, it searches for TaskId (or aws.ecs.task.id) key in the resource attribute map. For {NodeName}, it searches for NodeName (or k8s.node.name)
  • List of valid placeholders:
    • {ClusterName}: aws.ecs.cluster.name
    • {TaskId}: aws.ecs.task.id
    • {NodeName}: k8s.node.name
    • {PodName}: pod
    • {ServiceName}: service.name
    • {ContainerInstanceId}: aws.ecs.container.instance.id
    • {TaskDefinitionFamily}: aws.ecs.task.family
    • {InstanceId}: service.instance.id
    • {FaasName}: faas.name
    • {FaasVersion}: faas.version
• log_stream_name: The stream name of the CloudWatch Logs. If it does not exist it will be created automatically. It supports the same placeholders as log_group_name

The following settings can be optionally configured:

• region: The AWS region where the log stream is in. Region must be specified if it is not already set in the default credential chain.
• endpoint: The CloudWatch Logs service endpoint which the requests are forwarded to. See the CloudWatch Logs endpoints for a list.
• log_retention: LogRetention is the option to set the log retention policy for only newly created CloudWatch Log Groups. Defaults to Never Expire if not specified or set to 0. Possible values for retention in days are 1, 3, 5, 7, 14, 30, 60, 90, 120, 150, 180, 365, 400, 545, 731, 1827, 2192, 2557, 2922, 3288, or 3653.
• tags: Tags is the option to set tags for the CloudWatch Log Group. If specified, please add at most 50 tags. Input is a string to string map like so: { 'key': 'value' }. Keys must be between 1-128 characters and follow the regex pattern: ^([\p{L}\p{Z}\p{N}_.:/=+\-@]+)$(alphanumerics, whitespace, and _.:/=+-!). Values must be between 1-256 characters and follow the regex pattern: ^([\p{L}\p{Z}\p{N}_.:/=+\-@]\*)$(alphanumerics, whitespace, and _.:/=+-!). Link to tagging restrictions
• raw_log: Boolean default false. If set to true, only the log message will be exported to CloudWatch Logs. This needs to be set to true for EMF logs.
• role_arn: IAM role to upload logs to a different account.
• external_id: Shared identitier used when assuming an IAM role in an external AWS account. See AWS IAM Guide
• sending_queue: Parameters for the sending queue, where you can control parallelism and the size of the sending buffer. Obs.: this component will always have a sending queue enabled.
  • num_consumers: Number of consumers that will consume from the sending queue. This parameter controls how many consumers will consume from the sending queue in parallel.
  • queue_size: Maximum number of batches kept in memory before dropping; ignored if enabled is false

Examples

Simplest configuration:

exporters:
  awscloudwatchlogs:
    log_group_name: "testing-logs"
    log_stream_name: "testing-integrations-stream"

Example configuration for EMF logs:

exporters:
  awscloudwatchlogs:
    log_group_name: "testing-logs-emf"
    log_stream_name: "testing-integrations-stream-emf"
    raw_log: true
    region: "us-east-1"
    endpoint: "logs.us-east-1.amazonaws.com"
    log_retention: 365
    tags: { "sampleKey": "sampleValue" }

Additional Notes

• If the log group and/or log stream are specified in an EMF log, that EMF log will be exported to that log group and/or log stream (i.e. ignores the log group and log stream defined in the configuration)
  • The log group and log stream will also be created automatically if they do not already exist.
  • Example of an EMF log with log group and log stream:

{
  "_aws": {
    "Timestamp": 1574109732004,
    "LogGroupName": "Foo",
    "LogStreamName": "Bar",
    "CloudWatchMetrics": [
      {
        "Namespace": "MyApp",
        "Dimensions": [["Operation"]],
        "Metrics": [
          {
            "Name": "ProcessingLatency",
            "Unit": "Milliseconds",
            "StorageResolution": 60
          }
        ]
      }
    ]
  },
  "Operation": "Aggregator",
  "ProcessingLatency": 100
}

• Resource ARNs (Amazon Resource Name (ARN) of the AWS resource running the collector) are currently not supported with the CloudWatch Logs Exporter.

Documentation

Overview

Package awscloudwatchlogsexporter provides an AWS CloudWatch Logs exporter for the OpenTelemetry collector. This package is subject to change and may break configuration settings and behavior.

Index

• func NewFactory() exporter.Factory
• type Config
  • func (config *Config) Validate() error

Functions

func NewFactory

func NewFactory() exporter.Factory

Types

type Config

type Config struct {
	configretry.BackOffConfig `mapstructure:"retry_on_failure"`

	// LogGroupName is the name of CloudWatch log group which defines group of log streams
	// that share the same retention, monitoring, and access control settings.
	LogGroupName string `mapstructure:"log_group_name"`

	// LogStreamName is the name of CloudWatch log stream which is a sequence of log events
	// that share the same source.
	LogStreamName string `mapstructure:"log_stream_name"`

	// Endpoint is the CloudWatch Logs service endpoint which the requests
	// are forwarded to. https://docs.aws.amazon.com/general/latest/gr/cwl_region.html
	// e.g. logs.us-east-1.amazonaws.com
	// Optional.
	Endpoint string `mapstructure:"endpoint"`

	// LogRetention is the option to set the log retention policy for the CloudWatch Log Group. Defaults to Never Expire if not specified or set to 0
	// Possible values are 1, 3, 5, 7, 14, 30, 60, 90, 120, 150, 180, 365, 400, 545, 731, 1827, 2192, 2557, 2922, 3288, or 3653
	LogRetention int32 `mapstructure:"log_retention"`

	// Tags is the option to set tags for the CloudWatch Log Group.  If specified, please add at least 1 and at most 50 tags.  Input is a string to string map like so: { 'key': 'value' }
	// Keys must be between 1-128 characters and follow the regex pattern: ^([\p{L}\p{Z}\p{N}_.:/=+\-@]+)$
	// Values must be between 1-256 characters and follow the regex pattern: ^([\p{L}\p{Z}\p{N}_.:/=+\-@]*)$
	Tags map[string]string `mapstructure:"tags"`

	// Queue settings frm the exporterhelper
	QueueSettings configoptional.Optional[exporterhelper.QueueBatchConfig] `mapstructure:"sending_queue"`

	awsutil.AWSSessionSettings `mapstructure:",squash"`

	// Export raw log string instead of log wrapper
	// Required for emf logs
	RawLog bool `mapstructure:"raw_log,omitempty"`
	// contains filtered or unexported fields
}

Config represent a configuration for the CloudWatch logs exporter.

func (*Config) Validate

func (config *Config) Validate() error

Validate config