opensearchexporter

README

OpenSearch Exporter

Status
Stability	development: logs
	alpha: traces
Distributions	contrib
Issues
Code Owners	@Aneurysm9, @MitchellGale, @MaxKsyunz, @YANG-DB

OpenSearch exporter supports sending OpenTelemetry signals as documents to OpenSearch.

The documents are sent using observability catalog schema.

Configuration options

Indexing Options

• dataset (default=default) a user-provided label to classify source of telemetry. It is used to construct the name of the destination index or data stream.
• namespace (default=namespace) a user-provided label to group telemetry. It is used to construct the name of the destination index or data stream.

HTTP Connection Options

OpenSearch export supports standard HTTP client settings.

• http.endpoint (required) <url>:<port> of OpenSearch node to send data to.

TLS settings

Supports standard TLS settings as part of HTTP settings. See TLS Configuration/Client Settings.

Retry Options

• retry_on_failure: See retry_on_failure

Timeout Options

• timeout : See timeout

Bulk Indexer Options

• bulk_action (optional): the action for ingesting data. Only create and index are allowed here.

Example

extensions:
  basicauth/client:
  client_auth:
    username: username
    password: password
    
exporters:
  opensearch/trace:
    http:
      endpoint: https://opensearch.example.com:9200
      auth:
        authenticator: basicauth/client
# ······
service:
  pipelines:
    traces:
      receivers: [otlp]
      exporters: [opensearch/trace]
      processors: [batch]

Documentation

Overview

Package opensearchexporter contains an opentelemetry-collector exporter for OpenSearch.

Index

• func NewFactory() exporter.Factory
• type Config
  • func (cfg *Config) Validate() error
• type MappingMode
  • func (m MappingMode) String() string
• type MappingsSettings

Functions

func NewFactory

func NewFactory() exporter.Factory

NewFactory creates a factory for OpenSearch exporter.

Types

type Config

type Config struct {
	confighttp.ClientConfig        `mapstructure:"http"`
	configretry.BackOffConfig      `mapstructure:"retry_on_failure"`
	exporterhelper.TimeoutSettings `mapstructure:",squash"`
	MappingsSettings               `mapstructure:"mapping"`

	// The Observability indices would follow the recommended for immutable data stream ingestion pattern using
	// the data_stream concepts. See https://opensearch.org/docs/latest/dashboards/im-dashboards/datastream/
	// Index pattern will follow the next naming template ss4o_{type}-{dataset}-{namespace}
	Dataset   string `mapstructure:"dataset"`
	Namespace string `mapstructure:"namespace"`

	// LogsIndex configures the index, index alias, or data stream name logs should be indexed in.
	// https://opensearch.org/docs/latest/im-plugin/index/
	// https://opensearch.org/docs/latest/dashboards/im-dashboards/datastream/
	LogsIndex string `mapstructure:"logs_index"`

	// BulkAction configures the action for ingesting data. Only `create` and `index` are allowed here.
	// If not specified, the default value `create` will be used.
	BulkAction string `mapstructure:"bulk_action"`
}

Config defines configuration for OpenSearch exporter.

func (*Config) Validate

func (cfg *Config) Validate() error

Validate validates the opensearch server configuration.

type MappingMode

type MappingMode int

const (
	MappingSS4O MappingMode = iota
	MappingECS
	MappingFlattenAttributes
)

Enum values for MappingMode.

func (MappingMode) String

func (m MappingMode) String() string

type MappingsSettings

type MappingsSettings struct {
	// Mode configures the field mappings.
	// Supported modes are the following:
	//
	//   ss4o: exports logs in the Simple Schema for Observability standard.
	//   This mode is enabled by default.
	//   See: https://opensearch.org/docs/latest/observing-your-data/ss4o/
	//
	//   ecs: maps fields defined in the OpenTelemetry Semantic Conventions
	//   to the Elastic Common Schema.
	//   See: https://www.elastic.co/guide/en/ecs/current/index.html
	//
	//   flatten_attributes: uses the ECS mapping but flattens all resource and
	//   log attributes in the record to the top-level.
	Mode string `mapstructure:"mode"`

	// Additional field mappings.
	Fields map[string]string `mapstructure:"fields"`

	// File to read additional fields mappings from.
	File string `mapstructure:"file"`

	// Field to store timestamp in.  If not set uses the default @timestamp
	TimestampField string `mapstructure:"timestamp_field"`

	// Whether to store timestamp in Epoch miliseconds
	UnixTimestamp bool `mapstructure:"unix_timestamp"`

	// Try to find and remove duplicate fields
	Dedup bool `mapstructure:"dedup"`

	Dedot bool `mapstructure:"dedot"`
}