sumologicexporter

README

Sumo Logic Exporter

Status
Stability	beta: metrics, logs, traces
Distributions	contrib
Issues
Code coverage
Code Owners	@rnishtala-sumo, @pankaj101A, @jagan2221
Emeritus	@aboguszewski-sumo, @kasia-kujawa, @mat-rumian, @sumo-drosiek, @chan-tim-sumo, @echlebek, @amdprophet

Migration to new architecture

This exporter is undergoing major changes right now.

We are in the process of moving the Sumo Logic exporter into this repository.

The following options are no longer supported:

• metric_format: {carbon2, graphite}
• metadata_attributes: [<regex>]
• graphite_template: <template>
• source_category: <template>
• source_name: <template>
• source_host: <template>

After the new exporter will be moved to this repository:

• carbon2 and graphite are no longer supported and prometheus or otlp format should be used

• all resource level attributes are treated as metadata_attributes so this option is no longer supported. You can use Group by Attributes processor to move attributes from record level to resource level. For example:

  # before switch to new collector
  exporters:
    sumologic:
      metadata_attribute:
        - my_attribute
  # after switch to new collector
  processors:
    groupbyattrs:
      keys:
        - my_attribute
  

• Source templates (source_category, source_name and source_host) are no longer supported. We recommend to use Transform Processor. For example:

  # before switch to new collector
  exporters:
    sumologic:
      source_category: "%{foo}/constant/%{bar}"
  # after switch to new collector
  processors:
    transformprocessor:
      log_statements:
        context: log
        statements:
          # set default value to unknown
          - set(attributes["foo"], "unknown") where attributes["foo"] == nil
          - set(attributes["bar"], "unknown") where attributes["foo"] == nil
          # set _sourceCategory as "%{foo}/constant/%{bar}"
          - set(resource.attributes["_sourceCategory"], Concat([attributes["foo"], "/constant/", attributes["bar"]], ""))
  

Configuration

This exporter supports sending logs and metrics data to Sumo Logic. Traces are exported using the native otlp_http exporter.

Configuration is specified via the yaml in the following structure:

exporters:
  # ...
  sumologic:
    # unique URL generated for your HTTP Source, this is the address to send data to
    endpoint: <HTTP_Source_URL>
    # Compression encoding format, empty string means no compression, default = gzip
    compression: {gzip, deflate, zstd, ""}
    # max HTTP request body size in bytes before compression (if applied),
    # default = 1_048_576 (1MB)
    max_request_body_size: <max_request_body_size>

    # format to use when sending logs to Sumo Logic, default = otlp,
    # see Sumo Logic documentation for details regarding log formats:
    # https://help.sumologic.com/docs/send-data/opentelemetry-collector/data-source-configurations/mapping-records-resources/
    log_format: {otlp, json, text}

    # format to use when sending metrics to Sumo Logic, default = otlp,
    # NOTE: only `otlp` is supported when used with sumologicextension
    metric_format: {otlp, prometheus}

    # Decompose OTLP Histograms into individual metrics, similar to how they're represented in Prometheus format.
    # The Sumo OTLP source currently doesn't support Histograms, and they are quietly dropped. This option produces
    # metrics similar to when metric_format is set to prometheus.
    # default = false
    decompose_otlp_histograms: {true, false}

    # Decompose OTLP Summaries into individual metrics (quantiles as gauges, count/sum as counters).
    # The Sumo OTLP source doesn't properly support Summary metrics with quantile dimensions. This option produces
    # metrics similar to when metric_format is set to prometheus, extracting quantile values as separate gauge
    # datapoints with a 'quantile' attribute, and count/sum as cumulative counter metrics.
    # default = false
    decompose_otlp_summaries: {true, false}

    # timeout is the timeout for every attempt to send data to the backend,
    # maximum connection timeout is 55s, default = 30s
    timeout: <timeout>

    # defines if sticky session support is enable.
    # default=false
    sticky_session_enabled: {true, false}

    # for below described queueing and retry related configuration please refer to:
    # https://github.com/open-telemetry/opentelemetry-collector/blob/main/exporter/exporterhelper/README.md#configuration

    retry_on_failure:
      # default = true
      enabled: {true, false}
      # time to wait after the first failure before retrying;
      # ignored if enabled is false, default = 5s
      initial_interval: <initial_interval>
      # multiplier is the factor applied to the previous delay to calculate the next retry delay;
      # ignored if enabled is false, default = 1.2
      multiplier: <multiplier>
      # is the upper bound on backoff; ignored if enabled is false, default = 5m
      max_interval: <max_interval>
      # is the maximum amount of time spent trying to send a batch;
      # ignored if enabled is false, default = 1h
      max_elapsed_time: <max_elapsed_time>

    sending_queue:
      # default = false
      enabled: {true, false}
      # number of consumers that dequeue batches; ignored if enabled is false,
      # default = 10
      num_consumers: <num_consumers>
      # when set, enables persistence and uses the component specified as a storage extension for the persistent queue
      # make sure to configure and add a `file_storage` extension in `service.extensions`.
      # default = None
      storage: <storage_name>
      # maximum number of batches kept in memory before data;
      # ignored if enabled is false, default = 1000
      #
      # user should calculate this as num_seconds * requests_per_second where:
      # num_seconds is the number of seconds to buffer in case of a backend outage,
      # requests_per_second is the average number of requests per seconds.
      queue_size: <queue_size>
      # BatchConfig it configures how the requests are consumed from the queue and batch together during consumption.
      # To enable the batching with sumologic specific default setting add the key with empty value
      # batch: {}
      batch:
        # FlushTimeout sets the time after which a batch will be sent regardless of its size.
        # default = 1s
        flush_timeout: <>
        # Sizer determines the type of size measurement used by the batch.
        # If not configured, use the same configuration as the queue.
        # It accepts "requests", "items", or "bytes".
        # default = requests
        sizer: <>
        # MinSize defines the configuration for the minimum size of a batch.
        # default = 1024
        min_size: <>
        # MaxSize defines the configuration for the minimum size of a batch.
        # default = 2048
        max_size: <>

Source Templates

Source Templates are no longer supported. Please follow Migration to new architecture

Documentation

Index

• Constants
• func NewFactory() exporter.Factory
• type Config
  • func (cfg *Config) Validate() error
• type LogFormatType
• type MetricFormatType
• type PipelineType
• type TraceFormatType

Constants

const (
	// TextFormat represents log_format: text
	TextFormat LogFormatType = "text"
	// JSONFormat represents log_format: json
	JSONFormat LogFormatType = "json"
	// OTLPLogFormat represents log_format: otlp
	OTLPLogFormat LogFormatType = "otlp"
	// RemovedGraphiteFormat represents the no longer supported graphite metric format
	RemovedGraphiteFormat MetricFormatType = "graphite"
	// RemovedCarbon2Format represents the no longer supported carbon2 metric format
	RemovedCarbon2Format MetricFormatType = "carbon2"
	// PrometheusFormat represents metric_format: prometheus
	PrometheusFormat MetricFormatType = "prometheus"
	// OTLPMetricFormat represents metric_format: otlp
	OTLPMetricFormat MetricFormatType = "otlp"
	// OTLPTraceFormat represents trace_format: otlp
	OTLPTraceFormat TraceFormatType = "otlp"
	// NoCompression represents disabled compression
	NoCompression configcompression.Type = ""
	// MetricsPipeline represents metrics pipeline
	MetricsPipeline PipelineType = "metrics"
	// LogsPipeline represents metrics pipeline
	LogsPipeline PipelineType = "logs"
	// TracesPipeline represents traces pipeline
	TracesPipeline PipelineType = "traces"

	// DefaultCompress defines default Compress
	DefaultCompress bool = true
	// DefaultCompressEncoding defines default CompressEncoding
	DefaultCompressEncoding configcompression.Type = "gzip"
	// DefaultMaxRequestBodySize defines default MaxRequestBodySize in bytes
	DefaultMaxRequestBodySize int = 1 * 1024 * 1024
	// DefaultLogFormat defines default LogFormat
	DefaultLogFormat LogFormatType = OTLPLogFormat
	// DefaultMetricFormat defines default MetricFormat
	DefaultMetricFormat MetricFormatType = OTLPMetricFormat
	// DefaultClient defines default Client
	DefaultClient string = "otelcol"
	// DefaultLogKey defines default LogKey value
	DefaultLogKey string = "log"
	// DefaultDropRoutingAttribute defines default DropRoutingAttribute
	DefaultDropRoutingAttribute string = ""
	// DefaultStickySessionEnabled defines default StickySessionEnabled value
	DefaultStickySessionEnabled bool = false
	// DefaultRetryOnFailureMultiplier defines default retry_on_failure multiplier value
	DefaultRetryOnFailureMultiplier float64 = 1.2
	// DefaultRetryOnFailureMaxInterval defines default retry_on_failure max_interval value
	DefaultRetryOnFailureMaxInterval time.Duration = 5 * time.Minute
	// DefaultRetryOnFailureMaxElapsedTime defines default retry_on_failure max_elapsed_time value
	DefaultRetryOnFailureMaxElapsedTime time.Duration = 1 * time.Hour
)

Functions

func NewFactory

func NewFactory() exporter.Factory

NewFactory returns a new factory for the sumologic exporter.

Types

type Config

type Config struct {
	confighttp.ClientConfig   `mapstructure:",squash"`                                 // squash ensures fields are correctly decoded in embedded struct.
	QueueSettings             configoptional.Optional[exporterhelper.QueueBatchConfig] `mapstructure:"sending_queue"`
	configretry.BackOffConfig `mapstructure:"retry_on_failure"`

	// Compression encoding format, either empty string, gzip or deflate (default gzip)
	// Empty string means no compression
	// NOTE: CompressEncoding is deprecated and will be removed in an upcoming release
	CompressEncoding *configcompression.Type `mapstructure:"compress_encoding"`

	// Max HTTP request body size in bytes before compression (if applied).
	// By default 1MB is recommended.
	MaxRequestBodySize int `mapstructure:"max_request_body_size"`

	// Logs related configuration
	// Format to post logs into Sumo. (default json)
	//   * text - Logs will appear in Sumo Logic in text format.
	//   * json - Logs will appear in Sumo Logic in json format.
	//   * otlp - Logs will be send in otlp format and will appear in Sumo Logic:
	//     * in json format if record level attributes exists
	//     * in text format in case of no level attributes
	// See Sumo Logic documentation for more details:
	// https://help.sumologic.com/docs/send-data/opentelemetry-collector/data-source-configurations/mapping-records-resources/
	LogFormat LogFormatType `mapstructure:"log_format"`

	// Metrics related configuration
	// The format of metrics you will be sending, either otlp or prometheus (Default is otlp)
	MetricFormat MetricFormatType `mapstructure:"metric_format"`

	// Decompose OTLP Histograms into individual metrics, similar to how they're represented in Prometheus format
	DecomposeOtlpHistograms bool `mapstructure:"decompose_otlp_histograms"`

	// Decompose OTLP Summaries into individual metrics (quantiles as gauges, count/sum as counters)
	DecomposeOtlpSummaries bool `mapstructure:"decompose_otlp_summaries"`

	// Sumo specific options
	// Name of the client
	Client string `mapstructure:"client"`

	// StickySessionEnabled defines if sticky session support is enable.
	// By default this is false.
	StickySessionEnabled bool `mapstructure:"sticky_session_enabled"`
}

Config defines configuration for Sumo Logic exporter.

func (*Config) Validate

func (cfg *Config) Validate() error

type LogFormatType

type LogFormatType string

LogFormatType represents log_format

type MetricFormatType

type MetricFormatType string

MetricFormatType represents metric_format

type PipelineType

type PipelineType string

PipelineType represents type of the pipeline

type TraceFormatType

type TraceFormatType string

TraceFormatType represents trace_format