fluentforwardexporter

README

Fluent Forward Exporter

Status
Stability	development: logs
Distributions	contrib
Issues
Code Owners	@r0mdau

Forward is the protocol used by Fluentd to route message between peers.

• Protocol specification: Forward protocol specification v1
• Library used IBM/fluent-forward-go (MIT License)

Getting Started

Settings

Property	Default value	Type	Description
endpoint.tcp_addr		string	MANDATORY Target URL to send Forward log streams to
endpoint.validate_tcp_resolution	false	bool	Controls whether to validate the tcp address and fail at startup.
connection_timeout	30s	time.Duration	Maximum amount of time a dial will wait for a connect to complete
tls.insecure	true	bool	If set to true, the connection is not secured with TLS.
tls.insecure_skip_verify	false	bool	Controls whether the exporter verifies the server's certificate chain and host name. If true, any certificate is accepted and any host name. This mode is susceptible to man-in-the-middle attacks
tls.ca_file	""	string	Used for mTLS. Path to the CA cert. For a client this verifies the server certificate
tls.cert_file	""	string	Used for mTLS. Path to the client TLS cert to use
tls.key_file	""	string	Used for mTLS. Path to the client TLS key to use
shared_key	""	string	A key string known by the server, used for authorization
require_ack	false	bool	Protocol delivery acknowledgment for log streams : true = at-least-once, false = at-most-once
tag	"tag"	string	Fluentd tag is a string separated by '.'s (e.g. myapp.access), and is used as the directions for Fluentd's internal routing engine
compress_gzip	false	bool	Transparent data compression. You can use this feature to reduce the transferred payload size
default_labels_enabled	true	map[string]bool	If omitted then default labels will be added. If one of the labels is omitted then this label will be added

See the default values in the method createDefaultConfig() in factory.go file.

Example, for default_labels_enabled that will add only the timestamp attribute in the log record:

exporters:
  fluentforward:
    endpoint:
      tcp_addr: a.new.fluentforward.target:24224
    connection_timeout: 10s
    require_ack: true
    tag: nginx
    compress_gzip: true
    default_labels_enabled:
      timestamp: true
      level: false
      message: false

But a best practice is to have at least timestamp, level and message in the exported log record to a Fluent endpoint.

Example with TLS enabled and shared key:

exporters:
  fluentforward:
    endpoint:
      tcp_addr: a.new.fluentforward.target:24224
    connection_timeout: 10s
    tls:
      insecure: false
    shared_key: otelcol-dev

Example with mutual TLS authentication (mTLS):

exporters:
  fluentforward:
    endpoint:
      tcp_addr: a.new.fluentforward.target:24224
    connection_timeout: 10s
    tls:
      insecure: false
      ca_file: ca.crt.pem
      cert_file: client.crt.pem
      key_file: client.key.pem

Severity

OpenTelemetry uses record.severity to track log levels.

Advanced Configuration

Queued retry capabilities are enabled by default, see the Exporter Helper queuing and retry settings to fine tune them.

Example usage:

exporters:
  fluentforward:
    endpoint:
      tcp_addr: a.new.fluentforward.target:24224
    connection_timeout: 10s
    retry_on_failure:
      enabled: true
      initial_interval: 5s
      max_interval: 30s
      max_elapsed_time: 5m
    sending_queue:
      enabled: true
      num_consumers: 10
      queue_size: 2000

Documentation

Overview

Package fluentforwardexporter exports log events using the Fluent Forward protocol

Index

• func NewFactory() exporter.Factory
• type Config
  • func (config *Config) Validate() error
• type Endpoint
• type TCPClientSettings

Functions

func NewFactory

func NewFactory() exporter.Factory

NewFactory creates a factory for the fluentforward exporter.

Types

type Config

type Config struct {
	TCPClientSettings `mapstructure:",squash"` // squash ensures fields are correctly decoded in embedded struct.

	// RequireAck enables the acknowledgement feature.
	RequireAck bool `mapstructure:"require_ack"`

	// The Fluent tag parameter used for routing
	Tag string `mapstructure:"tag"`

	// CompressGzip enables gzip compression for the payload.
	CompressGzip bool `mapstructure:"compress_gzip"`

	// DefaultLabelsEnabled is a map of default attributes to be added to each log record.
	DefaultLabelsEnabled map[string]bool `mapstructure:"default_labels_enabled"`

	QueueBatchConfig          configoptional.Optional[exporterhelper.QueueBatchConfig] `mapstructure:"sending_queue"`
	configretry.BackOffConfig `mapstructure:"retry_on_failure"`
}

Config defines configuration for fluentforward exporter.

func (*Config) Validate

func (config *Config) Validate() error

Validate checks if the configuration is valid

type Endpoint

type Endpoint struct {
	// TCPAddr is the address of the server to connect to.
	TCPAddr string `mapstructure:"tcp_addr"`
	// Controls whether to validate the tcp address.
	ValidateTCPResolution bool `mapstructure:"validate_tcp_resolution"`
}

Endpoint defines the address of the server to connect to.

type TCPClientSettings

type TCPClientSettings struct {
	// Endpoint to send logs to.
	Endpoint `mapstructure:"endpoint"`

	// Connection Timeout parameter configures `net.Dialer`.
	ConnectionTimeout time.Duration `mapstructure:"connection_timeout"`

	// ClientConfig struct exposes TLS client configuration.
	ClientConfig configtls.ClientConfig `mapstructure:"tls"`

	// SharedKey is used for authorization with the server that knows it.
	SharedKey string `mapstructure:"shared_key"`
}

TCPClientSettings defines common settings for a TCP client.