encoding

README

Encoding extensions

The encoding extensions can be used by compatible receivers or exporters to encode or decode data into/from a specific format. This is useful when the data is being sent to/from a system that expects a specific format and doesn't support the OpenTelemetry protocol.

🚧 Under active development 🚧

Component Milestones

To help track what work needs to be done with this component, these are the currently active goals being worked towards.

Development

• Add encoding extensions support additionally to the existing ways of configuring encodings (where applicable) to the following components:
  • file receiver
  • file exporter
  • kafka receiver
  • kafka exporter
  • kinesis exporter
  • pulsar receiver
  • pulsar exporter
• Add encoding extensions for open source formats, ie: otlp, zipkin, jaeger
• Deprecate the previously available ways of configuring encodings (where applicable).
• Remove the previously available ways of configuring encodings in favour of using the encoding extension.

Example configuration

extensions:
  zipkin_encoding:
    protocol: zipkin_proto
    version: v2

receivers:
  kafka:
    traces:
      encoding: zipkin_encoding
    # ... other configuration values

Documentation

Index

• type DecoderOption
  • func WithFlushBytes(b int64) DecoderOption
  • func WithFlushItems(i int64) DecoderOption
  • func WithOffset(offset int64) DecoderOption
• type DecoderOptions
  • func NewDecoderOptions(opts ...DecoderOption) DecoderOptions
• type LogsDecoder
• type LogsDecoderExtension
• type LogsDecoderFactory
• type LogsMarshalerExtension
• type LogsUnmarshalerExtension
• type MetricsDecoder
• type MetricsDecoderExtension
• type MetricsDecoderFactory
• type MetricsMarshalerExtension
• type MetricsUnmarshalerExtension
• type ProfilesMarshalerExtension
• type ProfilesUnmarshalerExtension
• type TracesMarshalerExtension
• type TracesUnmarshalerExtension

Types

type DecoderOption

type DecoderOption func(*DecoderOptions)

DecoderOption defines the functional option for DecoderOptions.

func WithFlushBytes

func WithFlushBytes(b int64) DecoderOption

WithFlushBytes sets the number of bytes after stream decoder should flush. Use WithFlushBytes(0) to disable flushing by byte count.

func WithFlushItems

func WithFlushItems(i int64) DecoderOption

WithFlushItems sets the number of items after stream decoder should flush. Use WithFlushItems(0) to disable flushing by item count.

func WithOffset

func WithOffset(offset int64) DecoderOption

WithOffset defines the initial stream offset for the stream. The exact meaning of the offset may vary by decoder (e.g. bytes, lines, records).

type DecoderOptions

type DecoderOptions struct {
	FlushBytes int64
	FlushItems int64
	Offset     int64
}

DecoderOptions configures the behavior of stream decoding. FlushBytes and FlushItems control how often the decoder should flush decoded data from the stream. Offset defines the initial stream offset for the stream. Use NewDecoderOptions to construct with default options.

func NewDecoderOptions

func NewDecoderOptions(opts ...DecoderOption) DecoderOptions

type LogsDecoder

type LogsDecoder interface {
	// DecodeLogs is expected to be called iteratively to read all derived plog.Logs batches from the stream.
	// The last batch of logs should be returned with a nil error. io.EOF error should follow on the subsequent call.
	DecodeLogs() (plog.Logs, error)
	// Offset returns the offset after the most recent batch read from the stream, or the initial offset.
	// The exact meaning of the offset may vary by decoder (e.g. bytes, lines, records).
	// You may use this value with WithOffset option to resume reading from the same offset when retrying after a failure.
	Offset() int64
}

LogsDecoder unmarshals logs from a stream, returning one batch per DecodeLogs call.

type LogsDecoderExtension

type LogsDecoderExtension interface {
	extension.Extension
	LogsDecoderFactory
}

LogsDecoderExtension is an extension that unmarshals logs from a stream.

type LogsDecoderFactory

type LogsDecoderFactory interface {
	NewLogsDecoder(reader io.Reader, options ...DecoderOption) (LogsDecoder, error)
}

LogsDecoderFactory creates LogsDecoder instances for streaming log deserialization.

type LogsMarshalerExtension

type LogsMarshalerExtension interface {
	extension.Extension
	plog.Marshaler
}

LogsMarshalerExtension is an extension that marshals logs.

type LogsUnmarshalerExtension

type LogsUnmarshalerExtension interface {
	extension.Extension
	plog.Unmarshaler
}

LogsUnmarshalerExtension is an extension that unmarshals logs.

type MetricsDecoder

type MetricsDecoder interface {
	// DecodeMetrics is expected to be called iteratively to read all derived pmetric.Metrics batches from the stream.
	// The last batch of metrics should be returned with a nil error. io.EOF error should follow on the subsequent call.
	DecodeMetrics() (pmetric.Metrics, error)
	// Offset returns the offset after the most recent batch read from the stream, or the initial offset.
	// The exact meaning of the offset may vary by decoder (e.g. bytes, lines, records).
	// You may use this value with WithOffset option to resume reading from the same offset when retrying after a failure.
	Offset() int64
}

MetricsDecoder unmarshals metrics from a stream, returning one batch per DecodeMetrics call.

type MetricsDecoderExtension

type MetricsDecoderExtension interface {
	extension.Extension
	MetricsDecoderFactory
}

MetricsDecoderExtension is an extension that unmarshals metrics from a stream.

type MetricsDecoderFactory

type MetricsDecoderFactory interface {
	NewMetricsDecoder(reader io.Reader, options ...DecoderOption) (MetricsDecoder, error)
}

MetricsDecoderFactory creates MetricsDecoder instances for streaming metric deserialization.

type MetricsMarshalerExtension

type MetricsMarshalerExtension interface {
	extension.Extension
	pmetric.Marshaler
}

MetricsMarshalerExtension is an extension that marshals metrics.

type MetricsUnmarshalerExtension

type MetricsUnmarshalerExtension interface {
	extension.Extension
	pmetric.Unmarshaler
}

MetricsUnmarshalerExtension is an extension that unmarshals metrics.

type ProfilesMarshalerExtension

type ProfilesMarshalerExtension interface {
	extension.Extension
	pprofile.Marshaler
}

ProfilesMarshalerExtension is an extension that marshals profiles.

type ProfilesUnmarshalerExtension

type ProfilesUnmarshalerExtension interface {
	extension.Extension
	pprofile.Unmarshaler
}

ProfilesUnmarshalerExtension is an extension that unmarshals Profiles.

type TracesMarshalerExtension

type TracesMarshalerExtension interface {
	extension.Extension
	ptrace.Marshaler
}

TracesMarshalerExtension is an extension that marshals traces.

type TracesUnmarshalerExtension

type TracesUnmarshalerExtension interface {
	extension.Extension
	ptrace.Unmarshaler
}

TracesUnmarshalerExtension is an extension that unmarshals traces.