helper

package
v0.156.0 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Jul 7, 2026 License: Apache-2.0 Imports: 24 Imported by: 21

Documentation

Index

Constants

View Source 
const (
	// HTTP2xx is a special key that is represents a range from 200 to 299. Literal value is "2xx"
	HTTP2xx = "2xx"

	// HTTP3xx is a special key that is represents a range from 300 to 399. Literal value is "3xx"
	HTTP3xx = "3xx"

	// HTTP4xx is a special key that is represents a range from 400 to 499. Literal value is "4xx"
	HTTP4xx = "4xx"

	// HTTP5xx is a special key that is represents a range from 500 to 599. Literal value is "5xx"
	HTTP5xx = "5xx"
)
View Source 
const DropOnError = "drop"

DropOnError specifies an on_error mode for dropping entries after an error.

View Source 
const DropOnErrorQuiet = "drop_quiet"

DropOnErrorQuiet specifies an on_error mode for dropping entries after an error but without logging on error level

View Source 
const EpochKey = "epoch"

EpochKey is literally "epoch" and can parse seconds and/or subseconds

View Source 
const GotimeKey = "gotime"

GotimeKey is literally "gotime" and uses Golang's native time.Parse

View Source 
const NativeKey = "native" // provided for operator development

NativeKey is literally "native" and refers to Golang's native time.Time

View Source 
const SendOnError = "send"

SendOnError specifies an on_error mode for sending entries after an error.

View Source 
const SendOnErrorQuiet = "send_quiet"

SendOnErrorQuiet specifies an on_error mode for sending entries after an error but without logging on error level

View Source 
const StrptimeKey = "strptime"

StrptimeKey is literally "strptime", and is the default layout type

Variables

View Source 
var ErrEntryHandled = errors.New("entry handled by parser in quiet mode")

ErrEntryHandled signals that ParseWith already handled the entry (logged and optionally written downstream) and the caller must not write it again or propagate an error.

Functions

func ExprCompile added in v0.85.0 

func ExprCompile(input string) (*vm.Program, error)

func ExprCompileBool added in v0.83.0 

func ExprCompileBool(input string) (*vm.Program, error)

func GetExprEnv 

func GetExprEnv(e *entry.Entry) map[string]any

GetExprEnv returns a map of key/value pairs that can be be used to evaluate an expression

func MatchValues added in v0.101.0 

func MatchValues(value string, regexp *regexp.Regexp) (map[string]any, error)

func PutExprEnv 

func PutExprEnv(e map[string]any)

PutExprEnv adds a key/value pair that will can be used to evaluate an expression

Types

type Attributer 

type Attributer struct {
	// contains filtered or unexported fields
}

Attributer is a helper that adds attributes to an entry

func (*Attributer) Attribute 

func (l *Attributer) Attribute(e *entry.Entry) error

Attribute will add attributes to an entry

type AttributerConfig 

type AttributerConfig struct {
	Attributes map[string]ExprStringConfig `mapstructure:"attributes"`
}

AttributerConfig is the configuration of a attributer

func NewAttributerConfig 

func NewAttributerConfig() AttributerConfig

NewAttributerConfig creates a new attributer config with default values

func (AttributerConfig) Build 

func (c AttributerConfig) Build() (Attributer, error)

Build will build a attributer from the supplied configuration

type BasicConfig 

type BasicConfig struct {
	OperatorID   string `mapstructure:"id"`
	OperatorType string `mapstructure:"type"`
}

BasicConfig provides a basic implemention for an operator config.

func NewBasicConfig 

func NewBasicConfig(operatorID, operatorType string) BasicConfig

NewBasicConfig creates a new basic config

func (BasicConfig) Build 

func (c BasicConfig) Build(set component.TelemetrySettings) (BasicOperator, error)

Build will build a basic operator.

func (BasicConfig) ID 

func (c BasicConfig) ID() string

ID will return the operator id.

func (*BasicConfig) SetID 

func (c *BasicConfig) SetID(id string)

SetID will Update the operator id.

func (BasicConfig) Type 

func (c BasicConfig) Type() string

Type will return the operator type.

type BasicOperator 

type BasicOperator struct {
	OperatorID   string
	OperatorType string
	// contains filtered or unexported fields
}

BasicOperator provides a basic implementation of an operator.

func (*BasicOperator) ID 

func (p *BasicOperator) ID() string

ID will return the operator id.

func (*BasicOperator) Logger 

func (p *BasicOperator) Logger() *zap.Logger

Logger returns the operator's scoped logger.

func (*BasicOperator) Start 

func (*BasicOperator) Start(operator.Persister) error

Start will start the operator.

func (*BasicOperator) Stop 

func (*BasicOperator) Stop() error

Stop will stop the operator.

func (*BasicOperator) Type 

func (p *BasicOperator) Type() string

Type will return the operator type.

type BatchingLogEmitter added in v0.122.0 

type BatchingLogEmitter struct {
	OutputOperator
	// contains filtered or unexported fields
}

BatchingLogEmitter is a stanza operator that emits log entries to the consumer callback function `consumerFunc` with batching

func NewBatchingLogEmitter added in v0.122.0 

func NewBatchingLogEmitter(set component.TelemetrySettings, consumerFunc func(context.Context, []*entry.Entry), opts ...EmitterOption) *BatchingLogEmitter

NewBatchingLogEmitter creates a new receiver output

func (*BatchingLogEmitter) Process added in v0.122.0 

func (e *BatchingLogEmitter) Process(ctx context.Context, ent *entry.Entry) error

Process will emit an entry to the consumerFunc

func (*BatchingLogEmitter) ProcessBatch added in v0.122.0 

func (e *BatchingLogEmitter) ProcessBatch(ctx context.Context, entries []*entry.Entry) error

ProcessBatch emits the entries to the consumerFunc

func (*BatchingLogEmitter) Start added in v0.122.0 

func (e *BatchingLogEmitter) Start(_ operator.Persister) error

Start starts the goroutine(s) required for this operator

func (*BatchingLogEmitter) Stop added in v0.122.0 

func (e *BatchingLogEmitter) Stop() error

Stop will close the log channel and stop running goroutines

type ByteSize 

type ByteSize int64

func (*ByteSize) UnmarshalText 

func (h *ByteSize) UnmarshalText(text []byte) (err error)

type EmitterOption added in v0.100.0 

type EmitterOption interface {
	// contains filtered or unexported methods
}

func WithFlushInterval added in v0.100.0 

func WithFlushInterval(flushInterval time.Duration) EmitterOption

func WithMaxBatchSize added in v0.100.0 

func WithMaxBatchSize(maxBatchSize uint) EmitterOption

type ExprString 

type ExprString struct {
	SubStrings []string
	SubExprs   []*vm.Program
}

An ExprString is made up of a list of string literals interleaved with expressions. len(SubStrings) == len(SubExprs) + 1

func (*ExprString) Render 

func (e *ExprString) Render(env map[string]any) (string, error)

Render will render an ExprString as a string

type ExprStringConfig 

type ExprStringConfig string

ExprStringConfig is a string that represents an expression

func (ExprStringConfig) Build 

func (e ExprStringConfig) Build() (*ExprString, error)

Build creates an ExprStr string from the specified config

type IPResolver 

type IPResolver struct {
	// contains filtered or unexported fields
}

func NewIPResolver added in v0.53.0 

func NewIPResolver() *IPResolver

Create new resolver

func (*IPResolver) GetHostFromIP added in v0.53.0 

func (r *IPResolver) GetHostFromIP(ip string) (host string)

GetHostFromIp returns hostname for given ip It is taken from cache if exists, otherwise lookup is performed and result is put into cache

func (*IPResolver) Stop 

func (r *IPResolver) Stop()

Stop cache invalidation

type Identifier 

type Identifier struct {
	// contains filtered or unexported fields
}

Identifier is a helper that adds values to the resource of an entry

func (*Identifier) Identify 

func (i *Identifier) Identify(e *entry.Entry) error

Identify will add values to the resource of an entry

type IdentifierConfig 

type IdentifierConfig struct {
	Resource map[string]ExprStringConfig `mapstructure:"resource"`
}

IdentifierConfig is the configuration of a resource identifier

func NewIdentifierConfig 

func NewIdentifierConfig() IdentifierConfig

NewIdentifierConfig creates a new identifier config with default values

func (IdentifierConfig) Build 

func (c IdentifierConfig) Build() (Identifier, error)

Build will build an identifier from the supplied configuration

type InputConfig 

type InputConfig struct {
	AttributerConfig `mapstructure:",squash"`
	IdentifierConfig `mapstructure:",squash"`
	WriterConfig     `mapstructure:",squash"`
}

InputConfig provides a basic implementation of an input operator config.

func NewInputConfig 

func NewInputConfig(operatorID, operatorType string) InputConfig

NewInputConfig creates a new input config with default values.

func (InputConfig) Build 

func (c InputConfig) Build(set component.TelemetrySettings) (InputOperator, error)

Build will build a base producer.

type InputOperator 

type InputOperator struct {
	Attributer
	Identifier
	WriterOperator
}

InputOperator provides a basic implementation of an input operator.

func (*InputOperator) CanProcess 

func (*InputOperator) CanProcess() bool

CanProcess will always return false for an input operator.

func (*InputOperator) NewEntry 

func (i *InputOperator) NewEntry(value any) (*entry.Entry, error)

NewEntry will create a new entry using the `attributes`, and `resource` configuration.

func (*InputOperator) Process 

func (i *InputOperator) Process(_ context.Context, _ *entry.Entry) error

Process will always return an error if called.

func (*InputOperator) ProcessBatch added in v0.121.0 

func (i *InputOperator) ProcessBatch(_ context.Context, _ []*entry.Entry) error

ProcessBatch will always return an error if called.

type LogEmitter added in v0.100.0 

type LogEmitter interface {
	operator.Operator
	Start(operator.Persister) error
	Stop() error
	ProcessBatch(context.Context, []*entry.Entry) error
	Process(context.Context, *entry.Entry) error
}

type OutputConfig 

type OutputConfig struct {
	BasicConfig `mapstructure:",squash"`
	// contains filtered or unexported fields
}

OutputConfig provides a basic implementation of an output operator config.

func NewOutputConfig 

func NewOutputConfig(operatorID, operatorType string) OutputConfig

NewOutputConfig creates a new output config

func (OutputConfig) Build 

func (c OutputConfig) Build(set component.TelemetrySettings) (OutputOperator, error)

Build will build an output operator.

type OutputOperator 

type OutputOperator struct {
	BasicOperator
	// contains filtered or unexported fields
}

OutputOperator provides a basic implementation of an output operator.

func (*OutputOperator) CanOutput 

func (*OutputOperator) CanOutput() bool

CanOutput will always return false for an output operator.

func (*OutputOperator) CanProcess 

func (*OutputOperator) CanProcess() bool

CanProcess will always return true for an output operator.

func (*OutputOperator) GetOutputIDs 

func (*OutputOperator) GetOutputIDs() []string

GetOutputIDs will always return an empty array for an output ID.

func (*OutputOperator) Outputs 

func (*OutputOperator) Outputs() []operator.Operator

Outputs will always return an empty array for an output operator.

func (*OutputOperator) SetOutputIDs 

func (*OutputOperator) SetOutputIDs([]string)

SetOutputIDs will return nothing and does nothing.

func (*OutputOperator) SetOutputs 

func (*OutputOperator) SetOutputs(_ []operator.Operator) error

SetOutputs will return an error if called.

type ParseFunction 

type ParseFunction = func(any) (any, error)

ParseFunction is function that parses a raw value.

type ParserConfig 

type ParserConfig struct {
	TransformerConfig `mapstructure:",squash"`
	ParseFrom         entry.Field         `mapstructure:"parse_from"`
	ParseTo           entry.RootableField `mapstructure:"parse_to"`
	BodyField         *entry.Field        `mapstructure:"body"`
	TimeParser        *TimeParser         `mapstructure:"timestamp,omitempty"`
	SeverityConfig    *SeverityConfig     `mapstructure:"severity,omitempty"`
	TraceParser       *TraceParser        `mapstructure:"trace,omitempty"`
	ScopeNameParser   *ScopeNameParser    `mapstructure:"scope_name,omitempty"`
}

ParserConfig provides the basic implementation of a parser config.

func NewParserConfig 

func NewParserConfig(operatorID, operatorType string) ParserConfig

NewParserConfig creates a new parser config with default values

func (ParserConfig) Build 

func (c ParserConfig) Build(set component.TelemetrySettings) (ParserOperator, error)

Build will build a parser operator.

type ParserOperator 

type ParserOperator struct {
	TransformerOperator
	ParseFrom       entry.Field
	ParseTo         entry.Field
	BodyField       *entry.Field
	TimeParser      *TimeParser
	SeverityParser  *SeverityParser
	TraceParser     *TraceParser
	ScopeNameParser *ScopeNameParser
}

ParserOperator provides a basic implementation of a parser operator.

func (*ParserOperator) ParseWith 

func (p *ParserOperator) ParseWith(ctx context.Context, entry *entry.Entry, parse ParseFunction, write WriteFunction) error

ParseWith will process an entry's field with a parser function. In quiet on_error modes any entry-level error is handled internally and ErrEntryHandled is returned so callers do not write or propagate again.

func (*ParserOperator) ProcessBatchWith added in v0.132.0 

func (p *ParserOperator) ProcessBatchWith(ctx context.Context, entries []*entry.Entry, parse ParseFunction) error

func (*ParserOperator) ProcessBatchWithCallback added in v0.132.0 

func (p *ParserOperator) ProcessBatchWithCallback(ctx context.Context, entries []*entry.Entry, parse ParseFunction, cb func(*entry.Entry) error) error

func (*ParserOperator) ProcessWith 

func (p *ParserOperator) ProcessWith(ctx context.Context, entry *entry.Entry, parse ParseFunction) error

ProcessWith will run ParseWith on the entry, then forward the entry on to the next operators.

func (*ParserOperator) ProcessWithCallback 

func (p *ParserOperator) ProcessWithCallback(ctx context.Context, entry *entry.Entry, parse ParseFunction, cb func(*entry.Entry) error) error

type ProcessFunction added in v0.121.0 

type ProcessFunction = func(context.Context, *entry.Entry) error

ProcessFunction is a function that processes an entry.

type ScopeNameParser 

type ScopeNameParser struct {
	ParseFrom entry.Field `mapstructure:"parse_from,omitempty"`
	// contains filtered or unexported fields
}

ScopeNameParser is a helper that parses severity onto an entry.

func NewScopeNameParser 

func NewScopeNameParser() ScopeNameParser

NewScopeNameParser creates a new scope parser with default values

func (*ScopeNameParser) Parse 

func (p *ScopeNameParser) Parse(ent *entry.Entry) error

Parse will parse severity from a field and attach it to the entry

type SeverityConfig added in v0.55.0 

type SeverityConfig struct {
	ParseFrom     *entry.Field   `mapstructure:"parse_from,omitempty"`
	Preset        string         `mapstructure:"preset,omitempty"`
	Mapping       map[string]any `mapstructure:"mapping,omitempty"`
	OverwriteText bool           `mapstructure:"overwrite_text,omitempty"`
}

SeverityConfig allows users to specify how to parse a severity from a field.

func NewSeverityConfig added in v0.55.0 

func NewSeverityConfig() SeverityConfig

NewSeverityConfig creates a new severity parser config

func (*SeverityConfig) Build added in v0.55.0 

func (c *SeverityConfig) Build(_ component.TelemetrySettings) (SeverityParser, error)

Build builds a SeverityParser from a SeverityConfig

type SeverityParser 

type SeverityParser struct {
	ParseFrom entry.Field
	Mapping   severityMap
	// contains filtered or unexported fields
}

SeverityParser is a helper that parses severity onto an entry.

func (*SeverityParser) Parse 

func (p *SeverityParser) Parse(ent *entry.Entry) error

Parse will parse severity from a field and attach it to the entry

type SpanIDConfig added in v0.53.0 

type SpanIDConfig struct {
	ParseFrom *entry.Field `mapstructure:"parse_from,omitempty"`
	// contains filtered or unexported fields
}

type SynchronousLogEmitter added in v0.122.0 

type SynchronousLogEmitter struct {
	OutputOperator
	// contains filtered or unexported fields
}

SynchronousLogEmitter is a stanza operator that emits log entries to the consumer callback function `consumerFunc` synchronously

func NewSynchronousLogEmitter added in v0.122.0 

func NewSynchronousLogEmitter(set component.TelemetrySettings, consumerFunc func(context.Context, []*entry.Entry)) *SynchronousLogEmitter

func (*SynchronousLogEmitter) Process added in v0.122.0 

func (e *SynchronousLogEmitter) Process(ctx context.Context, ent *entry.Entry) error

func (*SynchronousLogEmitter) ProcessBatch added in v0.122.0 

func (e *SynchronousLogEmitter) ProcessBatch(ctx context.Context, entries []*entry.Entry) error

func (*SynchronousLogEmitter) Start added in v0.122.0 

func (*SynchronousLogEmitter) Start(operator.Persister) error

func (*SynchronousLogEmitter) Stop added in v0.122.0 

func (*SynchronousLogEmitter) Stop() error

type TimeParser 

type TimeParser struct {
	ParseFrom         *entry.Field      `mapstructure:"parse_from"`
	Layout            string            `mapstructure:"layout"`
	LayoutType        string            `mapstructure:"layout_type"`
	Location          string            `mapstructure:"location"`
	TimeZoneLocations map[string]string `mapstructure:"time_zone_locations"` // optional: abbreviation → IANA location name
	// contains filtered or unexported fields
}

TimeParser is a helper that parses time onto an entry.

func NewTimeParser 

func NewTimeParser() TimeParser

NewTimeParser creates a new time parser with default values

func (*TimeParser) IsZero 

func (t *TimeParser) IsZero() bool

IsZero returns true if the TimeParser is not a valid config

func (*TimeParser) Parse 

func (t *TimeParser) Parse(entry *entry.Entry) error

Parse will parse time from a field and attach it to the entry

func (*TimeParser) Unmarshal added in v0.61.0 

func (t *TimeParser) Unmarshal(component *confmap.Conf) error

Unmarshal starting from default settings

func (*TimeParser) Validate 

func (t *TimeParser) Validate() error

Validate validates a TimeParser, and reconfigures it if necessary

type TraceFlagsConfig 

type TraceFlagsConfig struct {
	ParseFrom *entry.Field `mapstructure:"parse_from,omitempty"`
	// contains filtered or unexported fields
}

type TraceIDConfig added in v0.53.0 

type TraceIDConfig struct {
	ParseFrom *entry.Field `mapstructure:"parse_from,omitempty"`
	// contains filtered or unexported fields
}

type TraceParser 

type TraceParser struct {
	TraceID    *TraceIDConfig    `mapstructure:"trace_id,omitempty"`
	SpanID     *SpanIDConfig     `mapstructure:"span_id,omitempty"`
	TraceFlags *TraceFlagsConfig `mapstructure:"trace_flags,omitempty"`
	// contains filtered or unexported fields
}

TraceParser is a helper that parses trace spans (and flags) onto an entry.

func NewTraceParser 

func NewTraceParser() TraceParser

NewTraceParser creates a new trace parser with default values

func (*TraceParser) Parse 

func (t *TraceParser) Parse(entry *entry.Entry) error

Parse will parse a trace (trace_id, span_id and flags) from a field and attach it to the entry

func (*TraceParser) Validate 

func (t *TraceParser) Validate() error

Validate validates a TraceParser, and reconfigures it if necessary

type TransformFunction 

type TransformFunction = func(*entry.Entry) error

TransformFunction is function that transforms an entry.

type TransformerConfig 

type TransformerConfig struct {
	WriterConfig `mapstructure:",squash"`
	OnError      string `mapstructure:"on_error"`
	IfExpr       string `mapstructure:"if"`
}

TransformerConfig provides a basic implementation of a transformer config.

func NewTransformerConfig 

func NewTransformerConfig(operatorID, operatorType string) TransformerConfig

NewTransformerConfig creates a new transformer config with default values

func (TransformerConfig) Build 

func (c TransformerConfig) Build(set component.TelemetrySettings) (TransformerOperator, error)

Build will build a transformer operator.

type TransformerOperator 

type TransformerOperator struct {
	WriterOperator
	OnError string
	IfExpr  *vm.Program
}

TransformerOperator provides a basic implementation of a transformer operator.

func (*TransformerOperator) CanProcess 

func (*TransformerOperator) CanProcess() bool

CanProcess will always return true for a transformer operator.

func (*TransformerOperator) HandleEntryError 

func (t *TransformerOperator) HandleEntryError(ctx context.Context, entry *entry.Entry, err error) error

HandleEntryError handles an entry error using the on_error strategy. In quiet modes (drop_quiet, send_quiet) the processing error is swallowed, but a downstream write error from send_quiet is still returned so the pipeline can react to delivery failures.

func (*TransformerOperator) HandleEntryErrorWithWrite added in v0.132.0 

func (t *TransformerOperator) HandleEntryErrorWithWrite(ctx context.Context, entry *entry.Entry, err error, write WriteFunction) error

HandleEntryErrorWithWrite is like HandleEntryError but uses the supplied write function. In quiet modes the processing error is swallowed; a downstream write error from send_quiet is still returned.

func (*TransformerOperator) ProcessBatchWith added in v0.121.0 

func (*TransformerOperator) ProcessBatchWith(ctx context.Context, entries []*entry.Entry, process ProcessFunction) error

func (*TransformerOperator) ProcessBatchWithTransform added in v0.132.0 

func (t *TransformerOperator) ProcessBatchWithTransform(ctx context.Context, entries []*entry.Entry, transform TransformFunction) error

func (*TransformerOperator) ProcessWith 

func (t *TransformerOperator) ProcessWith(ctx context.Context, entry *entry.Entry, transform TransformFunction) error

ProcessWith will process an entry with a transform function.

func (*TransformerOperator) Skip 

func (t *TransformerOperator) Skip(_ context.Context, entry *entry.Entry) (bool, error)

type WriteFunction added in v0.132.0 

type WriteFunction = func(context.Context, *entry.Entry) error

type WriterConfig 

type WriterConfig struct {
	BasicConfig `mapstructure:",squash"`
	OutputIDs   []string `mapstructure:"output"`
}

WriterConfig is the configuration of a writer operator.

func NewWriterConfig 

func NewWriterConfig(operatorID, operatorType string) WriterConfig

NewWriterConfig creates a new writer config

func (WriterConfig) Build 

func (c WriterConfig) Build(set component.TelemetrySettings) (WriterOperator, error)

Build will build a writer operator from the config.

type WriterOperator 

type WriterOperator struct {
	BasicOperator
	OutputIDs       []string
	OutputOperators []operator.Operator
}

WriterOperator is an operator that can write to other operators.

func (*WriterOperator) CanOutput 

func (*WriterOperator) CanOutput() bool

CanOutput always returns true for a writer operator.

func (*WriterOperator) GetOutputIDs 

func (w *WriterOperator) GetOutputIDs() []string

GetOutputIDs returns the output IDs of the writer operator.

func (*WriterOperator) Outputs 

func (w *WriterOperator) Outputs() []operator.Operator

Outputs returns the outputs of the writer operator.

func (*WriterOperator) SetOutputIDs 

func (w *WriterOperator) SetOutputIDs(opIDs []string)

SetOutputIDs will set the outputs of the operator.

func (*WriterOperator) SetOutputs 

func (w *WriterOperator) SetOutputs(operators []operator.Operator) error

SetOutputs will set the outputs of the operator.

func (*WriterOperator) Write 

func (w *WriterOperator) Write(ctx context.Context, e *entry.Entry) error

Write will write an entry to the outputs of the operator.

func (*WriterOperator) WriteBatch added in v0.121.0 

func (w *WriterOperator) WriteBatch(ctx context.Context, entries []*entry.Entry) error

WriteBatch writes a batch of entries to the outputs of the operator. A batch is a collection of entries that are sent in one go.

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.