telegraf

README

Telegraf

Telegraf is an agent for collecting, processing, aggregating, and writing metrics. Based on a plugin system to enable developers in the community to easily add support for additional metric collection. There are four distinct types of plugins:

1. Input Plugins collect metrics from the system, services, or 3rd party APIs
2. Processor Plugins transform, decorate, and/or filter metrics
3. Aggregator Plugins create aggregate metrics (e.g. mean, min, max, quantiles, etc.)
4. Output Plugins write metrics to various destinations

New plugins are designed to be easy to contribute, pull requests are welcomed, and we work to incorporate as many pull requests as possible. Consider looking at the list of external plugins as well.

Minimum Requirements

Telegraf shares the same minimum requirements as Go:

• Linux kernel version 2.6.23 or later
• Windows 7 or later
• FreeBSD 11.2 or later
• MacOS 10.11 El Capitan or later

Obtaining Telegraf

View the changelog for the latest updates and changes by version.

Binary Downloads

Binary downloads are available from the InfluxData downloads page or from each GitHub Releases page.

Package Repository

InfluxData also provides a package repo that contains both DEB and RPM downloads.

For deb-based platforms (e.g. Ubuntu and Debian) run the following to add the repo key and setup a new sources.list entry:

# influxdata-archive_compat.key GPG fingerprint:
#     9D53 9D90 D332 8DC7 D6C8 D3B9 D8FF 8E1F 7DF8 B07E
wget -q https://repos.influxdata.com/influxdata-archive_compat.key
echo '393e8779c89ac8d958f81f942f9ad7fb82a25e133faddaf92e15b16e6ac9ce4c influxdata-archive_compat.key' | sha256sum -c && cat influxdata-archive_compat.key | gpg --dearmor | sudo tee /etc/apt/trusted.gpg.d/influxdata-archive_compat.gpg > /dev/null
echo 'deb [signed-by=/etc/apt/trusted.gpg.d/influxdata-archive_compat.gpg] https://repos.influxdata.com/debian stable main' | sudo tee /etc/apt/sources.list.d/influxdata.list
sudo apt-get update && sudo apt-get install telegraf

For RPM-based platforms (e.g. RHEL, CentOS) use the following to create a repo file and install telegraf:

# influxdata-archive_compat.key GPG fingerprint:
#     9D53 9D90 D332 8DC7 D6C8 D3B9 D8FF 8E1F 7DF8 B07E
cat <<EOF | sudo tee /etc/yum.repos.d/influxdata.repo
[influxdata]
name = InfluxData Repository - Stable
baseurl = https://repos.influxdata.com/stable/\$basearch/main
enabled = 1
gpgcheck = 1
gpgkey = https://repos.influxdata.com/influxdata-archive_compat.key
EOF
sudo yum install telegraf

Build From Source

Telegraf requires Go version 1.20 or newer and the Makefile requires GNU make.

On Windows, the makefile requires the use of a bash terminal to support all makefile targets. An easy option to get bash for windows is using the version that comes with git for windows.

1. Install Go

2. Clone the Telegraf repository:

   git clone https://github.com/influxdata/telegraf.git
   

3. Run make build from the source directory

   cd telegraf
   make build
   

Nightly Builds

Nightly builds are available, generated from the master branch.

3rd Party Builds

Builds for other platforms or package formats are provided by members of theTelegraf community. These packages are not built, tested, or supported by the Telegraf project or InfluxData. Please get in touch with the package author if support is needed:

• Ansible Role
• Chocolatey by ripclawffb
• Scoop
• Snap by Laurent Sesquès (sajoupa)
• Homebrew

Getting Started

See usage with:

telegraf --help

Generate a telegraf config file

telegraf config > telegraf.conf

Generate config with only cpu input & influxdb output plugins defined

telegraf config --section-filter agent:inputs:outputs --input-filter cpu --output-filter influxdb

Run a single telegraf collection, outputting metrics to stdout

telegraf --config telegraf.conf --test

Run telegraf with all plugins defined in config file

telegraf --config telegraf.conf

Run telegraf, enabling the cpu & memory input, and influxdb output plugins

telegraf --config telegraf.conf --input-filter cpu:mem --output-filter influxdb

Contribute to the Project

Telegraf is an MIT licensed open source project and we love our community. The fastest way to get something fixed is to open a PR. Check out our contributing guide if you're interested in helping out. Also, join us on our Community Slack or Community Page if you have questions or comments for our engineering teams.

If your completely new to Telegraf and InfluxDB, you can also enroll for free at InfluxDB university to take courses to learn more.

Documentation

Latest Release Documentation

For documentation on the latest development code see the documentation index.

• Input Plugins
• Output Plugins
• Processor Plugins
• Aggregator Plugins

Documentation

Index

• Variables
• type Accumulator
• type AggregatingOutput
• type Aggregator
• type DeliveryInfo
• type DeprecationInfo
• type Escalation
  • func (e Escalation) String() string
• type Field
• type Initializer
• type Input
• type Logger
• type Metric
• type Output
• type Parser
• type ParserFunc
• type ParserFuncPlugin
• type ParserPlugin
• type PluginDescriber
• type PluginWithID
• type Processor
• type ResolveFunc
• type SecretStore
• type ServiceInput
• type StatefulPlugin
• type StreamingProcessor
• type Tag
• type TemplateMetric
• type TrackingAccumulator
• type TrackingID
• type ValueType

Variables

var Debug bool

Types

type Accumulator

type Accumulator interface {
	// AddFields adds a metric to the accumulator with the given measurement
	// name, fields, and tags (and timestamp). If a timestamp is not provided,
	// then the accumulator sets it to "now".
	AddFields(measurement string,
		fields map[string]interface{},
		tags map[string]string,
		t ...time.Time)

	// AddGauge is the same as AddFields, but will add the metric as a "Gauge" type
	AddGauge(measurement string,
		fields map[string]interface{},
		tags map[string]string,
		t ...time.Time)

	// AddCounter is the same as AddFields, but will add the metric as a "Counter" type
	AddCounter(measurement string,
		fields map[string]interface{},
		tags map[string]string,
		t ...time.Time)

	// AddSummary is the same as AddFields, but will add the metric as a "Summary" type
	AddSummary(measurement string,
		fields map[string]interface{},
		tags map[string]string,
		t ...time.Time)

	// AddHistogram is the same as AddFields, but will add the metric as a "Histogram" type
	AddHistogram(measurement string,
		fields map[string]interface{},
		tags map[string]string,
		t ...time.Time)

	// AddMetric adds a metric to the accumulator.
	AddMetric(Metric)

	// SetPrecision sets the timestamp rounding precision. All metrics
	// added to the accumulator will have their timestamp rounded to the
	// nearest multiple of precision.
	SetPrecision(precision time.Duration)

	// Report an error.
	AddError(err error)

	// Upgrade to a TrackingAccumulator with space for maxTracked
	// metrics/batches.
	WithTracking(maxTracked int) TrackingAccumulator
}

Accumulator allows adding metrics to the processing flow.

type AggregatingOutput

type AggregatingOutput interface {
	Output

	// Add the metric to the aggregator
	Add(in Metric)
	// Push returns the aggregated metrics and is called every flush interval.
	Push() []Metric
	// Reset signals that the aggregator period is completed.
	Reset()
}

AggregatingOutput adds aggregating functionality to an Output. May be used if the Output only accepts a fixed set of aggregations over a time period. These functions may be called concurrently to the Write function.

type Aggregator

type Aggregator interface {
	PluginDescriber

	// Add the metric to the aggregator.
	Add(in Metric)

	// Push pushes the current aggregates to the accumulator.
	Push(acc Accumulator)

	// Reset resets the aggregators caches and aggregates.
	Reset()
}

Aggregator is an interface for implementing an Aggregator plugin. the RunningAggregator wraps this interface and guarantees that Add, Push, and Reset can not be called concurrently, so locking is not required when implementing an Aggregator plugin.

type DeliveryInfo

type DeliveryInfo interface {
	// ID is the TrackingID
	ID() TrackingID

	// Delivered returns true if the metric was processed successfully.
	Delivered() bool
}

DeliveryInfo provides the results of a delivered metric group.

type DeprecationInfo

type DeprecationInfo struct {
	// Since specifies the version since when the plugin is deprecated
	Since string
	// RemovalIn optionally specifies the version when the plugin is scheduled for removal
	RemovalIn string
	// Notice for the user on suggested replacements etc.
	Notice string
}

DeprecationInfo contains information for marking a plugin deprecated.

type Escalation

type Escalation int

Escalation level for the plugin or option

const (
	// None means no deprecation
	None Escalation = iota
	// Warn means deprecated but still within the grace period
	Warn
	// Error means deprecated and beyond grace period
	Error
)

func (Escalation) String

func (e Escalation) String() string

type Field

type Field struct {
	Key   string
	Value interface{}
}

Field represents a single field key and value.

type Initializer

type Initializer interface {
	// Init performs one time setup of the plugin and returns an error if the
	// configuration is invalid.
	Init() error
}

Initializer is an interface that all plugin types: Inputs, Outputs, Processors, and Aggregators can optionally implement to initialize the plugin.

type Input

type Input interface {
	PluginDescriber

	// Gather takes in an accumulator and adds the metrics that the Input
	// gathers. This is called every agent.interval
	Gather(Accumulator) error
}

type Logger

type Logger interface {
	// Errorf logs an error message, patterned after log.Printf.
	Errorf(format string, args ...interface{})
	// Error logs an error message, patterned after log.Print.
	Error(args ...interface{})
	// Debugf logs a debug message, patterned after log.Printf.
	Debugf(format string, args ...interface{})
	// Debug logs a debug message, patterned after log.Print.
	Debug(args ...interface{})
	// Warnf logs a warning message, patterned after log.Printf.
	Warnf(format string, args ...interface{})
	// Warn logs a warning message, patterned after log.Print.
	Warn(args ...interface{})
	// Infof logs an information message, patterned after log.Printf.
	Infof(format string, args ...interface{})
	// Info logs an information message, patterned after log.Print.
	Info(args ...interface{})
}

Logger defines an plugin-related interface for logging.

type Metric

type Metric interface {
	// Name is the primary identifier for the Metric and corresponds to the
	// measurement in the InfluxDB data model.
	Name() string

	// Tags returns the tags as a map.  This method is deprecated, use TagList instead.
	Tags() map[string]string

	// TagList returns the tags as a slice ordered by the tag key in lexical
	// bytewise ascending order.  The returned value should not be modified,
	// use the AddTag or RemoveTag methods instead.
	TagList() []*Tag

	// Fields returns the fields as a map.  This method is deprecated, use FieldList instead.
	Fields() map[string]interface{}

	// FieldList returns the fields as a slice in an undefined order.  The
	// returned value should not be modified, use the AddField or RemoveField
	// methods instead.
	FieldList() []*Field

	// Time returns the timestamp of the metric.
	Time() time.Time

	// Type returns a general type for the entire metric that describes how you
	// might interpret, aggregate the values. Used by prometheus and statsd.
	Type() ValueType

	// SetName sets the metric name.
	SetName(name string)

	// AddPrefix adds a string to the front of the metric name.  It is
	// equivalent to m.SetName(prefix + m.Name()).
	//
	// This method is deprecated, use SetName instead.
	AddPrefix(prefix string)

	// AddSuffix appends a string to the back of the metric name.  It is
	// equivalent to m.SetName(m.Name() + suffix).
	//
	// This method is deprecated, use SetName instead.
	AddSuffix(suffix string)

	// GetTag returns the value of a tag and a boolean to indicate if it was set.
	GetTag(key string) (string, bool)

	// HasTag returns true if the tag is set on the Metric.
	HasTag(key string) bool

	// AddTag sets the tag on the Metric.  If the Metric already has the tag
	// set then the current value is replaced.
	AddTag(key, value string)

	// RemoveTag removes the tag if it is set.
	RemoveTag(key string)

	// GetField returns the value of a field and a boolean to indicate if it was set.
	GetField(key string) (interface{}, bool)

	// HasField returns true if the field is set on the Metric.
	HasField(key string) bool

	// AddField sets the field on the Metric.  If the Metric already has the field
	// set then the current value is replaced.
	AddField(key string, value interface{})

	// RemoveField removes the tag if it is set.
	RemoveField(key string)

	// SetTime sets the timestamp of the Metric.
	SetTime(t time.Time)

	// HashID returns an unique identifier for the series.
	HashID() uint64

	// Copy returns a deep copy of the Metric.
	Copy() Metric

	// Accept marks the metric as processed successfully and written to an
	// output.
	Accept()

	// Reject marks the metric as processed unsuccessfully.
	Reject()

	// Drop marks the metric as processed successfully without being written
	// to any output.
	Drop()
}

Metric is the type of data that is processed by Telegraf. Input plugins, and to a lesser degree, Processor and Aggregator plugins create new Metrics and Output plugins write them.

type Output

type Output interface {
	PluginDescriber

	// Connect to the Output; connect is only called once when the plugin starts
	Connect() error
	// Close any connections to the Output. Close is called once when the output
	// is shutting down. Close will not be called until all writes have finished,
	// and Write() will not be called once Close() has been, so locking is not
	// necessary.
	Close() error
	// Write takes in group of points to be written to the Output
	Write(metrics []Metric) error
}

type Parser

type Parser interface {
	// Parse takes a byte buffer separated by newlines
	// ie, `cpu.usage.idle 90\ncpu.usage.busy 10`
	// and parses it into telegraf metrics
	//
	// Must be thread-safe.
	Parse(buf []byte) ([]Metric, error)

	// ParseLine takes a single string metric
	// ie, "cpu.usage.idle 90"
	// and parses it into a telegraf metric.
	//
	// Must be thread-safe.
	ParseLine(line string) (Metric, error)

	// SetDefaultTags tells the parser to add all of the given tags
	// to each parsed metric.
	// NOTE: do _not_ modify the map after you've passed it here!!
	SetDefaultTags(tags map[string]string)
}

Parser is an interface defining functions that a parser plugin must satisfy.

type ParserFunc

type ParserFunc func() (Parser, error)

type ParserFuncPlugin

type ParserFuncPlugin interface {
	// GetParser returns a new parser.
	SetParserFunc(fn ParserFunc)
}

ParserFuncPlugin is an interface for plugins that are able to parse arbitrary data formats.

type ParserPlugin

type ParserPlugin interface {
	// SetParser sets the parser function for the interface
	SetParser(parser Parser)
}

ParserPlugin is an interface for plugins that are able to parse arbitrary data formats.

type PluginDescriber

type PluginDescriber interface {
	// SampleConfig returns the default configuration of the Plugin
	SampleConfig() string
}

PluginDescriber contains the functions all plugins must implement to describe themselves to Telegraf. Note that all plugins may define a logger that is not part of the interface, but will receive an injected logger if it's set. eg: Log telegraf.Logger `toml:"-"`

type PluginWithID

type PluginWithID interface {
	// ID returns the ID of the plugin instance. This function has to be
	// callable directly after the plugin's Init() function if there is any!
	ID() string
}

PluginWithID allows a plugin to overwrite its identifier of the plugin instance by a user specified value. By default the ID is generated using the plugin's configuration.

type Processor

type Processor interface {
	PluginDescriber

	// Apply the filter to the given metric.
	Apply(in ...Metric) []Metric
}

Processor is a processor plugin interface for defining new inline processors. these are extremely efficient and should be used over StreamingProcessor if you do not need asynchronous metric writes.

type ResolveFunc

type ResolveFunc func() ([]byte, bool, error)

ResolveFunc is a function to resolve the secret. The returned flag indicates if the resolver is static (false), i.e. the secret will not change over time, or dynamic (true) to handle secrets that change over time (e.g. TOTP).

type SecretStore

type SecretStore interface {
	Initializer
	PluginDescriber

	// Get searches for the given key and return the secret
	Get(key string) ([]byte, error)

	// Set sets the given secret for the given key
	Set(key, value string) error

	// List lists all known secret keys
	List() ([]string, error)

	// GetResolver returns a function to resolve the given key.
	GetResolver(key string) (ResolveFunc, error)
}

SecretStore is an interface defining functions that a secret-store plugin must satisfy.

type ServiceInput

type ServiceInput interface {
	Input

	// Start the ServiceInput.  The Accumulator may be retained and used until
	// Stop returns.
	Start(Accumulator) error

	// Stop stops the services and closes any necessary channels and connections.
	// Metrics should not be written out to the accumulator once stop returns, so
	// Stop() should stop reading and wait for any in-flight metrics to write out
	// to the accumulator before returning.
	Stop()
}

type StatefulPlugin

type StatefulPlugin interface {
	// GetState returns the current state of the plugin to persist
	// The returned state can be of any time as long as it can be
	// serialized to JSON. The best choice is a structure defined in
	// your plugin.
	// Note: This function has to be callable directly after the
	// plugin's Init() function if there is any!
	GetState() interface{}

	// SetState is called by the Persister once after loading and
	// initialization (after Init() function).
	SetState(state interface{}) error
}

StatefulPlugin contains the functions that plugins must implement to persist an internal state across Telegraf runs. Note that plugins may define a persister that is not part of the interface, but can be used to trigger state updates by the plugin if it exists in the plugin struct, eg: Persister telegraf.StatePersister `toml:"-"`

type StreamingProcessor

type StreamingProcessor interface {
	PluginDescriber

	// Start is called once when the plugin starts; it is only called once per
	// plugin instance, and never in parallel.
	// Start should return once it is ready to receive metrics.
	// The passed in accumulator is the same as the one passed to Add(), so you
	// can choose to save it in the plugin, or use the one received from Add().
	Start(acc Accumulator) error

	// Add is called for each metric to be processed. The Add() function does not
	// need to wait for the metric to be processed before returning, and it may
	// be acceptable to let background goroutine(s) handle the processing if you
	// have slow processing you need to do in parallel.
	// Keep in mind Add() should not spawn unbounded goroutines, so you may need
	// to use a semaphore or pool of workers (eg: reverse_dns plugin does this)
	// Metrics you don't want to pass downstream should have metric.Drop() called,
	// rather than simply omitting the acc.AddMetric() call
	Add(metric Metric, acc Accumulator) error

	// Stop gives you an opportunity to gracefully shut down the processor.
	// Once Stop() is called, Add() will not be called any more. If you are using
	// goroutines, you should wait for any in-progress metrics to be processed
	// before returning from Stop().
	// When stop returns, you should no longer be writing metrics to the
	// accumulator.
	Stop()
}

StreamingProcessor is a processor that can take in a stream of messages

type Tag

type Tag struct {
	Key   string
	Value string
}

Tag represents a single tag key and value.

type TemplateMetric

type TemplateMetric interface {
	Name() string
	Tag(key string) string
	Field(key string) interface{}
}

TemplateMetric is an interface to use in templates (e.g text/template) to generate complex strings from metric properties e.g. '{{.Neasurement}}-{{.Tag "foo"}}-{{.Field "bar"}}'

type TrackingAccumulator

type TrackingAccumulator interface {
	Accumulator

	// Add the Metric and arrange for tracking feedback after processing.
	AddTrackingMetric(m Metric) TrackingID

	// Add a group of Metrics and arrange for a signal when the group has been
	// processed.
	AddTrackingMetricGroup(group []Metric) TrackingID

	// Delivered returns a channel that will contain the tracking results.
	Delivered() <-chan DeliveryInfo
}

TrackingAccumulator is an Accumulator that provides a signal when the metric has been fully processed. Sending more metrics than the accumulator has been allocated for without reading status from the Accepted or Rejected channels is an error.

type TrackingID

type TrackingID uint64

TrackingID uniquely identifies a tracked metric group

type ValueType

type ValueType int

ValueType is an enumeration of metric types that represent a simple value.

const (
	Counter ValueType
	Gauge
	Untyped
	Summary
	Histogram
)

Possible values for the ValueType enum.