telegraf

README

Telegraf

Telegraf is an agent written in Go for collecting, processing, aggregating, and writing metrics.

Design goals are to have a minimal memory footprint with a plugin system so that developers in the community can easily add support for collecting metrics from well known services (like Hadoop, Postgres, or Redis) and third party APIs (like Mailchimp, AWS CloudWatch, or Google Analytics).

Telegraf is plugin-driven and has the concept of 4 distinct plugins:

1. Input Plugins collect metrics from the system, services, or 3rd party APIs
2. Processor Plugins transform, decorate, and/or filter metrics
3. Aggregator Plugins create aggregate metrics (e.g. mean, min, max, quantiles, etc.)
4. Output Plugins write metrics to various destinations

For more information on Processor and Aggregator plugins please read this.

New plugins are designed to be easy to contribute, we'll eagerly accept pull requests and will manage the set of plugins that Telegraf supports. See the contributing guide for instructions on writing new plugins.

Installation:

You can either download the binaries directly from the downloads page.

A few alternate installs are available here as well:

FreeBSD tarball:

Latest:

• https://dl.influxdata.com/telegraf/releases/telegraf-VERSION_freebsd_amd64.tar.gz

Ansible Role:

Ansible role: https://github.com/rossmcdonald/telegraf

From Source:

Telegraf manages dependencies via gdm, which gets installed via the Makefile if you don't have it already. You also must build with golang version 1.8+.

1. Install Go
2. Setup your GOPATH
3. Run go get github.com/influxdata/telegraf
4. Run cd $GOPATH/src/github.com/influxdata/telegraf
5. Run make

How to use it:

See usage with:

telegraf --help

Generate a telegraf config file:

telegraf config > telegraf.conf

Generate config with only cpu input & influxdb output plugins defined

telegraf --input-filter cpu --output-filter influxdb config

Run a single telegraf collection, outputing metrics to stdout

telegraf --config telegraf.conf -test

Run telegraf with all plugins defined in config file

telegraf --config telegraf.conf

Run telegraf, enabling the cpu & memory input, and influxdb output plugins

telegraf --config telegraf.conf -input-filter cpu:mem -output-filter influxdb

Configuration

See the configuration guide for a rundown of the more advanced configuration options.

Input Plugins

• aerospike
• amqp_consumer (rabbitmq)
• apache
• aws cloudwatch
• bcache
• cassandra
• ceph
• cgroup
• chrony
• consul
• conntrack
• couchbase
• couchdb
• disque
• dns query time
• docker
• dovecot
• elasticsearch
• exec (generic executable plugin, support JSON, influx, graphite and nagios)
• filestat
• haproxy
• hddtemp
• http_response
• httpjson (generic JSON-emitting http service plugin)
• internal
• influxdb
• ipmi_sensor
• iptables
• jolokia
• kubernetes
• leofs
• lustre2
• mailchimp
• memcached
• mesos
• mongodb
• mysql
• net_response
• nginx
• nsq
• nstat
• ntpq
• phpfpm
• phusion passenger
• ping
• postgresql
• postgresql_extensible
• powerdns
• procstat
• prometheus
• puppetagent
• rabbitmq
• raindrops
• redis
• rethinkdb
• riak
• sensors
• snmp
• snmp_legacy
• sql server (microsoft)
• twemproxy
• varnish
• zfs
• zookeeper
• win_perf_counters (windows performance counters)
• sysstat
• system
  • cpu
  • mem
  • net
  • netstat
  • disk
  • diskio
  • swap
  • processes
  • kernel (/proc/stat)
  • kernel (/proc/vmstat)

Telegraf can also collect metrics via the following service plugins:

• http_listener
• kafka_consumer
• mqtt_consumer
• nats_consumer
• nsq_consumer
• logparser
• statsd
• socket_listener
• tail
• tcp_listener
• udp_listener
• webhooks
  • filestack
  • github
  • mandrill
  • rollbar

Processor Plugins

• printer

Aggregator Plugins

• minmax

Output Plugins

• influxdb
• amon
• amqp (rabbitmq)
• aws kinesis
• aws cloudwatch
• datadog
• discard
• file
• graphite
• graylog
• instrumental
• kafka
• librato
• mqtt
• nats
• nsq
• opentsdb
• prometheus
• riemann
• riemann_legacy
• socket_writer
• tcp
• udp

Contributing

Please see the contributing guide for details on contributing a plugin to Telegraf.

Documentation

Index

• type Accumulator
• type Aggregator
• type Input
• type Metric
• type Output
• type Processor
• type ServiceInput
• type ServiceOutput
• type ValueType

Types

type Accumulator

type Accumulator interface {
	// AddFields adds a metric to the accumulator with the given measurement
	// name, fields, and tags (and timestamp). If a timestamp is not provided,
	// then the accumulator sets it to "now".
	// Create a point with a value, decorating it with tags
	// NOTE: tags is expected to be owned by the caller, don't mutate
	// it after passing to Add.
	AddFields(measurement string,
		fields map[string]interface{},
		tags map[string]string,
		t ...time.Time)

	// AddGauge is the same as AddFields, but will add the metric as a "Gauge" type
	AddGauge(measurement string,
		fields map[string]interface{},
		tags map[string]string,
		t ...time.Time)

	// AddCounter is the same as AddFields, but will add the metric as a "Counter" type
	AddCounter(measurement string,
		fields map[string]interface{},
		tags map[string]string,
		t ...time.Time)

	SetPrecision(precision, interval time.Duration)

	AddError(err error)
}

Accumulator is an interface for "accumulating" metrics from plugin(s). The metrics are sent down a channel shared between all plugins.

type Aggregator

type Aggregator interface {
	// SampleConfig returns the default configuration of the Input.
	SampleConfig() string

	// Description returns a one-sentence description on the Input.
	Description() string

	// Add the metric to the aggregator.
	Add(in Metric)

	// Push pushes the current aggregates to the accumulator.
	Push(acc Accumulator)

	// Reset resets the aggregators caches and aggregates.
	Reset()
}

Aggregator is an interface for implementing an Aggregator plugin. the RunningAggregator wraps this interface and guarantees that Add, Push, and Reset can not be called concurrently, so locking is not required when implementing an Aggregator plugin.

type Input

type Input interface {
	// SampleConfig returns the default configuration of the Input
	SampleConfig() string

	// Description returns a one-sentence description on the Input
	Description() string

	// Gather takes in an accumulator and adds the metrics that the Input
	// gathers. This is called every "interval"
	Gather(Accumulator) error
}

type Metric

type Metric interface {
	// Serialize serializes the metric into a line-protocol byte buffer,
	// including a newline at the end.
	Serialize() []byte
	// same as Serialize, but avoids an allocation.
	// returns number of bytes copied into dst.
	SerializeTo(dst []byte) int
	// String is the same as Serialize, but returns a string.
	String() string
	// Copy deep-copies the metric.
	Copy() Metric
	// Split will attempt to return multiple metrics with the same timestamp
	// whose string representations are no longer than maxSize.
	// Metrics with a single field may exceed the requested size.
	Split(maxSize int) []Metric

	// Tag functions
	HasTag(key string) bool
	AddTag(key, value string)
	RemoveTag(key string)

	// Field functions
	HasField(key string) bool
	AddField(key string, value interface{})
	RemoveField(key string) error

	// Name functions
	SetName(name string)
	SetPrefix(prefix string)
	SetSuffix(suffix string)

	// Getting data structure functions
	Name() string
	Tags() map[string]string
	Fields() map[string]interface{}
	Time() time.Time
	UnixNano() int64
	Type() ValueType
	Len() int // returns the length of the serialized metric, including newline
	HashID() uint64

	// aggregator things:
	SetAggregate(bool)
	IsAggregate() bool
}

type Output

type Output interface {
	// Connect to the Output
	Connect() error
	// Close any connections to the Output
	Close() error
	// Description returns a one-sentence description on the Output
	Description() string
	// SampleConfig returns the default configuration of the Output
	SampleConfig() string
	// Write takes in group of points to be written to the Output
	Write(metrics []Metric) error
}

type Processor

type Processor interface {
	// SampleConfig returns the default configuration of the Input
	SampleConfig() string

	// Description returns a one-sentence description on the Input
	Description() string

	// Apply the filter to the given metric
	Apply(in ...Metric) []Metric
}

type ServiceInput

type ServiceInput interface {
	// SampleConfig returns the default configuration of the Input
	SampleConfig() string

	// Description returns a one-sentence description on the Input
	Description() string

	// Gather takes in an accumulator and adds the metrics that the Input
	// gathers. This is called every "interval"
	Gather(Accumulator) error

	// Start starts the ServiceInput's service, whatever that may be
	Start(Accumulator) error

	// Stop stops the services and closes any necessary channels and connections
	Stop()
}

type ServiceOutput

type ServiceOutput interface {
	// Connect to the Output
	Connect() error
	// Close any connections to the Output
	Close() error
	// Description returns a one-sentence description on the Output
	Description() string
	// SampleConfig returns the default configuration of the Output
	SampleConfig() string
	// Write takes in group of points to be written to the Output
	Write(metrics []Metric) error
	// Start the "service" that will provide an Output
	Start() error
	// Stop the "service" that will provide an Output
	Stop()
}

type ValueType

type ValueType int

ValueType is an enumeration of metric types that represent a simple value.

const (
	Counter ValueType
	Gauge
	Untyped
)

Possible values for the ValueType enum.