kinesis

README

Amazon Kinesis Output for Telegraf

This is an experimental plugin that is still in the early stages of development. It will batch up all of the Points in one Put request to Kinesis. This should save the number of API requests by a considerable level.

About Kinesis

This is not the place to document all of the various Kinesis terms however it maybe useful for users to review Amazons official documentation which is available here.

Amazon Authentication

This plugin uses a credential chain for Authentication with the Kinesis API endpoint. In the following order the plugin will attempt to authenticate.

1. Assumed credentials via STS if role_arn attribute is specified (source credentials are evaluated from subsequent rules)
2. Explicit credentials from access_key, secret_key, and token attributes
3. Shared profile from profile attribute
4. Environment Variables
5. Shared Credentials
6. EC2 Instance Profile

Config

For this output plugin to function correctly the following variables must be configured.

• region
• streamname
• partitionkey

region

The region is the Amazon region that you wish to connect to. Examples include but are not limited to

• us-west-1
• us-west-2
• us-east-1
• ap-southeast-1
• ap-southeast-2

streamname

The streamname is used by the plugin to ensure that data is sent to the correct Kinesis stream. It is important to note that the stream MUST be pre-configured for this plugin to function correctly. If the stream does not exist the plugin will result in telegraf exiting with an exit code of 1.

partitionkey

This is used to group data within a stream. Currently this plugin only supports a single partitionkey. Manually configuring different hosts, or groups of hosts with manually selected partitionkeys might be a workable solution to scale out.

format

The format configuration value has been designated to allow people to change the format of the Point as written to Kinesis. Right now there are two supported formats string and custom.

string

String is defined using the default Point.String() value and translated to []byte for the Kinesis stream.

custom

Custom is a string defined by a number of values in the FormatMetric() function.

Documentation

Index

• type KinesisOutput
  • func (k *KinesisOutput) Close() error
  • func (k *KinesisOutput) Connect() error
  • func (k *KinesisOutput) Description() string
  • func (k *KinesisOutput) SampleConfig() string
  • func (k *KinesisOutput) SetSerializer(serializer serializers.Serializer)
  • func (k *KinesisOutput) Write(metrics []telegraf.Metric) error

Types

type KinesisOutput

type KinesisOutput struct {
	Region    string `toml:"region"`
	AccessKey string `toml:"access_key"`
	SecretKey string `toml:"secret_key"`
	RoleARN   string `toml:"role_arn"`
	Profile   string `toml:"profile"`
	Filename  string `toml:"shared_credential_file"`
	Token     string `toml:"token"`

	StreamName   string `toml:"streamname"`
	PartitionKey string `toml:"partitionkey"`
	Debug        bool   `toml:"debug"`
	// contains filtered or unexported fields
}

func (*KinesisOutput) Close

func (k *KinesisOutput) Close() error

func (*KinesisOutput) Connect

func (k *KinesisOutput) Connect() error

func (*KinesisOutput) Description

func (k *KinesisOutput) Description() string

func (*KinesisOutput) SampleConfig

func (k *KinesisOutput) SampleConfig() string

func (*KinesisOutput) SetSerializer

func (k *KinesisOutput) SetSerializer(serializer serializers.Serializer)

func (*KinesisOutput) Write

func (k *KinesisOutput) Write(metrics []telegraf.Metric) error