s3

README

S3 Plugin

The S3 plugin archives every flush to S3 as a separate S3 object.

This plugin is still in an experimental state.

Documentation

Index

• Constants
• Variables
• func EncodeInterMetricCSV(d samplers.InterMetric, w *csv.Writer, partitionDate *time.Time, ...) error
• func EncodeInterMetricsCSV(metrics []samplers.InterMetric, delimiter rune, includeHeaders bool, ...) (io.ReadSeeker, error)
• func S3Path(hostname string, ft filetype) *string
• type S3Plugin
  • func (p *S3Plugin) Flush(ctx context.Context, metrics []samplers.InterMetric) error
  • func (p *S3Plugin) Name() string
  • func (p *S3Plugin) S3Post(hostname string, data io.ReadSeeker, ft filetype) error

Constants

const (
	// the order in which these appear determines the
	// order of the fields in the resultant TSV
	TsvName tsvField = iota
	TsvTags
	TsvMetricType

	// The hostName of the server flushing the data
	TsvVeneurHostname

	TsvInterval

	TsvTimestamp
	TsvValue

	// This is the _partition field
	// required by the Redshift IncrementalLoader.
	// For our purposes, the current date is a good partition.
	TsvPartition
)

const PartitionDateFormat = "20060102"

const RedshiftDateFormat = "2006-01-02 03:04:05"

Variables

var S3Bucket = "stripe-veneur"

var S3ClientUninitializedError = errors.New("s3 client has not been initialized")

Functions

func EncodeInterMetricCSV

func EncodeInterMetricCSV(d samplers.InterMetric, w *csv.Writer, partitionDate *time.Time, hostName string, interval int) error

EncodeInterMetricCSV generates a newline-terminated CSV row that describes the data represented by the InterMetric. The caller is responsible for setting w.Comma as the appropriate delimiter. For performance, encodeCSV does not flush after every call; the caller is expected to flush at the end of the operation cycle

func EncodeInterMetricsCSV

func EncodeInterMetricsCSV(metrics []samplers.InterMetric, delimiter rune, includeHeaders bool, hostname string, interval int) (io.ReadSeeker, error)

EncodeInterMetricsCSV returns a reader containing the gzipped CSV representation of the InterMetric data, one row per InterMetric. the AWS sdk requires seekable input, so we return a ReadSeeker here

func S3Path

func S3Path(hostname string, ft filetype) *string

Types

type S3Plugin

type S3Plugin struct {
	Logger   *logrus.Logger
	Svc      s3iface.S3API
	S3Bucket string
	Hostname string
	Interval int
}

func (*S3Plugin) Flush

func (p *S3Plugin) Flush(ctx context.Context, metrics []samplers.InterMetric) error

func (*S3Plugin) Name

func (p *S3Plugin) Name() string

func (*S3Plugin) S3Post

func (p *S3Plugin) S3Post(hostname string, data io.ReadSeeker, ft filetype) error