s3

package
Version: v2.0.0+incompatible Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Jan 9, 2018 License: MIT Imports: 17 Imported by: 0

README

S3 Plugin

The S3 plugin archives every flush to S3 as a separate S3 object.

This plugin is still in an experimental state.

Documentation

Index

Constants

View Source
const (
	// the order in which these appear determines the
	// order of the fields in the resultant TSV
	TsvName tsvField = iota
	TsvTags
	TsvMetricType

	// The hostName of the server flushing the data
	TsvVeneurHostname

	TsvInterval

	TsvTimestamp
	TsvValue

	// This is the _partition field
	// required by the Redshift IncrementalLoader.
	// For our purposes, the current date is a good partition.
	TsvPartition
)
View Source
const PartitionDateFormat = "20060102"
View Source
const RedshiftDateFormat = "2006-01-02 03:04:05"

Variables

View Source
var S3Bucket = "stripe-veneur"
View Source
var S3ClientUninitializedError = errors.New("s3 client has not been initialized")

Functions

func EncodeInterMetricCSV added in v1.7.0

func EncodeInterMetricCSV(d samplers.InterMetric, w *csv.Writer, partitionDate *time.Time, hostName string, interval int) error

EncodeInterMetricCSV generates a newline-terminated CSV row that describes the data represented by the InterMetric. The caller is responsible for setting w.Comma as the appropriate delimiter. For performance, encodeCSV does not flush after every call; the caller is expected to flush at the end of the operation cycle

func EncodeInterMetricsCSV added in v1.7.0

func EncodeInterMetricsCSV(metrics []samplers.InterMetric, delimiter rune, includeHeaders bool, hostname string, interval int) (io.ReadSeeker, error)

EncodeInterMetricsCSV returns a reader containing the gzipped CSV representation of the InterMetric data, one row per InterMetric. the AWS sdk requires seekable input, so we return a ReadSeeker here

func S3Path

func S3Path(hostname string, ft filetype) *string

Types

type S3Plugin

type S3Plugin struct {
	Logger   *logrus.Logger
	Svc      s3iface.S3API
	S3Bucket string
	Hostname string
	Interval int
}

func (*S3Plugin) Flush

func (p *S3Plugin) Flush(ctx context.Context, metrics []samplers.InterMetric) error

func (*S3Plugin) Name

func (p *S3Plugin) Name() string

func (*S3Plugin) S3Post

func (p *S3Plugin) S3Post(hostname string, data io.ReadSeeker, ft filetype) error

Source Files

Directories

Path Synopsis

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL