cortex

README

Cortex Sink

This sink sends Veneur metrics to cortex via prometheus remote-write protocol

Configuration

metric_sinks:
  - kind: cortex
    name: cortex1
    config:
# The URL of the endpoint to send samples to.
      url: http://localhost:9090/api/v1/receive
# Timeout for requests to the remote write endpoint.
      remote_timeout: 30s
# Optional batch write size
# note: this doesn't mean if a given flush contains 13 metrics that only 10 will be written, rather that
# for a veneur flush interval the sink will do two writes: one for 10 metrics and another for 3 
      batch_write_size: 10
# Optional proxy URL.
      proxy_url: http://localhost:1080
# Optional added headers
      headers:
        My-Header: custom header content
# Optional basic auth
      basic_auth:
        username: user1
        password: the-password
# Optional authorization (Bearer or other), if no basic auth
      authorization:
        # Defaults to Bearer if not specified
        type: Bearer
        credentials: my-token

Duplicate Labels

Cortex does that handle duplicate labels, it will reject the write. So we've added logic to the cortex sink to dedupe labels with a last-one-wins approach.

For example, for the given metric:

my_metric{foo="bar", host="computer1", foo="baz"} 48.9

We'll dedupe it to:

my_metric{host="computer1", foo="baz"} 48.9

What about common tags?

Veneur has a mechanism for specifying tags that should be attached to all metrics. In the case these tags duplicate already existing one on the metric, the common tag will take precedence.

Documentation

Index

• Constants
• func Create(server *veneur.Server, name string, logger *logrus.Entry, config veneur.Config, ...) (sinks.MetricSink, error)
• func ParseConfig(name string, config interface{}) (veneur.MetricSinkConfig, error)
• type BasicAuthType
• type CortexMetricSink
  • func NewCortexMetricSink(URL string, timeout time.Duration, proxyURL string, logger *logrus.Entry, ...) (*CortexMetricSink, error)
  • func (s *CortexMetricSink) Flush(ctx context.Context, metrics []samplers.InterMetric) error
  • func (s *CortexMetricSink) FlushOtherSamples(context.Context, []ssf.SSFSample)
  • func (s *CortexMetricSink) Name() string
  • func (s *CortexMetricSink) Start(tc *trace.Client) error
• type CortexMetricSinkConfig

Constants

const (
	// MaxConns is the number of simultaneous connections we allow to one host
	MaxConns = 100
	// DefaultRemoteTimeout after which a write request will time out
	DefaultRemoteTimeout = time.Duration(30 * time.Second)
	// DefaultAuthorizationType if a credential is supplied
	DefaultAuthorizationType = "Bearer"
)

Functions

func Create

func Create(
	server *veneur.Server, name string, logger *logrus.Entry,
	config veneur.Config, sinkConfig veneur.MetricSinkConfig,
) (sinks.MetricSink, error)

Create creates a new Cortex sink. @see veneur.MetricSinkTypes

func ParseConfig

func ParseConfig(
	name string, config interface{},
) (veneur.MetricSinkConfig, error)

ParseConfig extracts Cortex specific fields from the global veneur config

Types

type BasicAuthType

type BasicAuthType struct {
	Username util.StringSecret `yaml:"username"`
	Password util.StringSecret `yaml:"password"`
}

type CortexMetricSink

type CortexMetricSink struct {
	URL           string
	RemoteTimeout time.Duration
	ProxyURL      string
	Client        *http.Client
	// contains filtered or unexported fields
}

CortexMetricSink writes metrics to one or more configured Cortex instances using the prometheus remote-write API. For specifications, see https://github.com/prometheus/compliance/tree/main/remote_write

func NewCortexMetricSink

func NewCortexMetricSink(URL string, timeout time.Duration, proxyURL string, logger *logrus.Entry, name string, tags map[string]string, headers map[string]string, basicAuth *BasicAuthType, batchWriteSize int) (*CortexMetricSink, error)

NewCortexMetricSink creates and returns a new instance of the sink

func (*CortexMetricSink) Flush

func (s *CortexMetricSink) Flush(ctx context.Context, metrics []samplers.InterMetric) error

Flush sends a batch of metrics to the configured remote-write endpoint

func (*CortexMetricSink) FlushOtherSamples

func (s *CortexMetricSink) FlushOtherSamples(context.Context, []ssf.SSFSample)

FlushOtherSamples would forward non-metric sanples like spans. Prometheus cannot receive them, so this is a no-op.

func (*CortexMetricSink) Name

func (s *CortexMetricSink) Name() string

Name returns the string cortex

func (*CortexMetricSink) Start

func (s *CortexMetricSink) Start(tc *trace.Client) error

Start sets up the HTTP client for writing to Cortex

type CortexMetricSinkConfig

type CortexMetricSinkConfig struct {
	URL            string            `yaml:"url"`
	RemoteTimeout  time.Duration     `yaml:"remote_timeout"`
	ProxyURL       string            `yaml:"proxy_url"`
	BatchWriteSize int               `yaml:"batch_write_size"`
	Headers        map[string]string `yaml:"headers"`
	BasicAuth      BasicAuthType     `yaml:"basic_auth"`
	Authorization  struct {
		Type       string            `yaml:"type"`
		Credential util.StringSecret `yaml:"credentials"`
	} `yaml:"authorization"`
}

TODO: implement queue config options, at least max_samples_per_send, max_shards, and capacity https://prometheus.io/docs/prometheus/latest/configuration/configuration/#remote_write