samplers

package
v13.1.0+incompatible Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Jan 7, 2020 License: MIT Imports: 16 Imported by: 107

Documentation

Index

Constants

This section is empty.

Variables

View Source 
var AggregatesLookup = map[string]Aggregate{
	"min":    AggregateMin,
	"max":    AggregateMax,
	"median": AggregateMedian,
	"avg":    AggregateAverage,
	"count":  AggregateCount,
	"sum":    AggregateSum,
	"hmean":  AggregateHarmonicMean,
}

Functions

func ParseEvent 

func ParseEvent(packet []byte) (*ssf.SSFSample, error)

ParseEvent parses a DogStatsD event packet and returns an SSF sample or an error on failure. To facilitate the many Datadog-specific values that are present in a DogStatsD event but not in an SSF sample, a series of special tags are set as defined in protocol/dogstatsd/protocol.go. Any sink that wants to consume these events will then need to implement FlushOtherSamples and unwind these special tags into whatever is appropriate for that sink.

func ParseTagSliceToMap added in v1.9.0 

func ParseTagSliceToMap(tags []string) map[string]string

ParseTagSliceToMap handles splitting a slice of string tags on `:` and creating a map from the parts.

func ValidMetric added in v1.5.2 

func ValidMetric(sample UDPMetric) bool

ValidMetric takes in an SSF sample and determines if it is valid or not.

Types

type Aggregate 

type Aggregate int
const (
	AggregateMin Aggregate = 1 << iota
	AggregateMax
	AggregateMedian
	AggregateAverage
	AggregateCount
	AggregateSum
	AggregateHarmonicMean
)

type Counter 

type Counter struct {
	Name string
	Tags []string
	// contains filtered or unexported fields
}

Counter is an accumulator

func NewCounter 

func NewCounter(Name string, Tags []string) *Counter

NewCounter generates and returns a new Counter.

func (*Counter) Combine 

func (c *Counter) Combine(other []byte) error

Combine merges the values seen with another set (marshalled as a byte slice)

func (*Counter) Export 

func (c *Counter) Export() (JSONMetric, error)

Export converts a Counter into a JSONMetric which reports the rate.

func (*Counter) Flush 

func (c *Counter) Flush(interval time.Duration) []InterMetric

Flush generates an InterMetric from the current state of this Counter.

func (*Counter) GetName added in v1.9.0 

func (c *Counter) GetName() string

GetName returns the name of the counter.

func (*Counter) Merge added in v1.9.0 

func (c *Counter) Merge(v *metricpb.CounterValue)

Merge adds the value from the input CounterValue to this one.

func (*Counter) Metric added in v1.9.0 

func (c *Counter) Metric() (*metricpb.Metric, error)

Metric returns a protobuf-compatible metricpb.Metric with values set at the time this function was called. This should be used to export a Counter for forwarding.

func (*Counter) Sample 

func (c *Counter) Sample(sample float64, sampleRate float32)

Sample adds a sample to the counter.

type DerivedMetricsProcessor added in v1.9.0 

type DerivedMetricsProcessor interface {
	SendSample(sample *ssf.SSFSample) error
}

DerivedMetricsProcessor processes any metric created from events or service checks into the worker channels for flushing

type Gauge 

type Gauge struct {
	Name string
	Tags []string
	// contains filtered or unexported fields
}

Gauge retains whatever the last value was.

func NewGauge 

func NewGauge(Name string, Tags []string) *Gauge

NewGauge generates an empty (valueless) Gauge

func (*Gauge) Combine added in v1.9.0 

func (g *Gauge) Combine(other []byte) error

Combine is pretty naïve for Gauges, as it just overwrites the value.

func (*Gauge) Export added in v1.9.0 

func (g *Gauge) Export() (JSONMetric, error)

Export converts a Gauge into a JSONMetric.

func (*Gauge) Flush 

func (g *Gauge) Flush() []InterMetric

Flush generates an InterMetric from the current state of this gauge.

func (*Gauge) GetName added in v1.9.0 

func (g *Gauge) GetName() string

GetName returns the name of the gauge.

func (*Gauge) Merge added in v1.9.0 

func (g *Gauge) Merge(v *metricpb.GaugeValue)

Merge sets the value of this Gauge to the value of the other.

func (*Gauge) Metric added in v1.9.0 

func (g *Gauge) Metric() (*metricpb.Metric, error)

Metric returns a protobuf-compatible metricpb.Metric with values set at the time this function was called. This should be used to export a Gauge for forwarding.

func (*Gauge) Sample 

func (g *Gauge) Sample(sample float64, sampleRate float32)

Sample takes on whatever value is passed in as a sample.

type Histo 

type Histo struct {
	Name  string
	Tags  []string
	Value *tdigest.MergingDigest
	// these values are computed from only the samples that came through this
	// veneur instance, ignoring any histograms merged from elsewhere
	// we separate them because they're easy to aggregate on the backend without
	// loss of granularity, and having host-local information on them might be
	// useful
	LocalWeight        float64
	LocalMin           float64
	LocalMax           float64
	LocalSum           float64
	LocalReciprocalSum float64
}

Histo is a collection of values that generates max, min, count, and percentiles over time.

func NewHist 

func NewHist(Name string, Tags []string) *Histo

NewHist generates a new Histo and returns it.

func (*Histo) Combine 

func (h *Histo) Combine(other []byte) error

Combine merges the values of a histogram with another histogram (marshalled as a byte slice)

func (*Histo) Export 

func (h *Histo) Export() (JSONMetric, error)

Export converts a Histogram into a JSONMetric

func (*Histo) Flush 

func (h *Histo) Flush(interval time.Duration, percentiles []float64, aggregates HistogramAggregates, global bool) []InterMetric

Flush generates InterMetrics for the current state of the Histo. percentiles indicates what percentiles should be exported from the histogram.

func (*Histo) GetName added in v1.9.0 

func (h *Histo) GetName() string

GetName returns the name of the Histo.

func (*Histo) Merge added in v1.9.0 

func (h *Histo) Merge(v *metricpb.HistogramValue)

Merge merges the t-digests of the two histograms and mutates the state of this one.

func (*Histo) Metric added in v1.9.0 

func (h *Histo) Metric() (*metricpb.Metric, error)

Metric returns a protobuf-compatible metricpb.Metric with values set at the time this function was called. This should be used to export a Histo for forwarding.

func (*Histo) Sample 

func (h *Histo) Sample(sample float64, sampleRate float32)

Sample adds the supplied value to the histogram.

type HistogramAggregates 

type HistogramAggregates struct {
	Value Aggregate
	Count int
}

type InterMetric added in v1.7.0 

type InterMetric struct {
	Name      string
	Timestamp int64
	Value     float64
	Tags      []string
	Type      MetricType
	Message   string
	HostName  string

	// Sinks, if non-nil, indicates which metric sinks a metric
	// should be inserted into. If nil, that means the metric is
	// meant to go to every sink.
	Sinks RouteInformation
}

InterMetric represents a metric that has been completed and is ready for flushing by sinks.

type InvalidMetrics added in v1.7.0 

type InvalidMetrics interface {
	error

	// Samples returns any samples that couldn't be parsed or validated.
	Samples() []*ssf.SSFSample
}

InvalidMetrics is an error type returned if any metric could not be parsed.

type JSONMetric 

type JSONMetric struct {
	MetricKey
	Tags []string `json:"tags"`
	// the Value is an internal representation of the metric's contents, eg a
	// gob-encoded histogram or hyperloglog.
	Value []byte `json:"value"`
}

JSONMetric is used to represent a metric that can be remarshaled with its internal state intact. It is used to send metrics from one Veneur to another.

type MetricKey 

type MetricKey struct {
	Name       string `json:"name"`
	Type       string `json:"type"`
	JoinedTags string `json:"tagstring"` // tags in deterministic order, joined with commas
}

MetricKey is a struct used to key the metrics into the worker's map. All fields must be comparable types.

func NewMetricKeyFromMetric added in v1.9.0 

func NewMetricKeyFromMetric(m *metricpb.Metric) MetricKey

NewMetricKeyFromMetric initializes a MetricKey from the protobuf-compatible metricpb.Metric

func (MetricKey) String added in v1.3.1 

func (m MetricKey) String() string

ToString returns a string representation of this MetricKey

type MetricScope 

type MetricScope int

MetricScope describes where the metric will be emitted. 

const (
	MixedScope MetricScope = iota
	LocalOnly
	GlobalOnly
)

func ScopeFromPB added in v1.9.0 

func ScopeFromPB(scope metricpb.Scope) MetricScope

ScopeFromPB creates an internal MetricScope type from the protobuf Scope type.

func (MetricScope) ToPB added in v1.9.0 

func (m MetricScope) ToPB() metricpb.Scope

ToPB maps the metric scope to a protobuf Scope type.

type MetricType added in v1.7.0 

type MetricType int

MetricType defines what kind of metric this is, so that we or our upstream sinks can do the right thing with it. 

const (
	// CounterMetric is a counter
	CounterMetric MetricType = iota
	// GaugeMetric is a gauge
	GaugeMetric
	// StatusMetric is a status (synonymous with a service check)
	StatusMetric
)

func (MetricType) String added in v1.7.0 

func (i MetricType) String() string

type RouteInformation added in v1.9.0 

type RouteInformation map[string]struct{}

RouteInformation is a key-only map indicating sink names that are supposed to receive a metric. A nil RouteInformation value corresponds to the "every sink" value; an entry in a non-nil RouteInformation means that the key should receive the metric.

func (RouteInformation) RouteTo added in v1.9.0 

func (ri RouteInformation) RouteTo(name string) bool

RouteTo returns true if the named sink should receive a metric according to the route table. A nil route table causes any sink to be eligible for the metric.

type Set 

type Set struct {
	Name string
	Tags []string
	Hll  *hyperloglog.Sketch
}

Set is a list of unique values seen.

func NewSet 

func NewSet(Name string, Tags []string) *Set

NewSet generates a new Set and returns it

func (*Set) Combine 

func (s *Set) Combine(other []byte) error

Combine merges the values seen with another set (marshalled as a byte slice)

func (*Set) Export 

func (s *Set) Export() (JSONMetric, error)

Export converts a Set into a JSONMetric which reports the Tags in the set.

func (*Set) Flush 

func (s *Set) Flush() []InterMetric

Flush generates an InterMetric for the state of this Set.

func (*Set) GetName added in v1.9.0 

func (s *Set) GetName() string

GetName returns the name of the set.

func (*Set) Merge added in v1.9.0 

func (s *Set) Merge(v *metricpb.SetValue) error

Merge combines the HyperLogLog with that of the input Set. Since the HyperLogLog is marshalled in the value, it unmarshals it first.

func (*Set) Metric added in v1.9.0 

func (s *Set) Metric() (*metricpb.Metric, error)

Metric returns a protobuf-compatible metricpb.Metric with values set at the time this function was called. This should be used to export a Set for forwarding.

func (*Set) Sample 

func (s *Set) Sample(sample string)

Sample checks if the supplied value has is already in the filter. If not, it increments the counter!

type SplitBytes 

type SplitBytes struct {
	// contains filtered or unexported fields
}

SplitBytes iterates over a byte buffer, returning chunks split by a given delimiter byte. It does not perform any allocations, and does not modify the buffer it is given. It is not safe for use by concurrent goroutines. 

sb := NewSplitBytes(buf, '\n')
for sb.Next() {
    fmt.Printf("%q\n", sb.Chunk())
}

The sequence of chunks returned by SplitBytes is equivalent to calling bytes.Split, except without allocating an intermediate slice.

func NewSplitBytes 

func NewSplitBytes(buf []byte, delim byte) *SplitBytes

NewSplitBytes initializes a SplitBytes struct with the provided buffer and delimiter.

func (*SplitBytes) Chunk 

func (sb *SplitBytes) Chunk() []byte

Chunk returns the current chunk.

func (*SplitBytes) Next 

func (sb *SplitBytes) Next() bool

Next advances SplitBytes to the next chunk, returning true if a new chunk actually exists and false otherwise.

type StatusCheck added in v1.9.0 

type StatusCheck struct {
	InterMetric
}

StatusCheck retains whatever the last value was.

func NewStatusCheck added in v1.9.0 

func NewStatusCheck(Name string, Tags []string) *StatusCheck

NewStatusCheck generates an empty (valueless) StatusCheck

func (*StatusCheck) Combine added in v1.9.0 

func (s *StatusCheck) Combine(other []byte) error

Combine is pretty naïve for StatusChecks, as it just overwrites the value.

func (*StatusCheck) Export added in v1.9.0 

func (s *StatusCheck) Export() (JSONMetric, error)

Export converts a StatusCheck into a JSONMetric.

func (*StatusCheck) Flush added in v1.9.0 

func (s *StatusCheck) Flush() []InterMetric

Flush generates an InterMetric from the current state of this status check.

func (*StatusCheck) Sample added in v1.9.0 

func (s *StatusCheck) Sample(sample float64, sampleRate float32, message string, hostname string)

Sample takes on whatever value is passed in as a sample.

type UDPMetric 

type UDPMetric struct {
	MetricKey
	Digest     uint32
	Value      interface{}
	SampleRate float32
	Tags       []string
	Scope      MetricScope
	Timestamp  int64
	Message    string
	HostName   string
}

UDPMetric is a representation of the sample provided by a client. The tag list should be deterministically ordered.

func ConvertIndicatorMetrics added in v1.9.0 

func ConvertIndicatorMetrics(span *ssf.SSFSpan, indicatorTimerName, objectiveTimerName string) (metrics []UDPMetric, err error)

ConvertIndicatorMetrics takes a span that may be an "indicator" span and returns metrics that can be determined from that span. Currently, it converts the span to two timer metrics for the duration of the span. One timer (the "indicator") is tagged with the span's service and error-ness. The other timer (the "objective") is tagged with the span's service, error-ness, and name. The name can be overridden with the ssf_objective tag.

func ConvertMetrics added in v1.8.0 

func ConvertMetrics(m *ssf.SSFSpan) ([]UDPMetric, error)

ConvertMetrics examines an SSF message, parses and returns a new array containing any metrics contained in the message. If any parse error occurs in processing any of the metrics, ExtractMetrics collects them into the error type InvalidMetrics and returns this error alongside any valid metrics that could be parsed.

func ConvertSpanUniquenessMetrics added in v1.9.0 

func ConvertSpanUniquenessMetrics(span *ssf.SSFSpan, rate float32) ([]UDPMetric, error)

ConvertSpanUniquenessMetrics takes a trace span and computes uniqueness metrics about it, returning UDPMetrics sampled at rate. Currently, the only metric returned is a Set counting the unique names per indicator span/service.

func ParseMetric 

func ParseMetric(packet []byte) (*UDPMetric, error)

ParseMetric converts the incoming packet from Datadog DogStatsD Datagram format in to a Metric. http://docs.datadoghq.com/guides/dogstatsd/#datagram-format

func ParseMetricSSF added in v1.5.0 

func ParseMetricSSF(metric *ssf.SSFSample) (UDPMetric, error)

ParseMetricSSF converts an incoming SSF packet to a Metric.

func ParseServiceCheck 

func ParseServiceCheck(packet []byte) (*UDPMetric, error)

ParseServiceCheck parses a packet that represents a service status check and returns a UDPMetric or an error on failure. The UDPMetric struct has explicit fields for each value of a service status check and does not require overloading magical tags for conversion.

Source Files

View all Source files

Directories

Path Synopsis
testutils

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.