dynhist

README

Dynamic histogram collector for Go

This library implements streaming counter with dynamic bucketing by value size.

Usage

c := dynhist.Collector{
    BucketsLimit: 10,
}
src := rand.NewSource(1)
r := rand.New(src)

for i := 0; i < 10000; i++ {
    c.Add(r.Float64())
}

fmt.Println(c.String())
// Output:
// [ min  max]   cnt total% (10000 events)
// [0.00 0.11]  1099 10.99% ..........
// [0.11 0.22]  1093 10.93% ..........
// [0.22 0.33]  1127 11.27% ...........
// [0.33 0.44]  1121 11.21% ...........
// [0.44 0.54]   999  9.99% .........
// [0.54 0.63]   964  9.64% .........
// [0.63 0.73]   953  9.53% .........
// [0.73 0.81]   841  8.41% ........
// [0.81 0.90]   797  7.97% .......
// [0.90 1.00]  1006 10.06% ..........

Documentation

Overview

Package dynhist implements dynamic histogram collector.

Index

• Constants
• func AvgWidth(b1, b2, bTot Bucket) float64
• func ExpWidth(sumWidthPow, spacingPow float64) func(b1, b2, bTot Bucket) float64
• func LatencyWidth(b1, b2, bTot Bucket) float64
• type Bucket
• type Collector
  • func (c *Collector) Add(v float64)
  • func (c *Collector) LoadFromRuntimeMetrics(h *metrics.Float64Histogram)
  • func (c *Collector) Percentile(percent float64) float64
  • func (c *Collector) PercentileSum(percent float64) float64
  • func (c *Collector) String() string

Examples

• AvgWidth
• ExpWidth

Constants

const DefaultBucketsLimit = 20

DefaultBucketsLimit is a default maximum number of buckets.

Functions

func AvgWidth

func AvgWidth(b1, b2, bTot Bucket) float64

AvgWidth is a weight function to maintain equal width of all buckets.

Should fit for normally distributed data.

Example

c := dynhist.Collector{
	BucketsLimit: 10,
}
src := rand.NewSource(1)
r := rand.New(src)

for i := 0; i < 10000; i++ {
	c.Add(r.Float64())
}

fmt.Println(c.String())

Output:

[ min  max]   cnt total% (total count: 10000)
[0.00 0.11]  1099 10.99% ..........
[0.11 0.22]  1093 10.93% ..........
[0.22 0.33]  1127 11.27% ...........
[0.33 0.44]  1121 11.21% ...........
[0.44 0.54]   999  9.99% .........
[0.54 0.63]   964  9.64% .........
[0.63 0.73]   953  9.53% .........
[0.73 0.81]   841  8.41% ........
[0.81 0.90]   797  7.97% .......
[0.90 1.00]  1006 10.06% ..........

func ExpWidth

func ExpWidth(sumWidthPow, spacingPow float64) func(b1, b2, bTot Bucket) float64

ExpWidth creates a weight function with exponential bucket width growing.

For exponentially distributed data, values of 1.2 for sumWidthPow and 1 for spacingPow should be a good fit. Increase sumWidthPow to widen buckets for lower values. Increase spacingPow to widen buckets for higher values.

Example

c := dynhist.Collector{
	BucketsLimit: 10,
	WeightFunc:   dynhist.ExpWidth(1.2, 0.9),
}
src := rand.NewSource(1)
r := rand.New(src)

for i := 0; i < 100000; i++ {
	c.Add(r.ExpFloat64())
}

fmt.Println(c.String())

Output:

[  min   max]    cnt total% (total count: 100000)
[ 0.00  0.07]   6577  6.58% ......
[ 0.07  0.22]  13380 13.38% .............
[ 0.22  0.45]  16002 16.00% ................
[ 0.45  1.11]  31072 31.07% ...............................
[ 1.11  1.77]  15975 15.97% ...............
[ 1.77  2.78]  10737 10.74% ..........
[ 2.78  4.37]   4993  4.99% ....
[ 4.37  6.50]   1121  1.12% .
[ 6.51  8.96]    134  0.13%
[ 9.03 10.80]      9  0.01%

func LatencyWidth

func LatencyWidth(b1, b2, bTot Bucket) float64

LatencyWidth is a weight function suitable for collecting latency information.

It makes wider buckets for higher values, narrow buckets for lower values.

Types

type Bucket

type Bucket struct {
	Min   float64
	Max   float64
	Count int
	Sum   float64
}

Bucket keeps count of values in boundaries.

type Collector

type Collector struct {
	sync.Mutex

	// BucketsLimit limits total number of buckets used.
	BucketsLimit int

	// Bucket keeps total count.
	Bucket

	// Buckets is a list of available buckets.
	Buckets []Bucket

	// PrintSum enables printing of a summary value in a bucket.
	PrintSum bool

	// RawValues stores incoming events, disabled by default. Use non-nil value to enable.
	RawValues []float64

	// WeightFunc calculates weight of adjacent buckets with total available. Pair with minimal weight is merged.
	// AvgWidth is used by default.
	// See also LatencyWidth, ExpWidth.
	WeightFunc func(b1, b2, bTot Bucket) float64
}

Collector groups and counts values by size using buckets.

func (*Collector) Add

func (c *Collector) Add(v float64)

Add collects value.

func (*Collector) LoadFromRuntimeMetrics

func (c *Collector) LoadFromRuntimeMetrics(h *metrics.Float64Histogram)

LoadFromRuntimeMetrics replaces existing buckets with data from metrics.Float64Histogram.

func (*Collector) Percentile

func (c *Collector) Percentile(percent float64) float64

Percentile returns maximum boundary for a fraction of values.

func (*Collector) PercentileSum

func (c *Collector) PercentileSum(percent float64) float64

PercentileSum returns maximum boundary for a sum of smaller values.

func (*Collector) String

func (c *Collector) String() string

String renders buckets value.