tdigest

README

tdigest

This is an implementation of Ted Dunning's t-digest in Go.

The implementation is based off Derrick Burns' C++ implementation.

Example

package main

import (
	"log"

	"github.com/influxdata/tdigest"
)

func main() {
	td := tdigest.NewWithCompression(1000)
	for _, x := range []float64{1, 2, 3, 4, 5, 5, 4, 3, 2, 1} {
		td.Add(x, 1)
	}

	// Compute Quantiles
	log.Println("50th", td.Quantile(0.5))
	log.Println("75th", td.Quantile(0.75))
	log.Println("90th", td.Quantile(0.9))
	log.Println("99th", td.Quantile(0.99))

	// Compute CDFs
	log.Println("CDF(1) = ", td.CDF(1))
	log.Println("CDF(2) = ", td.CDF(2))
	log.Println("CDF(3) = ", td.CDF(3))
	log.Println("CDF(4) = ", td.CDF(4))
	log.Println("CDF(5) = ", td.CDF(5))
}

TODO

Only the methods for a single TDigest have been implemented. The methods to merge two or more existing t-digests into a single t-digest have yet to be implemented.

Documentation

Index

• Constants
• type Centroid
  • func (c *Centroid) Add(r Centroid) error
  • func (c *Centroid) String() string
• type CentroidList
  • func NewCentroidList(centroids []Centroid) CentroidList
  • func (l *CentroidList) Clear()
  • func (l CentroidList) Len() int
  • func (l CentroidList) Less(i, j int) bool
  • func (l CentroidList) Swap(i, j int)
• type Error
  • func (e Error) Error() string
• type TDigest
  • func New() *TDigest
  • func NewWithCompression(c float64) *TDigest
  • func (t *TDigest) Add(x, w float64)
  • func (t *TDigest) AddCentroid(c Centroid)
  • func (t *TDigest) AddCentroidList(c CentroidList)
  • func (t *TDigest) CDF(x float64) float64
  • func (t *TDigest) Centroids() CentroidList
  • func (t *TDigest) Count() float64
  • func (t *TDigest) Quantile(q float64) float64
  • func (t *TDigest) Reset()

Constants

const ErrWeightLessThanZero = Error("centroid weight cannot be less than zero")

ErrWeightLessThanZero is used when the weight is not able to be processed.

Types

type Centroid

type Centroid struct {
	Mean   float64
	Weight float64
}

Centroid average position of all points in a shape

func (*Centroid) Add

func (c *Centroid) Add(r Centroid) error

Add averages the two centroids together and update this centroid

func (*Centroid) String

func (c *Centroid) String() string

type CentroidList

type CentroidList []Centroid

CentroidList is sorted by the Mean of the centroid, ascending.

func NewCentroidList

func NewCentroidList(centroids []Centroid) CentroidList

NewCentroidList creates a priority queue for the centroids

func (*CentroidList) Clear

func (l *CentroidList) Clear()

Clear clears the list.

func (CentroidList) Len

func (l CentroidList) Len() int

func (CentroidList) Less

func (l CentroidList) Less(i, j int) bool

func (CentroidList) Swap

func (l CentroidList) Swap(i, j int)

type Error

type Error string

Error is a domain error encountered while processing tdigests

func (Error) Error

func (e Error) Error() string

type TDigest

type TDigest struct {
	Compression float64
	// contains filtered or unexported fields
}

TDigest is a data structure for accurate on-line accumulation of rank-based statistics such as quantiles and trimmed means.

func New

func New() *TDigest

New initializes a new distribution with a default compression.

func NewWithCompression

func NewWithCompression(c float64) *TDigest

NewWithCompression initializes a new distribution with custom compression.

func (*TDigest) Add

func (t *TDigest) Add(x, w float64)

Add adds a value x with a weight w to the distribution.

func (*TDigest) AddCentroid

func (t *TDigest) AddCentroid(c Centroid)

AddCentroid adds a single centroid.

func (*TDigest) AddCentroidList

func (t *TDigest) AddCentroidList(c CentroidList)

AddCentroidList can quickly add multiple centroids.

func (*TDigest) CDF

func (t *TDigest) CDF(x float64) float64

CDF returns the cumulative distribution function for a given value x.

func (*TDigest) Centroids

func (t *TDigest) Centroids() CentroidList

Centroids returns a copy of processed centroids. Useful when aggregating multiple t-digests.

Pass in the CentroidList as the buffer to write into.

func (*TDigest) Count

func (t *TDigest) Count() float64

func (*TDigest) Quantile

func (t *TDigest) Quantile(q float64) float64

Quantile returns the (approximate) quantile of the distribution. Accepted values for q are between 0.0 and 1.0. Returns NaN if Count is zero or bad inputs.

func (*TDigest) Reset

func (t *TDigest) Reset()

Reset resets the distribution to its initial state.