skyline

README

Skyline's Algorithm

golang version based on https://github.com/etsy/skyline/blob/master/src/analyzer/algorithms.py#L230

some package are public domain code

Todo

add missed adfuller

Documentation

Index

• func EwmStd(series []float64, com float64) []float64
• func Ewma(series []float64, com float64) []float64
• func FirstHourAverage(timeseries []TimePoint, fullDuration int64) bool
• func Grubbs(timeseries []TimePoint) bool
• func Histogram(series []float64, bins int) ([]int, []float64)
• func HistogramBins(timeseries []TimePoint) bool
• func KolmogorovSmirnov(data1, data2 []float64, α float64) (bool, float64, float64)
• func KsTest(timeseries []TimePoint) bool
• func LeastSquares(timeseries []TimePoint) bool
• func MeanSubtractionCumulation(timeseries []TimePoint) bool
• func Median(series []float64) float64
• func MedianAbsoluteDeviation(timeseries []TimePoint) bool
• func SimpleStddevFromMovingAverage(timeseries []TimePoint) bool
• func StddevFromMovingAverage(timeseries []TimePoint) bool
• func TailAvg(series []float64) float64
• func TimeArray(timeseries []TimePoint) []int64
• func TimeArray64(timeseries []TimePoint) []float64
• func ValueArray(timeseries []TimePoint) []float64
• type TimePoint
  • func IsAnomalouslyAnomalous(trigger_history []TimePoint, new_trigger TimePoint) (bool, []TimePoint)

Functions

func EwmStd

func EwmStd(series []float64, com float64) []float64

EwmStd Exponentially-weighted moving std

func Ewma

func Ewma(series []float64, com float64) []float64

Ewma

func FirstHourAverage

func FirstHourAverage(timeseries []TimePoint, fullDuration int64) bool

FirstHourAverage function Calcuate the simple average over one hour, FULLDURATION seconds ago. A timeseries is anomalous if the average of the last three datapoints are outside of three standard deviations of this value.

func Grubbs

func Grubbs(timeseries []TimePoint) bool

Grubbs score A timeseries is anomalous if the Z score is greater than the Grubb's score.

func Histogram

func Histogram(series []float64, bins int) ([]int, []float64)

Histogram numpy.histogram

func HistogramBins

func HistogramBins(timeseries []TimePoint) bool

HistogramBins function A timeseries is anomalous if the average of the last three datapoints falls into a histogram bin with less than 20 other datapoints (you'll need to tweak that number depending on your data) Returns: the size of the bin which contains the tailAvg. Smaller bin size means more anomalous.

func KolmogorovSmirnov

func KolmogorovSmirnov(data1, data2 []float64, α float64) (bool, float64, float64)

KolmogorovSmirnov performs the two-sample Kolmogorov–Smirnov test. The null hypothesis is that the two datasets are coming from the same continuous distribution. The α parameter specifies the significance level. If the test rejects the null hypothesis, the function returns true; otherwise, false is returned. The second and third outputs of the function are the p-value and Kolmogorov–Smirnov statistic of the test, respectively.

https://en.wikipedia.org/wiki/Kolmogorov%E2%80%93Smirnov_test

func KsTest

func KsTest(timeseries []TimePoint) bool

KsTest function A timeseries is anomalous if 2 sample Kolmogorov-Smirnov test indicates that data distribution for last 10 minutes is different from last hour. It produces false positives on non-stationary series so Augmented Dickey-Fuller test applied to check for stationarity.

func LeastSquares

func LeastSquares(timeseries []TimePoint) bool

LeastSquares function A timeseries is anomalous if the average of the last three datapoints on a projected least squares model is greater than three sigma.

func MeanSubtractionCumulation

func MeanSubtractionCumulation(timeseries []TimePoint) bool

MeanSubtractionCumulation function A timeseries is anomalous if the value of the next datapoint in the series is farther than a standard deviation out in cumulative terms after subtracting the mean from each data point.

func Median

func Median(series []float64) float64

Median series.median

func MedianAbsoluteDeviation

func MedianAbsoluteDeviation(timeseries []TimePoint) bool

MedianAbsoluteDeviation function A timeseries is anomalous if the deviation of its latest datapoint with respect to the median is X times larger than the median of deviations.

func SimpleStddevFromMovingAverage

func SimpleStddevFromMovingAverage(timeseries []TimePoint) bool

SimpleStddevFromMovingAverage function A timeseries is anomalous if the absolute value of the average of the latest three datapoint minus the moving average is greater than one standard deviation of the average. This does not exponentially weight the MA and so is better for detecting anomalies with respect to the entire series.

func StddevFromMovingAverage

func StddevFromMovingAverage(timeseries []TimePoint) bool

StddevFromMovingAverage function A timeseries is anomalous if the absolute value of the average of the latest three datapoint minus the moving average is greater than one standard deviation of the moving average. This is better for finding anomalies with respect to the short term trends.

func TailAvg

func TailAvg(series []float64) float64

TailAvg is a utility function used to calculate the average of the last three datapoints in the series as a measure, instead of just the last datapoint. It reduces noise, but it also reduces sensitivity and increases the delay to detection.

func TimeArray

func TimeArray(timeseries []TimePoint) []int64

TimeArray return all timestamps in timeseries array

func TimeArray64

func TimeArray64(timeseries []TimePoint) []float64

TimeArray64 return all timestamps in timeseries array

func ValueArray

func ValueArray(timeseries []TimePoint) []float64

ValueArray return all values in timeseries array

Types

type TimePoint

type TimePoint interface {
	GetTimestamp() int64 //x time
	GetValue() float64   //y value
}

TimePoint is basic data struct

func IsAnomalouslyAnomalous

func IsAnomalouslyAnomalous(trigger_history []TimePoint, new_trigger TimePoint) (bool, []TimePoint)

IsAnomalouslyAnomalous function This method runs a meta-analysis on the metric to determine whether the metric has a past history of triggering. TODO: weight intervals based on datapoint