smartsifter

README

go-smartsifter

SmartSifter: On-line Unsupervised Outlier Detection Using Finite Mixtures with Discounting Learning Algorithms. This method is proposed by Yamanishi, K., Takeuchi, J., Williams, G. et al. (2004)

refs: http://cs.fit.edu/~pkc/id/related/yamanishi-kdd00.pdf

Usage

r := 0.1 // Discounting parameter.
alpha := 1.5 // Hyper parameter for continuous variables.
beta := 1.0 // Hyper parameter for categorical variables.
cellNum := 0 // Only continuous variables.
mixtureNum := 2 // Number of mixtures for GMM.
dim := 2 // Number of dimentions for GMM.

ss := smartsifter.NewSmartSifter(r, alpha, beta, cellNum, mixtureNum, dim)
logLoss := ss.Input(nil, []float64{0.1, 0.2}, true)
fmt.Println("Score using logLoss: %f\n", logLoss)

Examples

Online outlier detection for faithful data (Only continuous variables and using parametric method).

$ go run examples/example.go
$ wget -O plot.py https://gist.github.com/monochromegane/8b6a2a18084297e05f3d25bde2518a9c
$ python plot.py
$ convert -adjoin out/*.jpg out.gif

If you want to try categorical variables, pass -x parameter to examples.go.

TODO

• Implement non-parametric method (SDPU).
• Implement Hellinger score.
• Implement CLI.

License

MIT

Author

monochromegane

Documentation

Index

• type CategoricalAlgorithm
• type Categorizer
• type ContinuousAlgorithm
• type GMM
• type NoOpCategoricalAlgorithm
• type NoOpContinuousAlgorithm
• type SDEM
• type SDLE
• type SmartSifter
  • func NewSmartSifter(discount, alpha, beta float64, cellNum, mixtureNum, dim int) *SmartSifter
  • func (ss *SmartSifter) Input(X []int, Y []float64, update bool) float64

Types

type CategoricalAlgorithm

type CategoricalAlgorithm interface {
	// contains filtered or unexported methods
}

type Categorizer

type Categorizer interface {
	Index(x []int) int
	Size(idx int) int
}

type ContinuousAlgorithm

type ContinuousAlgorithm interface {
	// contains filtered or unexported methods
}

type GMM

type GMM struct {
}

type NoOpCategoricalAlgorithm

type NoOpCategoricalAlgorithm struct {
}

type NoOpContinuousAlgorithm

type NoOpContinuousAlgorithm struct {
}

type SDEM

type SDEM struct {
	// contains filtered or unexported fields
}

type SDLE

type SDLE struct {
	// contains filtered or unexported fields
}

type SmartSifter

type SmartSifter struct {
	// contains filtered or unexported fields
}

func NewSmartSifter

func NewSmartSifter(discount, alpha, beta float64, cellNum, mixtureNum, dim int) *SmartSifter

func (*SmartSifter) Input

func (ss *SmartSifter) Input(X []int, Y []float64, update bool) float64