filters

Documentation

Index

• type AbstractDiscretizeFilter
  • func (d *AbstractDiscretizeFilter) AddAttribute(a base.Attribute) error
  • func (d *AbstractDiscretizeFilter) GetAttributesAfterFiltering() []base.FilteredAttribute
• type BinaryConvertFilter
  • func NewBinaryConvertFilter() *BinaryConvertFilter
  • func (b *BinaryConvertFilter) AddAttribute(a base.Attribute) error
  • func (b *BinaryConvertFilter) GetAttributesAfterFiltering() []base.FilteredAttribute
  • func (b *BinaryConvertFilter) String() string
  • func (b *BinaryConvertFilter) Train() error
  • func (b *BinaryConvertFilter) Transform(a base.Attribute, n base.Attribute, attrBytes []byte) []byte
• type BinningFilter
  • func NewBinningFilter(d base.FixedDataGrid, bins int) *BinningFilter
  • func (b *BinningFilter) GetAttributesAfterFiltering() []base.FilteredAttribute
  • func (b *BinningFilter) String() string
  • func (b *BinningFilter) Train() error
  • func (b *BinningFilter) Transform(a base.Attribute, n base.Attribute, field []byte) []byte
• type ChiMergeFilter
  • func NewChiMergeFilter(d base.FixedDataGrid, significance float64) *ChiMergeFilter
  • func (c *ChiMergeFilter) GetAttributesAfterFiltering() []base.FilteredAttribute
  • func (c *ChiMergeFilter) String() string
  • func (c *ChiMergeFilter) Train() error
  • func (c *ChiMergeFilter) Transform(a base.Attribute, n base.Attribute, field []byte) []byte
• type FloatConvertFilter
  • func NewFloatConvertFilter() *FloatConvertFilter
  • func (f *FloatConvertFilter) AddAttribute(a base.Attribute) error
  • func (f *FloatConvertFilter) GetAttributesAfterFiltering() []base.FilteredAttribute
  • func (f *FloatConvertFilter) String() string
  • func (f *FloatConvertFilter) Train() error
  • func (f *FloatConvertFilter) Transform(a base.Attribute, n base.Attribute, attrBytes []byte) []byte
• type FrequencyTableEntry
  • func ChiMBuildFrequencyTable(attr base.Attribute, inst base.FixedDataGrid) []*FrequencyTableEntry
  • func (t *FrequencyTableEntry) String() string

Types

type AbstractDiscretizeFilter

type AbstractDiscretizeFilter struct {
	// contains filtered or unexported fields
}

func (*AbstractDiscretizeFilter) AddAttribute

func (d *AbstractDiscretizeFilter) AddAttribute(a base.Attribute) error

AddAttribute adds the AttributeSpec of the given attribute `a' to the AbstractFloatFilter for discretisation.

func (*AbstractDiscretizeFilter) GetAttributesAfterFiltering

func (d *AbstractDiscretizeFilter) GetAttributesAfterFiltering() []base.FilteredAttribute

GetAttributesAfterFiltering gets a list of before/after Attributes as base.FilteredAttributes

type BinaryConvertFilter

type BinaryConvertFilter struct {
	// contains filtered or unexported fields
}

BinaryConvertFilters convert a given DataGrid into one which only contains BinaryAttributes.

FloatAttributes are discretised into either 0 (if the value is 0) or 1 (if the value is not 0).

CategoricalAttributes are discretised into one or more new BinaryAttributes.

func NewBinaryConvertFilter

func NewBinaryConvertFilter() *BinaryConvertFilter

NewBinaryConvertFilter creates a blank BinaryConvertFilter

func (*BinaryConvertFilter) AddAttribute

func (b *BinaryConvertFilter) AddAttribute(a base.Attribute) error

AddAttribute adds a new Attribute to this Filter

func (*BinaryConvertFilter) GetAttributesAfterFiltering

func (b *BinaryConvertFilter) GetAttributesAfterFiltering() []base.FilteredAttribute

GetAttributesAfterFiltering returns the Attributes previously computed via Train()

func (*BinaryConvertFilter) String

func (b *BinaryConvertFilter) String() string

String gets a human-readable string

func (*BinaryConvertFilter) Train

func (b *BinaryConvertFilter) Train() error

Train converts the FloatAttributesinto equivalently named BinaryAttributes, leaves BinaryAttributes unmodified and processes CategoricalAttributes as follows.

If the CategoricalAttribute has two values, one of them is designated 0 and the other 1, and a single identically-named binary Attribute is returned.

If the CategoricalAttribute has more than two (n) values, the Filter generates n BinaryAttributes and sets each of them if the value's observed.

func (*BinaryConvertFilter) Transform

func (b *BinaryConvertFilter) Transform(a base.Attribute, n base.Attribute, attrBytes []byte) []byte

Transform converts the given byte sequence using the old Attribute into the new byte sequence.

If the old Attribute has a categorical value of at most two items, then a zero or non-zero byte sequence is returned.

If the old Attribute has a categorical value of at most n-items, then a non-zero or zero byte sequence is returned based on the value of the new Attribute passed in.

If the old Attribute is a float, it's value's unpacked and we check for non-zeroness

If the old Attribute is a BinaryAttribute, just return the input

type BinningFilter

type BinningFilter struct {
	AbstractDiscretizeFilter
	// contains filtered or unexported fields
}

BinningFilter does equal-width binning for numeric Attributes (aka "histogram binning")

func NewBinningFilter

func NewBinningFilter(d base.FixedDataGrid, bins int) *BinningFilter

NewBinningFilter creates a BinningFilter structure with some helpful default initialisations.

func (*BinningFilter) GetAttributesAfterFiltering

func (b *BinningFilter) GetAttributesAfterFiltering() []base.FilteredAttribute

GetAttributesAfterFiltering gets a list of before/after Attributes as base.FilteredAttributes

func (*BinningFilter) String

func (b *BinningFilter) String() string

func (*BinningFilter) Train

func (b *BinningFilter) Train() error

Train computes and stores the bin values for the training instances.

func (*BinningFilter) Transform

func (b *BinningFilter) Transform(a base.Attribute, n base.Attribute, field []byte) []byte

Transform takes an Attribute and byte sequence and returns the transformed byte sequence.

type ChiMergeFilter

type ChiMergeFilter struct {
	AbstractDiscretizeFilter

	Significance float64
	MinRows      int
	MaxRows      int
	// contains filtered or unexported fields
}

ChiMergeFilter implements supervised discretisation by merging successive numeric intervals if the difference in their class distribution is not statistically signficant. See Bramer, "Principles of Data Mining", 2nd Edition

pp 105--115

func NewChiMergeFilter

func NewChiMergeFilter(d base.FixedDataGrid, significance float64) *ChiMergeFilter

NewChiMergeFilter creates a ChiMergeFilter with some helpful intialisations.

func (*ChiMergeFilter) GetAttributesAfterFiltering

func (c *ChiMergeFilter) GetAttributesAfterFiltering() []base.FilteredAttribute

GetAttributesAfterFiltering gets a list of before/after Attributes as base.FilteredAttributes

func (*ChiMergeFilter) String

func (c *ChiMergeFilter) String() string

func (*ChiMergeFilter) Train

func (c *ChiMergeFilter) Train() error

func (*ChiMergeFilter) Transform

func (c *ChiMergeFilter) Transform(a base.Attribute, n base.Attribute, field []byte) []byte

Transform returns the byte sequence after discretisation

type FloatConvertFilter

type FloatConvertFilter struct {
	// contains filtered or unexported fields
}

FloatConvertFilters convert a given DataGrid into one which only contains BinaryAttributes.

FloatAttributes are discretised into either 0 (if the value is 0) or 1 (if the value is not 0).

CategoricalAttributes are discretised into one or more new BinaryAttributes.

func NewFloatConvertFilter

func NewFloatConvertFilter() *FloatConvertFilter

NewFloatConvertFilter creates a blank FloatConvertFilter

func (*FloatConvertFilter) AddAttribute

func (f *FloatConvertFilter) AddAttribute(a base.Attribute) error

AddAttribute adds a new Attribute to this Filter

func (*FloatConvertFilter) GetAttributesAfterFiltering

func (f *FloatConvertFilter) GetAttributesAfterFiltering() []base.FilteredAttribute

GetAttributesAfterFiltering returns the Attributes previously computed via Train()

func (*FloatConvertFilter) String

func (f *FloatConvertFilter) String() string

String gets a human-readable string

func (*FloatConvertFilter) Train

func (f *FloatConvertFilter) Train() error

Train converts the Attributes into equivalently named FloatAttributes, leaves FloatAttributes unmodified and processes CategoricalAttributes as follows.

If the CategoricalAttribute has two values, one of them is designated 0.0 and the other 1.0, and a single identically-named FloatAttribute is returned.

If the CategoricalAttribute has more than two (n) values, the Filter generates n FloatAttributes and sets each of them if the value's observed.

func (*FloatConvertFilter) Transform

func (f *FloatConvertFilter) Transform(a base.Attribute, n base.Attribute, attrBytes []byte) []byte

type FrequencyTableEntry

type FrequencyTableEntry struct {
	Value     float64
	Frequency map[string]int
}

FrequencyTableEntry is a struct holding a value and a map of frequency

func ChiMBuildFrequencyTable

func ChiMBuildFrequencyTable(attr base.Attribute, inst base.FixedDataGrid) []*FrequencyTableEntry

func (*FrequencyTableEntry) String

func (t *FrequencyTableEntry) String() string