tfidf

README

TF_IDF-Go

Implementation of TF_IDF in golang

Documentation

Overview

Package tfidf Provides a simple implementation of Term-Frequency Inverse-Document-Frequency in golang Can return the points in tf values or tf-idf values

Index

• type FloatMap
  • func (fmap FloatMap) Print()
• type TF_IDF
  • func (tf_idf *TF_IDF) AddToWordSet(corpus []WordSet)
  • func (tf_idf *TF_IDF) ComputeIDF()
  • func (tf_idf *TF_IDF) ComputeTF(corpus WordSet)
  • func (tf_idf *TF_IDF) ComputeTFIDF()
  • func (tf_idf *TF_IDF) GetAllPointsTF() []kmeans.Point
  • func (tf_idf *TF_IDF) GetAllPointsTFIDF() []kmeans.Point
  • func (tf_idf *TF_IDF) GetComputedIDF(countMap WordCounts) FloatMap
  • func (tf_idf *TF_IDF) GetComputedTF(corpus WordSet, counts WordCounts) FloatMap
  • func (tf_idf *TF_IDF) GetComputedTFIDF(tf FloatMap, idf FloatMap) FloatMap
  • func (tf_idf *TF_IDF) GetIDF() FloatMap
  • func (tf_idf *TF_IDF) GetPointByIndexTF(idx int) kmeans.Point
  • func (tf_idf *TF_IDF) GetPointByIndexTFIDF(idx int) kmeans.Point
  • func (tf_idf *TF_IDF) GetSetCount(corpus WordSet) WordCounts
  • func (tf_idf *TF_IDF) SetCount(corpus WordSet)
  • func (tf_idf *TF_IDF) SetCountIdx(corpus WordSet, idx int)
  • func (tf_idf *TF_IDF) SortMap()
• type WordCounts
  • func InitCounts(set WordSet) WordCounts
  • func (fmap WordCounts) Print()
• type WordSet
  • func (slice WordSet) Exists(word string) bool
  • func (slice WordSet) Print()
  • func (slice WordSet) ToLower() WordSet

Types

type FloatMap

type FloatMap map[string]float64

FloatMap - Dictionary of word by float for tf or idf

func (FloatMap) Print

func (fmap FloatMap) Print()

Print - prints the float values of the map

type TF_IDF

type TF_IDF struct {
	//List of words
	SetWord WordSet
	//Count of each word by set
	WordCountList []WordCounts
	//Term-Frequency
	Tf []FloatMap
	//Inverse Document Frequency
	Idf FloatMap
	//Term-Frequency Inverse Document Frequency
	TfIdf []FloatMap
}

TF_IDF - Struct for calculation TF-IDF

func (*TF_IDF) AddToWordSet

func (tf_idf *TF_IDF) AddToWordSet(corpus []WordSet)

AddToWordSet - adds the set to the list of sets

func (*TF_IDF) ComputeIDF

func (tf_idf *TF_IDF) ComputeIDF()

ComputeIDF - computes the inverse document frequency for the list of sets

func (*TF_IDF) ComputeTF

func (tf_idf *TF_IDF) ComputeTF(corpus WordSet)

ComputeTF - computes the term frequency for the word set

func (*TF_IDF) ComputeTFIDF

func (tf_idf *TF_IDF) ComputeTFIDF()

ComputeTFIDF - computes the tf-idf for the each set of words

func (*TF_IDF) GetAllPointsTF

func (tf_idf *TF_IDF) GetAllPointsTF() []kmeans.Point

GetAllPointsTF - gets all the points in the list

func (*TF_IDF) GetAllPointsTFIDF

func (tf_idf *TF_IDF) GetAllPointsTFIDF() []kmeans.Point

GetAllPointsTFIDF - gets all the points in the list

func (*TF_IDF) GetComputedIDF

func (tf_idf *TF_IDF) GetComputedIDF(countMap WordCounts) FloatMap

GetComputedIDF - computes the inverse document frequency for the list of sets

func (*TF_IDF) GetComputedTF

func (tf_idf *TF_IDF) GetComputedTF(corpus WordSet, counts WordCounts) FloatMap

GetComputedTF - computes the term frequency for the word set

func (*TF_IDF) GetComputedTFIDF

func (tf_idf *TF_IDF) GetComputedTFIDF(tf FloatMap, idf FloatMap) FloatMap

GetComputedTFIDF - computes the tf-idf for the each set of words

func (*TF_IDF) GetIDF

func (tf_idf *TF_IDF) GetIDF() FloatMap

GetIDF - gets all the points in the list

func (*TF_IDF) GetPointByIndexTF

func (tf_idf *TF_IDF) GetPointByIndexTF(idx int) kmeans.Point

GetPointByIndexTF - gets tf map of the point in the indes

func (*TF_IDF) GetPointByIndexTFIDF

func (tf_idf *TF_IDF) GetPointByIndexTFIDF(idx int) kmeans.Point

GetPointByIndexTFIDF - gets tfIdf map of the point in the index

func (*TF_IDF) GetSetCount

func (tf_idf *TF_IDF) GetSetCount(corpus WordSet) WordCounts

GetSetCount - returns count of words for corresponding set

func (*TF_IDF) SetCount

func (tf_idf *TF_IDF) SetCount(corpus WordSet)

SetCount - sets the counts of words for the set

func (*TF_IDF) SetCountIdx

func (tf_idf *TF_IDF) SetCountIdx(corpus WordSet, idx int)

SetCountIdx - sets the counts of words for the set in that idx

func (*TF_IDF) SortMap

func (tf_idf *TF_IDF) SortMap()

SortMap - sorts the set by key values

type WordCounts

type WordCounts map[string]int

WordCounts - Dictionary of word by count

func InitCounts

func InitCounts(set WordSet) WordCounts

InitCounts - returns a map of WordCounts with the set of words

func (WordCounts) Print

func (fmap WordCounts) Print()

Print - Prints the counts of the words

type WordSet

type WordSet []string

WordSet - Set of Words

func (WordSet) Exists

func (slice WordSet) Exists(word string) bool

Exists - checks if a word exists in a set

func (WordSet) Print

func (slice WordSet) Print()

Print - prints the set of words

func (WordSet) ToLower

func (slice WordSet) ToLower() WordSet

ToLower - returns the words in the set in lower case