tfidf

README

TF_IDF-Go

Implementation of TF_IDF in golang

Documentation

Overview

Package tfidf Provides a simple implementation of Term-Frequency Inverse-Document-Frequency in golang Can return the points in tf values or tf-idf values

Index

• type FloatMap
  • func (fmap FloatMap) Print()
• type TF_IDF
  • func (tf_idf *TF_IDF) AddToWordSet(corpus []WordSet)
  • func (tf_idf *TF_IDF) ComputeIDF()
  • func (tf_idf *TF_IDF) ComputeTF(corpus WordSet)
  • func (tf_idf *TF_IDF) GetAllPointsTFIDF() []kmeans.Point
  • func (tf_idf *TF_IDF) GetIDF() FloatMap
  • func (tf_idf *TF_IDF) GetPointByIndexTF(idx int) kmeans.Point
  • func (tf_idf *TF_IDF) GetPointByIndexTFIDF(idx int) kmeans.Point
  • func (tf_idf *TF_IDF) SetCount(corpus WordSet)
  • func (tf_idf *TF_IDF) SortMap()
• type WordCounts
  • func InitCounts(set WordSet) WordCounts
  • func (fmap WordCounts) Print()
• type WordSet
  • func (slice WordSet) Exists(word string) bool
  • func (slice WordSet) Print()
  • func (slice WordSet) ToLower() WordSet

Types

type FloatMap

type FloatMap map[string]float64

FloatMap - Dictionary of word by float for tf or idf

func (FloatMap) Print

func (fmap FloatMap) Print()

Print - prints the float values of the map

type TF_IDF

type TF_IDF struct {
	//List of words
	SetWord WordSet
	//Count of each word by set
	WordCountList []WordCounts
	//Term-Frequency
	Tf []FloatMap
	//Inverse Document Frequency
	Idf FloatMap
	//Term-Frequency Inverse Document Frequency
	TfIdf []FloatMap
}

TF_IDF - Struct for calculation TF-IDF

func (*TF_IDF) AddToWordSet

func (tf_idf *TF_IDF) AddToWordSet(corpus []WordSet)

AddToWordSet - adds the set to the list of sets

func (*TF_IDF) ComputeIDF

func (tf_idf *TF_IDF) ComputeIDF()

ComputeIDF - computes the inverse document frequency for the list of sets

func (*TF_IDF) ComputeTF

func (tf_idf *TF_IDF) ComputeTF(corpus WordSet)

ComputeTF - computes the term frequency for the word set

func (*TF_IDF) GetAllPointsTFIDF

func (tf_idf *TF_IDF) GetAllPointsTFIDF() []kmeans.Point

GetAllPointsTFIDF - gets all the points in the list

func (*TF_IDF) GetIDF

func (tf_idf *TF_IDF) GetIDF() FloatMap

GetIDF - gets all the points in the list

func (*TF_IDF) GetPointByIndexTF

func (tf_idf *TF_IDF) GetPointByIndexTF(idx int) kmeans.Point

GetPointByIndexTF - gets tf map of the point in the indes

func (*TF_IDF) GetPointByIndexTFIDF

func (tf_idf *TF_IDF) GetPointByIndexTFIDF(idx int) kmeans.Point

GetPointByIndexTFIDF - gets tf map of the point in the indes

func (*TF_IDF) SetCount

func (tf_idf *TF_IDF) SetCount(corpus WordSet)

SetCount - sets the counts of words for the set

func (*TF_IDF) SortMap

func (tf_idf *TF_IDF) SortMap()

SortMap - sorts the set by key values

type WordCounts

type WordCounts map[string]int

WordCounts - Dictionary of word by count

func InitCounts

func InitCounts(set WordSet) WordCounts

InitCounts - returns a map of WordCounts with the set of words

func (WordCounts) Print

func (fmap WordCounts) Print()

Print - Prints the counts of the words

type WordSet

type WordSet []string

WordSet - Set of Words

func (WordSet) Exists

func (slice WordSet) Exists(word string) bool

Exists - checks if a word exists in a set

func (WordSet) Print

func (slice WordSet) Print()

Print - prints the set of words

func (WordSet) ToLower

func (slice WordSet) ToLower() WordSet

ToLower - returns the words in the set in lower case