jtfidf

package module
Version: v0.0.0-...-11b6f76 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Jul 9, 2019 License: MIT Imports: 2 Imported by: 0

README

jtfidf

GoDoc CircleCI Go Report Card codecov

Package jtfidf provides calculations of TF(Term Frequency), IDF(Inverse Document Frequency) and TF-IDF values at Japanese documents.

Dependencies

This package uses kagome as Morphological Analyzer.

About how to calulate TF-IDF value

The calculation of the TF-IDF value in this package uses the IDF value plus 1. This is to prevent the TF-IDF value from becoming 0.

Install

go get -u github.com/ramenjuniti/jtfidf

Usage

All usage are described in GoDoc.

AllTf

AllTf returns all TF values in a doucument.

func ExampleAllTf() {
	fmt.Println(AllTf("寿司が食べたい。"))
	// Output: map[。:0.2 が:0.2 たい:0.2 寿司:0.2 食べ:0.2]
}
Tf

Tf returns TF value in a document.

func ExampleTf() {
	fmt.Println(Tf("寿司", "寿司が食べたい。"))
	// Output: 0.2
}
AllIdf

AllIdf returns all IDF values in documents.

func ExampleAllIdf() {
	ds := []string{
		"寿司が食べたい。",
	}
	fmt.Println(AllIdf(ds))
	// Output: map[。:0 が:0 たい:0 寿司:0 食べ:0]
}
Idf

Idf retuns IDF value in documents.

func ExampleIdf() {
	ds := []string{
		"寿司が食べたい。",
	}
	fmt.Println(Idf("寿司", ds))
	// Output: 0
}
AllTfidf

AllTfidf retuns all TF-IDF values in documents.

func ExampleAllTfidf() {
	ds := []string{
		"寿司が食べたい。",
	}
	fmt.Println(AllTfidf(ds))
	// Output: [map[。:0.2 が:0.2 たい:0.2 寿司:0.2 食べ:0.2]]
}
Tfidf

Tfidf returns TF-IDF value in documents.

func ExampleTfidf() {
	ds := []string{
		"寿司が食べたい。",
	}
	fmt.Println(Tfidf("寿司", ds[0], ds))
	// Output: 0.2
}

License

This software is released under the MIT License, see LICENSE.

Documentation

Overview

Package jtfidf provides calculations of TF(Term Frequency), IDF(Inverse Document Frequency) and TF-IDF values at Japanese documents.

Dependencies

This package uses [kagome](https://github.com/ikawaha/kagome) as Morphological Analyzer.

About how to calulate TF-IDF value

The calculation of the TF-IDF value in this package uses the IDF value plus 1. This is to prevent the TF-IDF value from becoming 0.

Index

Examples

Constants

This section is empty.

Variables

This section is empty.

Functions

func AllIdf

func AllIdf(ds []string) map[string]float64

AllIdf returns all IDF values in ds

Example
ds := []string{
	"寿司が食べたい。",
}
fmt.Println(AllIdf(ds))
Output:

map[。:0 が:0 たい:0 寿司:0 食べ:0]

func AllTf

func AllTf(d string) map[string]float64

AllTf returns all TF values in d

Example
fmt.Println(AllTf("寿司が食べたい。"))
Output:

map[。:0.2 が:0.2 たい:0.2 寿司:0.2 食べ:0.2]

func AllTfidf

func AllTfidf(ds []string) []map[string]float64

AllTfidf retuns all TF-IDF values in ds

Example
ds := []string{
	"寿司が食べたい。",
}
fmt.Println(AllTfidf(ds))
Output:

[map[。:0.2 が:0.2 たい:0.2 寿司:0.2 食べ:0.2]]

func Idf

func Idf(t string, ds []string) float64

Idf retuns t's IDF value in ds

Example
ds := []string{
	"寿司が食べたい。",
}
fmt.Println(Idf("寿司", ds))
Output:

0

func Tf

func Tf(t, d string) float64

Tf returns t's TF value in d

Example
fmt.Println(Tf("寿司", "寿司が食べたい。"))
Output:

0.2

func Tfidf

func Tfidf(t, d string, ds []string) float64

Tfidf returns t's TF-IDF value in ds

Example
ds := []string{
	"寿司が食べたい。",
}
fmt.Println(Tfidf("寿司", ds[0], ds))
Output:

0.2

Types

This section is empty.

Source Files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
t or T : Toggle theme light dark auto