cnsimhash

README

cnsimhash

Based on jieba cut words, support the Chinese simhash

Documentation

Index

• Constants
• func Compare(a uint64, b uint64) uint8
• func Distance(v1 uint64, v2 uint64) int
• func IDFPrint()
• func LoadDictionary(jiebapath, idfpath, stopwords, synonympath string) error
• func UnicodeSimhash(s string, topN int) (uint64, analyse.Segments, []string)

Constants

const (
	WORDS_ALL = -1
)

Functions

func Compare

func Compare(a uint64, b uint64) uint8

Compare calculates the Hamming distance between two 64-bit integers

Currently, this is calculated using the Kernighan method [1]. Other methods exist which may be more efficient and are worth exploring at some point

func Distance

func Distance(v1 uint64, v2 uint64) int

func IDFPrint

func IDFPrint()

func LoadDictionary

func LoadDictionary(jiebapath, idfpath, stopwords, synonympath string) error

To Load jieba, idf, stop words dictionaries

func UnicodeSimhash

func UnicodeSimhash(s string, topN int) (uint64, analyse.Segments, []string)

calculate unicode simhash with top n keywords calculate with all words if topN < 0