fcompl

README

fcompl

A fast phrases completion library using trie tree. Also provide path compression for lower memory usage

Common stopwords in English will also be removed

Support language: English, Chinese

Usage

Insert Phrases by reader

s := "A ball in the ground\nThe bat in the sky\nThe ball hit his head\n"
rd := bufio.NewReader(strings.NewReader(s))
f := fcompl.New(rd, true)   // true if you want to get rid of stopwords

Insert Phrases by yourself

f := fcompl.NewTrieNode()
f.InsertPhrase(lineStr, id)

Find completion IDs

f.Find("Ball") // [0, 2]
f.Find("bat") // [1]

Save and Load

// path compress and save into a file
fcompl.Compress(f)
f.Save("./foo.bin") // default using gob to serialize struct into file

// Load
f, err := fcompl.Load("./foo.bin")
// ...

When complete Chinese phrases, you should find in its pinyin format like

// eg.Want to complete '我是'
f.find("wo shi")

You can also control stop word filter by

fcompl.SetStopFilter(false)

Test coverage

PASS
coverage: 94.4% of statements
ok  	github.com/yujiahaol68/fcompl	1.049s	coverage: 94.4% of statements

Feel free to star if this small package helps. Issue is also welcome

Documentation

Index

• func Compress(root *TrieNode)
• func SetLimit(l int)
• func SetStopFilter(b bool)
• type TrieNode
  • func Load(filePath string) (*TrieNode, error)
  • func New(rd *bufio.Reader, stopFilter bool) *TrieNode
  • func NewTrieNode() *TrieNode
  • func (root *TrieNode) Find(text string) []int
  • func (tn *TrieNode) InsertPhrase(phrase string, index int)
  • func (root *TrieNode) Save(filePath string) error

Functions

func Compress

func Compress(root *TrieNode)

Compress cut the node below that no more branches to reduce the capacity of the whole tree

func SetLimit

func SetLimit(l int)

func SetStopFilter

func SetStopFilter(b bool)

Types

type TrieNode

type TrieNode struct {
	Children  map[string]*TrieNode
	ContainID []int
}

func Load

func Load(filePath string) (*TrieNode, error)

Load unserialize root TrieNode from file and decode using gob

func New

func New(rd *bufio.Reader, stopFilter bool) *TrieNode

New return the Trie root

func NewTrieNode

func NewTrieNode() *TrieNode

func (*TrieNode) Find

func (root *TrieNode) Find(text string) []int

func (*TrieNode) InsertPhrase

func (tn *TrieNode) InsertPhrase(phrase string, index int)

func (*TrieNode) Save

func (root *TrieNode) Save(filePath string) error

Save use gob to serialize TrieNode struct into file, remember to compress whole tree before save it