icanhazwordz

README

icanhazwordz

does this string contain words?

import (
    "github.com/zricethezav/icanhazwordz"
    "fmt"
)
func main() {
    searcher := icanhazwordz.NewSearcher(DefaultFilter)
    result := searcher.Find("The quick brown fox jumps over the lazy dog")
    for _, m := range result.AllMatches {
        fmt.Println(m.Word)
    }
}

Documentation

Overview

Have you ever asked yourself can i haz wordz? More specifically, does a string contain any english words? This package provides functionality to detect English words in a given text using, you guessed it, our favorite friend, ahocorasick.

Index

• Variables
• type Filter
• type Match
• type Result
• type Searcher
  • func NewSearcher(filter Filter) *Searcher
  • func (s *Searcher) Find(text string) Result
  • func (s *Searcher) GetWordCount() int
  • func (s *Searcher) GetWords() []string

Variables

var DefaultFilter = Filter{MinLength: 2, MaxLength: 0}

Default filter ignores single character words. If you want to include them, set MinLength to 1.

Types

type Filter

type Filter struct {
	MinLength   int // Minimum word length (inclusive)
	MaxLength   int // Maximum word length (inclusive, 0 means no limit)
	ExactLength int // Exact word length (0 means no exact match required)
}

Filter defines criteria for filtering words

type Match

type Match struct {
	Word     string `json:"word"`
	StartPos int    `json:"start_pos"`
	EndPos   int    `json:"end_pos"`
}

Match represents a found word match

type Result

type Result struct {
	WordCount   int      `json:"word_count"`
	UniqueWords []string `json:"unique_words"`
	Matches     []Match  `json:"matches"`
}

Result represents the result of English word detection

type Searcher

type Searcher struct {
	// contains filtered or unexported fields
}

Searcher provides nltk English word detection with configurable filters

func NewSearcher

func NewSearcher(filter Filter) *Searcher

NewSearcher creates a new searcher with the specified word filters

func (*Searcher) Find

func (s *Searcher) Find(text string) Result

Find analyzes text using this searcher's word list

func (*Searcher) GetWordCount

func (s *Searcher) GetWordCount() int

GetWordCount returns the number of words in this searcher's dictionary

func (*Searcher) GetWords

func (s *Searcher) GetWords() []string

GetWords returns all words in this searcher's dictionary