hlup

Documentation

Index

• func CompileExpression(rule string, optionsPatches ...options.PatchFunc) (expression.Expression, error)
• func TokenizeInput(input string, options ...tokenizeOptionsPatchFunc) expression.TokenStream
• func WithCustomReplaceTable(table map[rune]rune) tokenizeOptionsPatchFunc
• func WithCustomWordBreakersList(list []rune) tokenizeOptionsPatchFunc
• func WithExtendedReplaceTable(table map[rune]rune) tokenizeOptionsPatchFunc
• func WithExtendedWordBreakersList(list []rune) tokenizeOptionsPatchFunc

Functions

func CompileExpression

func CompileExpression(rule string, optionsPatches ...options.PatchFunc) (expression.Expression, error)

CompileExpression returns an expression that represents provided rule in specified language. NOTE: Default language is english. To change the language use options.WithLang

func TokenizeInput

func TokenizeInput(input string, options ...tokenizeOptionsPatchFunc) expression.TokenStream

TokenizeInput returns a token (word) sequence, dropping punctuation.

func WithCustomReplaceTable

func WithCustomReplaceTable(table map[rune]rune) tokenizeOptionsPatchFunc

WithCustomReplaceTable overrides rune replace lookup table with provided onces. Using this function is not recommended, consider using WithExtendedReplaceTable.

func WithCustomWordBreakersList

func WithCustomWordBreakersList(list []rune) tokenizeOptionsPatchFunc

WithCustomWordBreakersList overrides all known word breaker runes. Using this function is not recommended, consider using WithExtendedWordBreakersList.

func WithExtendedReplaceTable

func WithExtendedReplaceTable(table map[rune]rune) tokenizeOptionsPatchFunc

WithExtendedReplaceTable extends rune replace lookup table with provided onces.

func WithExtendedWordBreakersList

func WithExtendedWordBreakersList(list []rune) tokenizeOptionsPatchFunc

WithExtendedWordBreakersList extends word breaker runes set with list of provided ones.