Documentation ¶
Overview ¶
Package util contains useful shared functions
Index ¶
- func Check(err error)
- func UpperIndex(i int, l int) int
- func WithBayes(b bool) func(*Model) error
- func WithBayesThreshold(odds float64) func(*Model) error
- func WithLanguage(l lang.Language) func(*Model) error
- func WithResolverBatch(n int) func(*Model) error
- func WithResolverURL(url string) func(*Model) error
- func WithResolverVerification(f bool) func(*Model) error
- func WithResolverWorkers(n int) func(*Model) error
- type Model
- type Opt
- type Resolver
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
func UpperIndex ¶
UpperIndex takes an index of a token and length of the tokens slice and returns an upper index of what could be a slice of a name. We expect that that most of the names will fit into 5 words. Other cases would require more thorough algorithims that we can run later as plugins.
func WithBayes ¶
WithBayes is an option that forces running bayes name-finding even when the language is not supported by training sets.
func WithBayesThreshold ¶
WithBayesThreshold is an option for name finding, that sets new threshold for results from the Bayes name-finding. All the name candidates that have a higher threshold will appear in the resulting names output.
func WithLanguage ¶
WithLanguage option forces a specific language to be associated with a text.
func WithResolverBatch ¶
WithResolverBatch option sets the batch size of name-strings to send to the resolution service.
func WithResolverURL ¶
WithResolverURL option sets a new url for name resolution service.
func WithResolverWorkers ¶
WithResolverWorkers option sets the number of workers to process name-resolution jobs.
Types ¶
type Model ¶
type Model struct { // Language of the text Language lang.Language // Bayes flag forces to run Bayes name-finding on unknown languages Bayes bool // BayesOddsThreshold sets the limit of posterior odds. Everything bigger // that this limit will go to the names output. BayesOddsThreshold float64 // TextOdds captures "concentration" of names as it is found for the whole // text by heuristic name-finding. It should be close enough for real // number of names in text. We use it when we do not have local conentration // of names in a region of text. TextOdds bayes.LabelFreq // NameDistribution keeps data about position of names candidates and // their value according to heuristic and Bayes name-finding algorithms. // NameDistribution // ResolverConf Resolver }
Model keeps configuration variables