util

Documentation

Overview

Package util contains useful shared functions

Index

• func Check(err error)
• func UpperIndex(i int, l int) int
• func WithBayes(b bool) func(*Model) error
• func WithBayesThreshold(odds float64) func(*Model) error
• func WithLanguage(l lang.Language) func(*Model) error
• func WithResolverBatch(n int) func(*Model) error
• func WithResolverURL(url string) func(*Model) error
• func WithResolverWorkers(n int) func(*Model) error
• func WithSources(s []int) func(*Model) error
• func WithVerification(v bool) func(*Model) error
• type Model
  • func NewModel(opts ...Opt) *Model
• type Opt
• type Resolver

Functions

func Check

func Check(err error)

Check for 'boring' errors, and panic if error is not nil.

func UpperIndex

func UpperIndex(i int, l int) int

UpperIndex takes an index of a token and length of the tokens slice and returns an upper index of what could be a slice of a name. We expect that that most of the names will fit into 5 words. Other cases would require more thorough algorithims that we can run later as plugins.

func WithBayes

func WithBayes(b bool) func(*Model) error

WithBayes is an option that forces running bayes name-finding even when the language is not supported by training sets.

func WithBayesThreshold

func WithBayesThreshold(odds float64) func(*Model) error

WithBayesThreshold is an option for name finding, that sets new threshold for results from the Bayes name-finding. All the name candidates that have a higher threshold will appear in the resulting names output.

func WithLanguage

func WithLanguage(l lang.Language) func(*Model) error

WithLanguage option forces a specific language to be associated with a text.

func WithResolverBatch

func WithResolverBatch(n int) func(*Model) error

WithResolverBatch option sets the batch size of name-strings to send to the resolution service.

func WithResolverURL

func WithResolverURL(url string) func(*Model) error

WithResolverURL option sets a new url for name resolution service.

func WithResolverWorkers

func WithResolverWorkers(n int) func(*Model) error

WithResolverWorkers option sets the number of workers to process name-resolution jobs.

func WithSources

func WithSources(s []int) func(*Model) error

WithSources is an option that sets IDs of data sources used for verification. Results from these sources (if any) will be returned no matter what is the best matching result.

func WithVerification

func WithVerification(v bool) func(*Model) error

WithVerification is a flag that determines if names will be sent for validation to https://index.globalnames.org service.

Types

type Model

type Model struct {
	// Language of the text
	Language lang.Language
	// Bayes flag forces to run Bayes name-finding on unknown languages
	Bayes bool
	// BayesOddsThreshold sets the limit of posterior odds. Everything bigger
	// that this limit will go to the names output.
	BayesOddsThreshold float64
	// TextOdds captures "concentration" of names as it is found for the whole
	// text by heuristic name-finding. It should be close enough for real
	// number of names in text. We use it when we do not have local conentration
	// of names in a region of text.
	TextOdds bayes.LabelFreq
	// NameDistribution keeps data about position of names candidates and
	// their value according to heuristic and Bayes name-finding algorithms.
	// NameDistribution
	// ResolverConf
	Resolver
}

Model keeps configuration variables

func NewModel

func NewModel(opts ...Opt) *Model

NewModel creates Model object with default data, or with data coming from opts.

type Opt

type Opt func(g *Model) error

Opt are options for gnfinder's model

type Resolver

type Resolver struct {
	URL                string
	BatchSize          int
	Workers            int
	WaitTimeout        time.Duration
	Sources            []int
	Verify             bool
	AdvancedResolution bool
}

Resolver contains configuration of Resolver data