README
¶
Stores
Stores can be passed to the ngrams.NewIndex function to change the data storage mechanism. More details can be found in the ngrams README.
Memory Only store (default)
m := NewMemoryStore()
New stores can be created by satisfying the stores.Store interface.
Documentation
¶
Index ¶
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
This section is empty.
Types ¶
type Grams ¶
type Grams map[string]Variations
Grams is a map of Variations keyed on gram-key (eg. "to be"). This is primarily used by the in-memory store, but can also be used to structure data for other storage engines. Fun fact; the `gram` in n-gram derives from the Latin `gramma`, which is something that is written or drawn (such as a letter or number).
type MemoryStore ¶
type MemoryStore struct {
// sync implements a mutex for concurrent read/write.
sync.RWMutex
// contains filtered or unexported fields
}
MemoryStore is an in-memory ngram store. It complies with Store interface.
func (*MemoryStore) Add ¶
func (s *MemoryStore) Add(key, future string) error
Add adds an ngram to the store.
func (*MemoryStore) Any ¶
func (s *MemoryStore) Any() (k string, v Variations, err error)
Any returns a random ngram from the store.
func (*MemoryStore) Close ¶
func (s *MemoryStore) Close() error
Close gracefully disconnects the store. Because this is just in-memory, it will do nothing and return no errors.
func (*MemoryStore) Delete ¶
func (s *MemoryStore) Delete(key string) error
Delete removes an ngram to the store.
func (*MemoryStore) Get ¶
func (s *MemoryStore) Get(key string) (ok bool, v Variations)
Get gets an ngram variation from the store.
type Store ¶
type Store interface {
// Add adds a new ngram and key-variation pair to the index. It
// should take the key, which will be used to index the gram, and
// the future, which is a slice of consequent gram tokens.
// I like to add labels to my interface method inputs because it
// makes the context a little easier to understand for whoever
// follows my footsteps (sometimes that's future-me).
Add(key, future string) error
// Get returns the potential ngram variations for a key.
Get(key string) (bool, Variations)
// Delete removes an ngram key and all variations from the index.
Delete(key string) error
// Any returns a random ngram from the store.
Any() (string, Variations, error)
// Close is used to gracefully shutdown any connections.
Close() error
}
Store is a data storage mechanism for ngrams.
func NewMemoryStore ¶
func NewMemoryStore() Store
NewMemoryStore returns an in-memory ngram store. Ngrams added to the store are not persisted when the service restarts.
type Variations ¶
Variations contains the future-sequenced ngrams and the number of times they were indexed, keyed on gram-variation (eg. {"be or":3})
func (*Variations) NextWeightedRand ¶
func (v *Variations) NextWeightedRand() string
NextWeightedRand returns a random variation, probability-weighted by the number of times it was indexed. Using a linear scan ala https://blog.bruce-hill.com/a-faster-weighted-random-choice
Source Files