wordvectors

Documentation

Index

• Constants
• type Config
• type VectorMap
  • func NewVectorMap(config *Config) (*VectorMap, error)
  • func NewVectorMapFromFile(name string) (*VectorMap, error)
  • func (m *VectorMap) AverageVectors(vecs [][]float64) []float64
  • func (m *VectorMap) FlattenVectors(vecs [][]float64) []float64
  • func (m *VectorMap) PadVectors(vecs [][]float64, length int) [][]float64
  • func (m *VectorMap) SaveToFile(name string) error
  • func (m *VectorMap) SumVectors(vecs [][]float64) []float64
  • func (m *VectorMap) Vector(word string) (vector []float64, inVocabulary bool)
  • func (m *VectorMap) Vectors(words []string) [][]float64

Constants

const WordVectorsFile = "wv.gob"

Types

type Config

type Config struct {
	// WordVectorsFile is the path to the word vectors file
	WordVectorsFile string `mapstructure:"file_name"`

	// Truncate is a number between 0 and 1, which represents how many
	// words will be used from the word vector
	Truncate float32 `mapstructure:"truncate"`

	// SkipOOV makes the out-of-vocabulary words to be omitted
	SkipOOV bool `mapstructure:"skip_oov"`
}

Config contains configuration for fasttext word vectors

type VectorMap

type VectorMap struct {
	Map map[string][]float64
	// contains filtered or unexported fields
}

VectorMap contains a map of words and their vector, as well as the vector size

func NewVectorMap

func NewVectorMap(config *Config) (*VectorMap, error)

NewVectorMap loads word vector from a file, up to trunc percentage of words and returns a new VectorMap

func NewVectorMapFromFile

func NewVectorMapFromFile(name string) (*VectorMap, error)

func (*VectorMap) AverageVectors

func (m *VectorMap) AverageVectors(vecs [][]float64) []float64

AverageVectors averages an array of vector

func (*VectorMap) FlattenVectors

func (m *VectorMap) FlattenVectors(vecs [][]float64) []float64

FlattenVectors converts an array of vector into an array of vector, one after the other

func (*VectorMap) PadVectors

func (m *VectorMap) PadVectors(vecs [][]float64, length int) [][]float64

PadVectors pads an vector array to a length with zero vectors

func (*VectorMap) SaveToFile

func (m *VectorMap) SaveToFile(name string) error

SaveToFile saves a serializableVectorMap to the wordVectorsFile

func (*VectorMap) SumVectors

func (m *VectorMap) SumVectors(vecs [][]float64) []float64

SumVectors sums an array of vector

func (*VectorMap) Vector

func (m *VectorMap) Vector(word string) (vector []float64, inVocabulary bool)

Vector returns the vector for a certain word If the word is not found, a vector of zeros is returned

func (*VectorMap) Vectors

func (m *VectorMap) Vectors(words []string) [][]float64

Vectors returns the vector for a slice of wordsd