contextionary

package
v0.0.0-...-8832f83 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Mar 7, 2019 License: BSD-3-Clause Imports: 9 Imported by: 0

Documentation

Overview 

                         _       _
*__      _____  __ ___   ___  __ _| |_ ___
*\ \ /\ / / _ \/ _` \ \ / / |/ _` | __/ _ \
* \ V  V /  __/ (_| |\ V /| | (_| | ||  __/
*  \_/\_/ \___|\__,_| \_/ |_|\__,_|\__\___|
*
* Copyright © 2016 - 2019 Weaviate. All rights reserved.
* LICENSE: https://github.com/creativesoftwarefdn/weaviate/blob/develop/LICENSE.md
* DESIGN & CONCEPT: Bob van Luijt (@bobvanluijt)
* CONTACT: hello@creativesoftwarefdn.org

                         _       _
*__      _____  __ ___   ___  __ _| |_ ___
*\ \ /\ / / _ \/ _` \ \ / / |/ _` | __/ _ \
* \ V  V /  __/ (_| |\ V /| | (_| | ||  __/
*  \_/\_/ \___|\__,_| \_/ |_|\__,_|\__\___|
*
* Copyright © 2016 - 2019 Weaviate. All rights reserved.
* LICENSE: https://github.com/creativesoftwarefdn/weaviate/blob/develop/LICENSE.md
* DESIGN & CONCEPT: Bob van Luijt (@bobvanluijt)
* CONTACT: hello@creativesoftwarefdn.org

Package contextionary provides the toolset to add context to words. 

                         _       _
*__      _____  __ ___   ___  __ _| |_ ___
*\ \ /\ / / _ \/ _` \ \ / / |/ _` | __/ _ \
* \ V  V /  __/ (_| |\ V /| | (_| | ||  __/
*  \_/\_/ \___|\__,_| \_/ |_|\__,_|\__\___|
*
* Copyright © 2016 - 2019 Weaviate. All rights reserved.
* LICENSE: https://github.com/creativesoftwarefdn/weaviate/blob/develop/LICENSE.md
* DESIGN & CONCEPT: Bob van Luijt (@bobvanluijt)
* CONTACT: hello@creativesoftwarefdn.org

                         _       _
*__      _____  __ ___   ___  __ _| |_ ___
*\ \ /\ / / _ \/ _` \ \ / / |/ _` | __/ _ \
* \ V  V /  __/ (_| |\ V /| | (_| | ||  __/
*  \_/\_/ \___|\__,_| \_/ |_|\__,_|\__\___|
*
* Copyright © 2016 - 2019 Weaviate. All rights reserved.
* LICENSE: https://github.com/creativesoftwarefdn/weaviate/blob/develop/LICENSE.md
* DESIGN & CONCEPT: Bob van Luijt (@bobvanluijt)
* CONTACT: hello@creativesoftwarefdn.org

                         _       _
*__      _____  __ ___   ___  __ _| |_ ___
*\ \ /\ / / _ \/ _` \ \ / / |/ _` | __/ _ \
* \ V  V /  __/ (_| |\ V /| | (_| | ||  __/
*  \_/\_/ \___|\__,_| \_/ |_|\__,_|\__\___|
*
* Copyright © 2016 - 2019 Weaviate. All rights reserved.
* LICENSE: https://github.com/creativesoftwarefdn/weaviate/blob/develop/LICENSE.md
* DESIGN & CONCEPT: Bob van Luijt (@bobvanluijt)
* CONTACT: hello@creativesoftwarefdn.org

                         _       _
*__      _____  __ ___   ___  __ _| |_ ___
*\ \ /\ / / _ \/ _` \ \ / / |/ _` | __/ _ \
* \ V  V /  __/ (_| |\ V /| | (_| | ||  __/
*  \_/\_/ \___|\__,_| \_/ |_|\__,_|\__\___|
*
* Copyright © 2016 - 2019 Weaviate. All rights reserved.
* LICENSE: https://github.com/creativesoftwarefdn/weaviate/blob/develop/LICENSE.md
* DESIGN & CONCEPT: Bob van Luijt (@bobvanluijt)
* CONTACT: hello@creativesoftwarefdn.org

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

This section is empty.

Types

type CombinedIndex 

type CombinedIndex struct {
	// contains filtered or unexported fields
}

func CombineVectorIndices 

func CombineVectorIndices(indices []Contextionary) (*CombinedIndex, error)

Combine multiple indices, present them as one. It assumes that each index stores unique words

func (*CombinedIndex) GetDistance 

func (ci *CombinedIndex) GetDistance(a ItemIndex, b ItemIndex) (float32, error)

Compute the distance between two items.

func (*CombinedIndex) GetNnsByItem 

func (ci *CombinedIndex) GetNnsByItem(item ItemIndex, n int, k int) ([]ItemIndex, []float32, error)

Get the n nearest neighbours of item, examining k trees. Returns an array of indices, and of distances between item and the n-nearest neighbors.

func (*CombinedIndex) GetNnsByVector 

func (ci *CombinedIndex) GetNnsByVector(vector Vector, n int, k int) ([]ItemIndex, []float32, error)

Get the n nearest neighbours of item, examining k trees. Returns an array of indices, and of distances between item and the n-nearest neighbors.

func (*CombinedIndex) GetNumberOfItems 

func (ci *CombinedIndex) GetNumberOfItems() int

func (*CombinedIndex) GetVectorForItemIndex 

func (ci *CombinedIndex) GetVectorForItemIndex(item ItemIndex) (*Vector, error)

func (*CombinedIndex) GetVectorLength 

func (ci *CombinedIndex) GetVectorLength() int

func (*CombinedIndex) ItemIndexToWord 

func (ci *CombinedIndex) ItemIndexToWord(item ItemIndex) (string, error)

func (*CombinedIndex) VerifyDisjoint 

func (ci *CombinedIndex) VerifyDisjoint() error

Verify that all the indices are disjoint Returns nil on success, an error if the words in the indices are not disjoint.

func (*CombinedIndex) WordToItemIndex 

func (ci *CombinedIndex) WordToItemIndex(word string) ItemIndex

type Contextionary 

type Contextionary interface {
	// Return the number of items that is stored in the index.
	GetNumberOfItems() int

	// Returns the length of the used vectors.
	GetVectorLength() int

	// Look up a word, return an index.
	// Check for presence of the index with index.IsPresent()
	WordToItemIndex(word string) ItemIndex

	// Based on an index, return the assosiated word.
	ItemIndexToWord(item ItemIndex) (string, error)

	// Get the vector of an item index.
	GetVectorForItemIndex(item ItemIndex) (*Vector, error)

	// Compute the distance between two items.
	GetDistance(a ItemIndex, b ItemIndex) (float32, error)

	// Get the n nearest neighbours of item, examining k trees.
	// Returns an array of indices, and of distances between item and the n-nearest neighbors.
	GetNnsByItem(item ItemIndex, n int, k int) ([]ItemIndex, []float32, error)

	// Get the n nearest neighbours of item, examining k trees.
	// Returns an array of indices, and of distances between item and the n-nearest neighbors.
	GetNnsByVector(vector Vector, n int, k int) ([]ItemIndex, []float32, error)
}

Contextionary is the API to decouple the K-nn interface that is needed for Weaviate from a concrete implementation.

func LoadVectorFromDisk 

func LoadVectorFromDisk(annoy_index string, word_index_file_name string) (Contextionary, error)

type ItemIndex 

type ItemIndex int

ItemIndex is an opaque type that models an index number used to identify a word.

func (*ItemIndex) IsPresent 

func (i *ItemIndex) IsPresent() bool

IsPresent can be used after retrieving a word index (which does not error on its own), to see if the word was actually present in the contextionary.

type MemoryIndex 

type MemoryIndex struct {
	// contains filtered or unexported fields
}

func (MemoryIndex) GetDistance 

func (mi MemoryIndex) GetDistance(a ItemIndex, b ItemIndex) (float32, error)

Compute the distance between two items.

func (*MemoryIndex) GetNnsByItem 

func (mi *MemoryIndex) GetNnsByItem(item ItemIndex, n int, k int) ([]ItemIndex, []float32, error)

Get the n nearest neighbours of item, examining k trees. Returns an array of indices, and of distances between item and the n-nearest neighbors.

func (*MemoryIndex) GetNnsByVector 

func (mi *MemoryIndex) GetNnsByVector(vector Vector, n int, k int) ([]ItemIndex, []float32, error)

Get the n nearest neighbours of item, examining k trees. Returns an array of indices, and of distances between item and the n-nearest neighbors.

func (*MemoryIndex) GetNumberOfItems 

func (mi *MemoryIndex) GetNumberOfItems() int

Return the number of items that is stored in the index.

func (*MemoryIndex) GetVectorForItemIndex 

func (mi *MemoryIndex) GetVectorForItemIndex(item ItemIndex) (*Vector, error)

Get the vector of an item index.

func (*MemoryIndex) GetVectorLength 

func (mi *MemoryIndex) GetVectorLength() int

Returns the length of the used vectors.

func (*MemoryIndex) ItemIndexToWord 

func (mi *MemoryIndex) ItemIndexToWord(item ItemIndex) (string, error)

Based on an index, return the assosiated word.

func (*MemoryIndex) WordToItemIndex 

func (mi *MemoryIndex) WordToItemIndex(word string) ItemIndex

Look up a word, return an index. Perform binary search.

type MemoryIndexBuilder 

type MemoryIndexBuilder struct {
	// contains filtered or unexported fields
}

func InMemoryBuilder 

func InMemoryBuilder(dimensions int) *MemoryIndexBuilder

Construct a new builder.

func (*MemoryIndexBuilder) AddWord 

func (mib *MemoryIndexBuilder) AddWord(word string, vector Vector)

Add a word and it's vector to the builder.

func (*MemoryIndexBuilder) Build 

func (mib *MemoryIndexBuilder) Build(trees int) *MemoryIndex

Build an efficient lookup iddex from the builder.

type Vector 

type Vector struct {
	// contains filtered or unexported fields
}

Opque type that models a fixed-length vector.

func ComputeCentroid 

func ComputeCentroid(vectors []Vector) (*Vector, error)

func ComputeWeightedCentroid 

func ComputeWeightedCentroid(vectors []Vector, weights []float32) (*Vector, error)

func NewVector 

func NewVector(vector []float32) Vector

func (*Vector) Distance 

func (v *Vector) Distance(other *Vector) (float32, error)

func (*Vector) Equal 

func (v *Vector) Equal(other *Vector) (bool, error)

func (*Vector) EqualEpsilon 

func (v *Vector) EqualEpsilon(other *Vector, epsilon float32) (bool, error)

func (*Vector) Len 

func (v *Vector) Len() int

func (*Vector) ToString 

func (v *Vector) ToString() string

type Wordlist 

type Wordlist struct {
	// contains filtered or unexported fields
}

func LoadWordlist 

func LoadWordlist(path string) (*Wordlist, error)

func (*Wordlist) FindIndexByWord 

func (w *Wordlist) FindIndexByWord(_needle string) ItemIndex

func (*Wordlist) GetNumberOfWords 

func (w *Wordlist) GetNumberOfWords() ItemIndex

Source Files

View all Source files

Directories

Path Synopsis
_ _ *__ _____ __ ___ ___ __ _| |_ ___ *\ \ /\ / / _ \/ _` \ \ / / |/ _` | __/ _ \ * \ V V / __/ (_| |\ V /| | (_| | || __/ * \_/\_/ \___|\__,_| \_/ |_|\__,_|\__\___| * * Copyright © 2016 - 2019 Weaviate.
 _ _ *__ _____ __ ___ ___ __ _| |_ ___ *\ \ /\ / / _ \/ _` \ \ / / |/ _` | __/ _ \ * \ V V / __/ (_| |\ V /| | (_| | || __/ * \_/\_/ \___|\__,_| \_/ |_|\__,_|\__\___| * * Copyright © 2016 - 2019 Weaviate.
cmd
_ _ *__ _____ __ ___ ___ __ _| |_ ___ *\ \ /\ / / _ \/ _` \ \ / / |/ _` | __/ _ \ * \ V V / __/ (_| |\ V /| | (_| | || __/ * \_/\_/ \___|\__,_| \_/ |_|\__,_|\__\___| * * Copyright © 2016 - 2019 Weaviate.
 _ _ *__ _____ __ ___ ___ __ _| |_ ___ *\ \ /\ / / _ \/ _` \ \ / / |/ _` | __/ _ \ * \ V V / __/ (_| |\ V /| | (_| | || __/ * \_/\_/ \___|\__,_| \_/ |_|\__,_|\__\___| * * Copyright © 2016 - 2019 Weaviate.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.