Documentation
¶
Overview ¶
Package metric provides various similarity / distance metrics for comparing tensors, operating on the tensor.Tensor standard data representation.
Index ¶
- func ClosestRow(fun any, probe, vocab tensor.Tensor) tensor.Values
- func ClosestRowOut(fun any, probe, vocab tensor.Tensor, out tensor.Values) error
- func Correlation(a, b tensor.Tensor) tensor.Values
- func CorrelationOut(a, b tensor.Tensor, out tensor.Values) error
- func CorrelationOut64(a, b tensor.Tensor, out tensor.Values) (*tensor.Float64, error)
- func Cosine(a, b tensor.Tensor) tensor.Values
- func CosineOut(a, b tensor.Tensor, out tensor.Values) error
- func CosineOut64(a, b tensor.Tensor, out tensor.Values) (*tensor.Float64, error)
- func Covariance(a, b tensor.Tensor) tensor.Values
- func CovarianceMatrix(fun any, in tensor.Tensor) tensor.Values
- func CovarianceMatrixOut(fun any, in tensor.Tensor, out tensor.Values) error
- func CovarianceOut(a, b tensor.Tensor, out tensor.Values) error
- func CrossEntropy(a, b tensor.Tensor) tensor.Values
- func CrossEntropyOut(a, b tensor.Tensor, out tensor.Values) error
- func CrossMatrix(fun any, a, b tensor.Tensor) tensor.Values
- func CrossMatrixOut(fun any, a, b tensor.Tensor, out tensor.Values) error
- func DotProduct(a, b tensor.Tensor) tensor.Values
- func DotProductOut(a, b tensor.Tensor, out tensor.Values) error
- func Hamming(a, b tensor.Tensor) tensor.Values
- func HammingOut(a, b tensor.Tensor, out tensor.Values) error
- func InvCorrelation(a, b tensor.Tensor) tensor.Values
- func InvCorrelationOut(a, b tensor.Tensor, out tensor.Values) error
- func InvCosine(a, b tensor.Tensor) tensor.Values
- func InvCosineOut(a, b tensor.Tensor, out tensor.Values) error
- func L1Norm(a, b tensor.Tensor) tensor.Values
- func L1NormOut(a, b tensor.Tensor, out tensor.Values) error
- func L2Norm(a, b tensor.Tensor) tensor.Values
- func L2NormBinTol(a, b tensor.Tensor) tensor.Values
- func L2NormBinTolOut(a, b tensor.Tensor, out tensor.Values) error
- func L2NormOut(a, b tensor.Tensor, out tensor.Values) error
- func Matrix(fun any, in tensor.Tensor) tensor.Values
- func MatrixOut(fun any, in tensor.Tensor, out tensor.Values) error
- func SumSquares(a, b tensor.Tensor) tensor.Values
- func SumSquaresBinTol(a, b tensor.Tensor) tensor.Values
- func SumSquaresBinTolOut(a, b tensor.Tensor, out tensor.Values) error
- func SumSquaresBinTolScaleOut64(a, b tensor.Tensor) (scale64, ss64 *tensor.Float64, err error)
- func SumSquaresOut(a, b tensor.Tensor, out tensor.Values) error
- func SumSquaresOut64(a, b tensor.Tensor, out tensor.Values) (*tensor.Float64, error)
- func SumSquaresScaleOut64(a, b tensor.Tensor) (scale64, ss64 *tensor.Float64, err error)
- func Vectorize2Out64(a, b tensor.Tensor, iniX, iniY float64, ...) (ox64, oy64 *tensor.Float64)
- func Vectorize3Out64(a, b tensor.Tensor, iniX, iniY, iniZ float64, ...) (ox64, oy64, oz64 *tensor.Float64)
- func VectorizeOut64(a, b tensor.Tensor, out tensor.Values, ini float64, ...) *tensor.Float64
- func VectorizePre3Out64(a, b tensor.Tensor, iniX, iniY, iniZ float64, preA, preB *tensor.Float64, ...) (ox64, oy64, oz64 *tensor.Float64)
- func VectorizePreOut64(a, b tensor.Tensor, out tensor.Values, ini float64, preA, preB *tensor.Float64, ...) *tensor.Float64
- type MetricFunc
- type MetricOutFunc
- type Metrics
- func (m Metrics) Call(a, b tensor.Tensor) tensor.Values
- func (i Metrics) Desc() string
- func (m Metrics) Func() MetricFunc
- func (m Metrics) FuncName() string
- func (m Metrics) Increasing() bool
- func (i Metrics) Int64() int64
- func (i Metrics) MarshalText() ([]byte, error)
- func (i *Metrics) SetInt64(in int64)
- func (i *Metrics) SetString(s string) error
- func (i Metrics) String() string
- func (i *Metrics) UnmarshalText(text []byte) error
- func (i Metrics) Values() []enums.Enum
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
func ClosestRow ¶
ClosestRow returns the closest fit between probe pattern and patterns in a "vocabulary" tensor with outermost row dimension, using given metric function, which must fit the MetricFunc signature. The metric *must have the Increasing property*, i.e., larger = further. Output is a 1D tensor with 2 elements: the row index and metric value for that row. Note: this does _not_ use any existing Indexes for the probe, but does for the vocab, and the returned index is the logical index into any existing Indexes.
func ClosestRowOut ¶
ClosestRowOut returns the closest fit between probe pattern and patterns in a "vocabulary" tensor with outermost row dimension, using given metric function, which must fit the MetricFunc signature. The metric *must have the Increasing property*, i.e., larger = further. Output is a 1D tensor with 2 elements: the row index and metric value for that row. Note: this does _not_ use any existing Indexes for the probe, but does for the vocab, and the returned index is the logical index into any existing Indexes.
func Correlation ¶
Correlation computes the correlation between two vectors, in range (-1..1) as the mean of the co-product of each vector element minus the mean of that vector, normalized by the product of their standard deviations: cor(A,B) = E[(A - E(A))(B - E(B))] / sigma(A) sigma(B). (i.e., the standardized Covariance). Equivalent to the Cosine of mean-normalized vectors. See MetricFunc for general information.
func CorrelationOut ¶
CorrelationOut computes the correlation between two vectors, in range (-1..1) as the mean of the co-product of each vector element minus the mean of that vector, normalized by the product of their standard deviations: cor(A,B) = E[(A - E(A))(B - E(B))] / sigma(A) sigma(B). (i.e., the standardized Covariance). Equivalent to the Cosine of mean-normalized vectors.
func CorrelationOut64 ¶
CorrelationOut64 computes the correlation between two vectors, in range (-1..1) as the mean of the co-product of each vector element minus the mean of that vector, normalized by the product of their standard deviations: cor(A,B) = E[(A - E(A))(B - E(B))] / sigma(A) sigma(B). (i.e., the standardized covariance). Equivalent to the cosine of mean-normalized vectors. Returns the Float64 output values for subsequent use.
func Cosine ¶
Cosine computes the high-dimensional angle between two vectors, in range (-1..1) as the normalized dot product: dot product / sqrt(ssA * ssB). See also Correlation See MetricFunc for general information.
func CosineOut ¶
CosineOut computes the high-dimensional angle between two vectors, in range (-1..1) as the normalized dot product: dot product / sqrt(ssA * ssB). See also Correlation
func CosineOut64 ¶
CosineOut64 computes the high-dimensional angle between two vectors, in range (-1..1) as the normalized [Dot]: dot product / sqrt(ssA * ssB). See also Correlation.
func Covariance ¶
Covariance computes the co-variance between two vectors, i.e., the mean of the co-product of each vector element minus the mean of that vector: cov(A,B) = E[(A - E(A))(B - E(B))]. See MetricFunc for general information.
func CovarianceMatrix ¶
CovarianceMatrix generates the cells x cells square covariance matrix for all per-row cells of the given higher dimensional input tensor, which must have at least 2 dimensions: the outermost rows, and within that, 1+dimensional patterns (cells). Each value in the resulting matrix represents the extent to which the value of a given cell covaries across the rows of the tensor with the value of another cell. Uses the given metric function, typically Covariance or Correlation, The metric function must have the MetricFunc signature. Use Covariance if vars have similar overall scaling, which is typical in neural network models, and use Correlation if they are on very different scales, because it effectively rescales). The resulting matrix can be used as the input to PCA or SVD eigenvalue decomposition.
func CovarianceMatrixOut ¶
CovarianceMatrixOut generates the cells x cells square covariance matrix for all per-row cells of the given higher dimensional input tensor, which must have at least 2 dimensions: the outermost rows, and within that, 1+dimensional patterns (cells). Each value in the resulting matrix represents the extent to which the value of a given cell covaries across the rows of the tensor with the value of another cell. Uses the given metric function, typically Covariance or Correlation, The metric function must have the MetricFunc signature. Use Covariance if vars have similar overall scaling, which is typical in neural network models, and use Correlation if they are on very different scales, because it effectively rescales). The resulting matrix can be used as the input to PCA or SVD eigenvalue decomposition.
func CovarianceOut ¶
CovarianceOut computes the co-variance between two vectors, i.e., the mean of the co-product of each vector element minus the mean of that vector: cov(A,B) = E[(A - E(A))(B - E(B))].
func CrossEntropy ¶
CrossEntropy is a standard measure of the difference between two probabilty distributions, reflecting the additional entropy (uncertainty) associated with measuring probabilities under distribution b when in fact they come from distribution a. It is also the entropy of a plus the divergence between a from b, using Kullback-Leibler (KL) divergence. It is computed as: a * log(a/b) + (1-a) * log(1-a/1-b). See MetricFunc for general information.
func CrossEntropyOut ¶
CrossEntropyOut is a standard measure of the difference between two probabilty distributions, reflecting the additional entropy (uncertainty) associated with measuring probabilities under distribution b when in fact they come from distribution a. It is also the entropy of a plus the divergence between a from b, using Kullback-Leibler (KL) divergence. It is computed as: a * log(a/b) + (1-a) * log(1-a/1-b). See MetricOutFunc for general information.
func CrossMatrix ¶
CrossMatrix computes the distance / similarity matrix between two different sets of patterns in the two input tensors, where the patterns are in the sub-space cells of the tensors, which must have at least 2 dimensions: the outermost rows, and within that, 1+dimensional patterns that the given distance metric function is applied to, with the results filling in the cells of the output matrix. The metric function must have the MetricFunc signature. The rows of the output matrix are the rows of the first input tensor, and the columns of the output are the rows of the second input tensor.
func CrossMatrixOut ¶
CrossMatrixOut computes the distance / similarity matrix between two different sets of patterns in the two input tensors, where the patterns are in the sub-space cells of the tensors, which must have at least 2 dimensions: the outermost rows, and within that, 1+dimensional patterns that the given distance metric function is applied to, with the results filling in the cells of the output matrix. The metric function must have the MetricFunc signature. The rows of the output matrix are the rows of the first input tensor, and the columns of the output are the rows of the second input tensor.
func DotProduct ¶
DotProductOut computes the sum of the element-wise products of the two tensors (aka the inner product). See MetricFunc for general information.
func DotProductOut ¶
DotProductOut computes the sum of the element-wise products of the two tensors (aka the inner product). See MetricOutFunc for general information.
func Hamming ¶
Hamming computes the sum of 1s for every element that is different, i.e., "city block" distance. See MetricFunc for general information.
func HammingOut ¶
HammingOut computes the sum of 1s for every element that is different, i.e., "city block" distance. See MetricOutFunc for general information.
func InvCorrelation ¶
InvCorrelation computes 1 minus the correlation between two vectors, in range (-1..1) as the mean of the co-product of each vector element minus the mean of that vector, normalized by the product of their standard deviations: cor(A,B) = E[(A - E(A))(B - E(B))] / sigma(A) sigma(B). (i.e., the standardized covariance). Equivalent to the Cosine of mean-normalized vectors. This is useful for a difference measure instead of similarity, where more different vectors have larger metric values. See MetricFunc for general information.
func InvCorrelationOut ¶
InvCorrelationOut computes 1 minus the correlation between two vectors, in range (-1..1) as the mean of the co-product of each vector element minus the mean of that vector, normalized by the product of their standard deviations: cor(A,B) = E[(A - E(A))(B - E(B))] / sigma(A) sigma(B). (i.e., the standardized covariance). Equivalent to the Cosine of mean-normalized vectors. This is useful for a difference measure instead of similarity, where more different vectors have larger metric values.
func InvCosine ¶
InvCosine computes 1 minus the cosine between two vectors, in range (-1..1) as the normalized dot product: dot product / sqrt(ssA * ssB). This is useful for a difference measure instead of similarity, where more different vectors have larger metric values. See MetricFunc for general information.
func InvCosineOut ¶
InvCosineOut computes 1 minus the cosine between two vectors, in range (-1..1) as the normalized dot product: dot product / sqrt(ssA * ssB). This is useful for a difference measure instead of similarity, where more different vectors have larger metric values.
func L1Norm ¶
L1Norm computes the sum of the absolute value of differences between the tensor values, the L1 Norm. See MetricFunc for general information.
func L1NormOut ¶
L1NormOut computes the sum of the absolute value of differences between the tensor values, the L1 Norm. See MetricOutFunc for general information.
func L2Norm ¶
L2Norm computes the L2 Norm: square root of the sum of squares differences between tensor values, aka the Euclidean distance. See MetricFunc for general information.
func L2NormBinTol ¶
L2NormBinTol computes the L2 Norm square root of the sum of squares differences between tensor values (aka Euclidean distance), with binary tolerance: differences < 0.5 are thresholded to 0. See MetricFunc for general information.
func L2NormBinTolOut ¶
L2NormBinTolOut computes the L2 Norm square root of the sum of squares differences between tensor values (aka Euclidean distance), with binary tolerance: differences < 0.5 are thresholded to 0.
func L2NormOut ¶
L2NormOut computes the L2 Norm: square root of the sum of squares differences between tensor values, aka the Euclidean distance.
func Matrix ¶
Matrix computes the rows x rows square distance / similarity matrix between the patterns for each row of the given higher dimensional input tensor, which must have at least 2 dimensions: the outermost rows, and within that, 1+dimensional patterns (cells). Use tensor.NewRowCellsView to organize data into the desired split between a 1D outermost Row dimension and the remaining Cells dimension. The metric function must have the MetricFunc signature. The results fill in the elements of the output matrix, which is symmetric, and only the lower triangular part is computed, with results copied to the upper triangular region, for maximum efficiency.
func MatrixOut ¶
MatrixOut computes the rows x rows square distance / similarity matrix between the patterns for each row of the given higher dimensional input tensor, which must have at least 2 dimensions: the outermost rows, and within that, 1+dimensional patterns (cells). Use tensor.NewRowCellsView to organize data into the desired split between a 1D outermost Row dimension and the remaining Cells dimension. The metric function must have the MetricFunc signature. The results fill in the elements of the output matrix, which is symmetric, and only the lower triangular part is computed, with results copied to the upper triangular region, for maximum efficiency.
func SumSquares ¶
SumSquares computes the sum of squares differences between tensor values, See MetricFunc for general information.
func SumSquaresBinTol ¶
SumSquaresBinTol computes the sum of squares differences between tensor values, with binary tolerance: differences < 0.5 are thresholded to 0. See MetricFunc for general information.
func SumSquaresBinTolOut ¶
SumSquaresBinTolOut computes the sum of squares differences between tensor values, with binary tolerance: differences < 0.5 are thresholded to 0.
func SumSquaresBinTolScaleOut64 ¶
SumSquaresBinTolScaleOut64 computes the sum of squares differences between tensor values, with binary tolerance: differences < 0.5 are thresholded to 0. returning scale and ss factors aggregated separately for better numerical stability, per BLAS.
func SumSquaresOut ¶
SumSquaresOut computes the sum of squares differences between tensor values, See MetricOutFunc for general information.
func SumSquaresOut64 ¶
SumSquaresOut64 computes the sum of squares differences between tensor values, and returns the Float64 output values for use in subsequent computations.
func SumSquaresScaleOut64 ¶
SumSquaresScaleOut64 computes the sum of squares differences between tensor values, returning scale and ss factors aggregated separately for better numerical stability, per BLAS.
func Vectorize2Out64 ¶
func Vectorize2Out64(a, b tensor.Tensor, iniX, iniY float64, fun func(a, b, ox, oy float64) (float64, float64)) (ox64, oy64 *tensor.Float64)
Vectorize2Out64 is a version of VectorizeOut64 that separately aggregates two output values, x and y as tensor.Float64.
func Vectorize3Out64 ¶
func Vectorize3Out64(a, b tensor.Tensor, iniX, iniY, iniZ float64, fun func(a, b, ox, oy, oz float64) (float64, float64, float64)) (ox64, oy64, oz64 *tensor.Float64)
Vectorize3Out64 is a version of VectorizeOut64 that has 3 outputs instead of 1.
func VectorizeOut64 ¶
func VectorizeOut64(a, b tensor.Tensor, out tensor.Values, ini float64, fun func(a, b, agg float64) float64) *tensor.Float64
VectorizeOut64 is the general compute function for metric. This version makes a Float64 output tensor for aggregating and computing values, and then copies the results back to the original output. This allows metric functions to operate directly on integer valued inputs and produce sensible results. It returns the Float64 output tensor for further processing as needed. a and b are already enforced to be the same shape.
func VectorizePre3Out64 ¶
func VectorizePre3Out64(a, b tensor.Tensor, iniX, iniY, iniZ float64, preA, preB *tensor.Float64, fun func(a, b, preA, preB, ox, oy, oz float64) (float64, float64, float64)) (ox64, oy64, oz64 *tensor.Float64)
VectorizePre3Out64 is a version of VectorizePreOut64 that takes additional tensor.Float64 inputs of pre-computed values, e.g., the means of each output cell, and has 3 outputs instead of 1.
func VectorizePreOut64 ¶
func VectorizePreOut64(a, b tensor.Tensor, out tensor.Values, ini float64, preA, preB *tensor.Float64, fun func(a, b, preA, preB, agg float64) float64) *tensor.Float64
VectorizePreOut64 is a version of VectorizeOut64 that takes additional tensor.Float64 inputs of pre-computed values, e.g., the means of each output cell.
Types ¶
type MetricFunc ¶
MetricFunc is the function signature for a metric function, which is computed over the outermost row dimension and the output is the shape of the remaining inner cells (a scalar for 1D inputs). Use tensor.As1D, tensor.NewRowCellsView, tensor.Cells1D etc to reshape and reslice the data as needed. All metric functions skip over NaN's, as a missing value, and use the min of the length of the two tensors. Metric functions cannot be computed in parallel, e.g., using VectorizeThreaded or GPU, due to shared writing to the same output values. Special implementations are required if that is needed.
func AsMetricFunc ¶
func AsMetricFunc(fun any) (MetricFunc, error)
AsMetricFunc returns given function as a MetricFunc function, or an error if it does not fit that signature.
type MetricOutFunc ¶
MetricOutFunc is the function signature for a metric function, that takes output values as the final argument. See MetricFunc. This version is for computationally demanding cases and saves reallocation of output.
func AsMetricOutFunc ¶
func AsMetricOutFunc(fun any) (MetricOutFunc, error)
AsMetricOutFunc returns given function as a MetricFunc function, or an error if it does not fit that signature.
type Metrics ¶
type Metrics int32 //enums:enum -trim-prefix Metric
Metrics are standard metric functions
const ( // L2Norm is the square root of the sum of squares differences // between tensor values, aka the Euclidean distance. MetricL2Norm Metrics = iota // SumSquares is the sum of squares differences between tensor values. MetricSumSquares // L1Norm is the sum of the absolute value of differences // between tensor values, the L1 Norm. MetricL1Norm // Hamming is the sum of 1s for every element that is different, // i.e., "city block" distance. MetricHamming // L2NormBinTol is the [L2Norm] square root of the sum of squares // differences between tensor values, with binary tolerance: // differences < 0.5 are thresholded to 0. MetricL2NormBinTol // SumSquaresBinTol is the [SumSquares] differences between tensor values, // with binary tolerance: differences < 0.5 are thresholded to 0. MetricSumSquaresBinTol // InvCosine is 1-[Cosine], which is useful to convert it // to an Increasing metric where more different vectors have larger metric values. MetricInvCosine // InvCorrelation is 1-[Correlation], which is useful to convert it // to an Increasing metric where more different vectors have larger metric values. MetricInvCorrelation // CrossEntropy is a standard measure of the difference between two // probabilty distributions, reflecting the additional entropy (uncertainty) associated // with measuring probabilities under distribution b when in fact they come from // distribution a. It is also the entropy of a plus the divergence between a from b, // using Kullback-Leibler (KL) divergence. It is computed as: // a * log(a/b) + (1-a) * log(1-a/1-b). MetricCrossEntropy // DotProduct is the sum of the co-products of the tensor values. MetricDotProduct // Covariance is co-variance between two vectors, // i.e., the mean of the co-product of each vector element minus // the mean of that vector: cov(A,B) = E[(A - E(A))(B - E(B))]. MetricCovariance // Correlation is the standardized [Covariance] in the range (-1..1), // computed as the mean of the co-product of each vector // element minus the mean of that vector, normalized by the product of their // standard deviations: cor(A,B) = E[(A - E(A))(B - E(B))] / sigma(A) sigma(B). // Equivalent to the [Cosine] of mean-normalized vectors. MetricCorrelation // Cosine is high-dimensional angle between two vectors, // in range (-1..1) as the normalized [DotProduct]: // inner product / sqrt(ssA * ssB). See also [Correlation]. MetricCosine )
const MetricsN Metrics = 13
MetricsN is the highest valid value for type Metrics, plus one.
func MetricsValues ¶
func MetricsValues() []Metrics
MetricsValues returns all possible values for the type Metrics.
func (Metrics) Call ¶
Call calls a standard Metrics enum function on given tensors. Output results are in the out tensor.
func (Metrics) FuncName ¶
FuncName returns the package-qualified function name to use in tensor.Call to call this function.
func (Metrics) Increasing ¶
Increasing returns true if the distance metric is such that metric values increase as a function of distance (e.g., L2Norm) and false if metric values decrease as a function of distance (e.g., Cosine, Correlation)
func (Metrics) MarshalText ¶
MarshalText implements the encoding.TextMarshaler interface.
func (*Metrics) SetString ¶
SetString sets the Metrics value from its string representation, and returns an error if the string is invalid.
func (*Metrics) UnmarshalText ¶
UnmarshalText implements the encoding.TextUnmarshaler interface.