tensor

Documentation

Index

• Variables
• func AddBiasLastDimInGPU(out, bias *Tensor) bool
• func ClearPersistentGPU()
• func CopyData(dst, src []float32)
• func InitGPU() error
• func ShutdownGPU()
• type AllocReporter
  • func FirstReporter(rep ...AllocReporter) AllocReporter
• type Backend
• type ElemFunc
• type Tensor
  • func New(shape ...int) *Tensor
  • func NewGPU(shape []int, data []float32, rep AllocReporter) *Tensor
  • func NewRandom(shape []int, scale float32) *Tensor
  • func NewReported(rep AllocReporter, shape ...int) *Tensor
  • func NewWithData(shape []int, data []float32) *Tensor
  • func UnmarshalBinary(data []byte) (*Tensor, error)
  • func (a *Tensor) Add(b *Tensor) *Tensor
  • func (a *Tensor) AddIn(b *Tensor)
  • func (a *Tensor) AddMulNumIn(b *Tensor, n float32)
  • func (a *Tensor) AddNumIn(n float32)
  • func (a *Tensor) AddNumInAt(p []int, n float32)
  • func (a *Tensor) Apply(fn ElemFunc) *Tensor
  • func (a *Tensor) ApplyIn(fn ElemFunc)
  • func (a *Tensor) ArgMax() int
  • func (a *Tensor) CheckData()
  • func (a *Tensor) Clear()
  • func (a *Tensor) Copy() *Tensor
  • func (t *Tensor) Data() []float32
  • func (a *Tensor) Dim() int
  • func (t *Tensor) EnsureCPU()
  • func (t *Tensor) EnsureGPU() (cuda.DevicePtr, error)
  • func (a *Tensor) Exp() *Tensor
  • func (a *Tensor) ExpectShape(shape ...int)
  • func (a *Tensor) Fill(value float32)
  • func (t *Tensor) Free()
  • func (t *Tensor) GPUResident() bool
  • func (a *Tensor) Get(p ...int) float32
  • func (a *Tensor) Get2D(row, col int) float32
  • func (a *Tensor) IsSameShape(b *Tensor) bool
  • func (a *Tensor) Load(file io.Reader) error
  • func (a *Tensor) Log() *Tensor
  • func (t *Tensor) MarkCPUModified()
  • func (t *Tensor) MarkGPUModified()
  • func (a *Tensor) MarshalBinary() []byte
  • func (a *Tensor) MatMul(b *Tensor) *Tensor
  • func (a *Tensor) MatMulBatch(b *Tensor) *Tensor
  • func (a *Tensor) MatMulLastDim(b *Tensor) *Tensor
  • func (a *Tensor) Max() float32
  • func (a *Tensor) Mean() float32
  • func (a *Tensor) Mul(b *Tensor) *Tensor
  • func (a *Tensor) MulIn(b *Tensor)
  • func (a *Tensor) MulNum(n float32) *Tensor
  • func (a *Tensor) MulNumIn(n float32)
  • func (a *Tensor) Numel() int
  • func (a *Tensor) Permute(axes ...int) *Tensor
  • func (a *Tensor) ReduceToShape(shape []int) *Tensor
  • func (a *Tensor) Reshape(shape ...int) *Tensor
  • func (logits *Tensor) SampleSoftmax(temperature float32) int
  • func (a *Tensor) Save(file io.Writer) error
  • func (a *Tensor) SelectDim(dim, index int) *Tensor
  • func (a *Tensor) Set(p []int, value float32)
  • func (a *Tensor) Set1D(row int, value float32)
  • func (a *Tensor) Set2D(row, col int, value float32)
  • func (a *Tensor) SetData(b *Tensor)
  • func (a *Tensor) SliceDim(dim, start, end int) *Tensor
  • func (a *Tensor) Softmax() *Tensor
  • func (a *Tensor) Sub(b *Tensor) *Tensor
  • func (a *Tensor) SubIn(b *Tensor)
  • func (a *Tensor) SubNum(n float32) *Tensor
  • func (a *Tensor) Sum() float32
  • func (a *Tensor) SumDim(dim int) *Tensor
  • func (a *Tensor) Transpose() *Tensor
  • func (a *Tensor) TransposeInto(dst *Tensor)
  • func (a *Tensor) View(p ...int) *Tensor

Variables

var (
	Threads = max(1, runtime.NumCPU()/3*2) // number of threads used when possible
	Checks  = false                        // for development, will slow down
)

var (
	// UseGPU enables CUDA GEMM when InitGPU succeeded.
	UseGPU = false
)

Functions

func AddBiasLastDimInGPU

func AddBiasLastDimInGPU(out, bias *Tensor) bool

AddBiasLastDimInGPU adds 1D bias to last dim on GPU when possible.

func ClearPersistentGPU

func ClearPersistentGPU()

ClearPersistentGPU frees all persistent GPU allocations.

func CopyData

func CopyData(dst, src []float32)

CopyData copies src into dst using multiple threads for large buffers.

func InitGPU

func InitGPU() error

InitGPU loads libgoneural_cuda.so and enables GPU matmul.

func ShutdownGPU

func ShutdownGPU()

ShutdownGPU releases CUDA resources and disables GPU matmul.

Types

type AllocReporter

type AllocReporter interface {
	ReportAlloc(t *Tensor)
}

AllocReporter receives byte-size updates as tensors are allocated.

func FirstReporter

func FirstReporter(rep ...AllocReporter) AllocReporter

FirstReporter returns the optional reporter from a variadic argument list.

type Backend

type Backend interface {
	Init() error
	Shutdown()
	GemmF32(A, B, C []float32, M, N, K int) error
	GemmDeviceAt(aDev, bDev, cDev cuda.DevicePtr, aOff, bOff, cOff, M, N, K int) error
	GemmStridedBatchedAt(aDev, bDev, cDev cuda.DevicePtr, aOff, bOff, cOff, M, N, K, batchCount int) error
}

Backend runs GEMM on CPU or CUDA.

type ElemFunc

type ElemFunc func(i int, v float32) float32

type Tensor

type Tensor struct {
	Shape []int // row-major dimensions
	// contains filtered or unexported fields
}

Tensor is a shaped view into shared buffer storage.

func New

func New(shape ...int) *Tensor

New tensor of specified shape shape is copied (safe to modify) all data zero

func NewGPU

func NewGPU(shape []int, data []float32, rep AllocReporter) *Tensor

NewGPU creates a tensor with persistent GPU storage policy.

func NewRandom

func NewRandom(shape []int, scale float32) *Tensor

New tensor of specified shape, random values standard normal (Gaussian) distribution

func NewReported

func NewReported(rep AllocReporter, shape ...int) *Tensor

NewReported allocates a tensor and reports its size to rep when non-nil.

func NewWithData

func NewWithData(shape []int, data []float32) *Tensor

New tensor of specified shape and data (does not copy data)

func UnmarshalBinary

func UnmarshalBinary(data []byte) (*Tensor, error)

func (*Tensor) Add

func (a *Tensor) Add(b *Tensor) *Tensor

func (*Tensor) AddIn

func (a *Tensor) AddIn(b *Tensor)

func (*Tensor) AddMulNumIn

func (a *Tensor) AddMulNumIn(b *Tensor, n float32)

func (*Tensor) AddNumIn

func (a *Tensor) AddNumIn(n float32)

func (*Tensor) AddNumInAt

func (a *Tensor) AddNumInAt(p []int, n float32)

func (*Tensor) Apply

func (a *Tensor) Apply(fn ElemFunc) *Tensor

Applies specified function to each value and returns new tensor

func (*Tensor) ApplyIn

func (a *Tensor) ApplyIn(fn ElemFunc)

Applies specified function to each value in-place

func (*Tensor) ArgMax

func (a *Tensor) ArgMax() int

Gets index of the max value in 1D tensor

func (*Tensor) CheckData

func (a *Tensor) CheckData()

func (*Tensor) Clear

func (a *Tensor) Clear()

Clears all data to zero in-place

func (*Tensor) Copy

func (a *Tensor) Copy() *Tensor

New tensor clean copy //!

func (*Tensor) Data

func (t *Tensor) Data() []float32

Returns the CPU buffer, downloads from GPU when needed

func (*Tensor) Dim

func (a *Tensor) Dim() int

Number of dimensions

func (*Tensor) EnsureCPU

func (t *Tensor) EnsureCPU()

Ensures CPU buffer has valid data, downloads from GPU when needed

func (*Tensor) EnsureGPU

func (t *Tensor) EnsureGPU() (cuda.DevicePtr, error)

Ensures GPU buffer has valid data, uploads from CPU when needed

func (*Tensor) Exp

func (a *Tensor) Exp() *Tensor

Exponential function element-wise

func (*Tensor) ExpectShape

func (a *Tensor) ExpectShape(shape ...int)

Checks that tensor has exactly specified shape

func (*Tensor) Fill

func (a *Tensor) Fill(value float32)

Fills data with specified value in-place

func (*Tensor) Free

func (t *Tensor) Free()

Free releases this tensor handle; frees GPU when last handle is gone.

func (*Tensor) GPUResident

func (t *Tensor) GPUResident() bool

GPUResident reports whether this tensor's buffer has valid GPU data.

func (*Tensor) Get

func (a *Tensor) Get(p ...int) float32

Gets value at specified coordinate

func (*Tensor) Get2D

func (a *Tensor) Get2D(row, col int) float32

Gets value assuming 2D tensor

func (*Tensor) IsSameShape

func (a *Tensor) IsSameShape(b *Tensor) bool

Tells if tensors have exactly same shape

func (*Tensor) Load

func (a *Tensor) Load(file io.Reader) error

func (*Tensor) Log

func (a *Tensor) Log() *Tensor

Log function element-wise

func (*Tensor) MarkCPUModified

func (t *Tensor) MarkCPUModified()

Marks CPU buffer modified, GPU buffer becomes invalid

func (*Tensor) MarkGPUModified

func (t *Tensor) MarkGPUModified()

Marks GPU buffer modified, CPU buffer becomes invalid

func (*Tensor) MarshalBinary

func (a *Tensor) MarshalBinary() []byte

func (*Tensor) MatMul

func (a *Tensor) MatMul(b *Tensor) *Tensor

Matrix multiplication between tensors. Cross-dimension only happens between two inner dimensions [row,col], and above dimensions must match exactly. No automatic reshape is made, because would not be explicit, caller must reshape and potentially have inner dimension "1". Outer dimension broadcasting is supported.

func (*Tensor) MatMulBatch

func (a *Tensor) MatMulBatch(b *Tensor) *Tensor

Performs per-batch matrix multiplication for 3D tensors. For each batch: out[b] = a[b] @ b[b]

func (*Tensor) MatMulLastDim

func (a *Tensor) MatMulLastDim(b *Tensor) *Tensor

Matrix multiplication applied to the last dimension of A with 2D matrix B

func (*Tensor) Max

func (a *Tensor) Max() float32

Max scalar of all tensor values

func (*Tensor) Mean

func (a *Tensor) Mean() float32

Mean scalar of all values

func (*Tensor) Mul

func (a *Tensor) Mul(b *Tensor) *Tensor

Multiply element-wise

func (*Tensor) MulIn

func (a *Tensor) MulIn(b *Tensor)

Multiply element-wise in-place

func (*Tensor) MulNum

func (a *Tensor) MulNum(n float32) *Tensor

func (*Tensor) MulNumIn

func (a *Tensor) MulNumIn(n float32)

func (*Tensor) Numel

func (a *Tensor) Numel() int

Number of elements

func (*Tensor) Permute

func (a *Tensor) Permute(axes ...int) *Tensor

Reorders tensor according to specified axes. Axes must contain each dimension index exactly once, defining how the original axes are mapped into the output tensor.

func (*Tensor) ReduceToShape

func (a *Tensor) ReduceToShape(shape []int) *Tensor

Reduces tensor to match specified lower shape

func (*Tensor) Reshape

func (a *Tensor) Reshape(shape ...int) *Tensor

Reshapes tensor (data shared with source)

func (*Tensor) SampleSoftmax

func (logits *Tensor) SampleSoftmax(temperature float32) int

func (*Tensor) Save

func (a *Tensor) Save(file io.Writer) error

func (*Tensor) SelectDim

func (a *Tensor) SelectDim(dim, index int) *Tensor

Selects the slice at the given index along the specified dimension and removes that dimension. Returns a new tensor. Example: [B,T,D].SelectDim(1,t) -> [B,D] where T=t

func (*Tensor) Set

func (a *Tensor) Set(p []int, value float32)

Sets value at coordinates

func (*Tensor) Set1D

func (a *Tensor) Set1D(row int, value float32)

Sets value assuming 1D tensor

func (*Tensor) Set2D

func (a *Tensor) Set2D(row, col int, value float32)

Sets value assuming 2D tensor

func (*Tensor) SetData

func (a *Tensor) SetData(b *Tensor)

Sets data from another tensor, clean copy

func (*Tensor) SliceDim

func (a *Tensor) SliceDim(dim, start, end int) *Tensor

Returns a slice (copy) on the tensor sliced along the given dimension in the half-open range [start:end]. In other words restricts one dimension (dim) to range [start:end] while keeping all other dimensions unchanged.

func (*Tensor) Softmax

func (a *Tensor) Softmax() *Tensor

Turns a vector of arbitrary real numbers (logits) into a probability distribution: all values in [0,1] and sum to 1

func (*Tensor) Sub

func (a *Tensor) Sub(b *Tensor) *Tensor

NOTE: no broadcasting

func (*Tensor) SubIn

func (a *Tensor) SubIn(b *Tensor)

func (*Tensor) SubNum

func (a *Tensor) SubNum(n float32) *Tensor

func (*Tensor) Sum

func (a *Tensor) Sum() float32

Sum scalar of all values

func (*Tensor) SumDim

func (a *Tensor) SumDim(dim int) *Tensor

Sums the tensor along the specified dimension and removes that dimension.

func (*Tensor) Transpose

func (a *Tensor) Transpose() *Tensor

Swaps the two most inner dimensions

func (*Tensor) TransposeInto

func (a *Tensor) TransposeInto(dst *Tensor)

TransposeInto writes src^T into dst (swaps the two innermost dimensions).

func (*Tensor) View

func (a *Tensor) View(p ...int) *Tensor

Gets view to inner tensor at specified coordinate, shared data