layernorm

package
Version: v0.7.0 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: May 24, 2021 License: BSD-2-Clause Imports: 4 Imported by: 0

Documentation

Overview

Package layernorm implements the Layer Normalization (LayerNorm) i method.

Reference: "Layer normalization" by Jimmy Lei Ba, Jamie Ryan Kiros, and Geoffrey E Hinton (2016). (https://arxiv.org/pdf/1607.06450.pdf)

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

This section is empty.

Types

type Model

type Model struct {
	nn.BaseModel
	W nn.Param `spago:"type:weights"`
	B nn.Param `spago:"type:biases"`
}

Model contains the serializable parameters.

func New

func New(size int) *Model

New returns a new model with parameters initialized to zeros.

func (*Model) Forward added in v0.2.0

func (m *Model) Forward(xs ...ag.Node) []ag.Node

Forward performs the forward step for each input node and returns the result. y = (x - E\[x\]) / sqrt(VAR\[x\] + [EPS]) * g + b

Source Files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
t or T : Toggle theme light dark auto
y or Y : Canonical URL