deepboof.backward

Interface DFunctionBatchNorm<T extends deepboof.Tensor<T>>

All Superinterfaces:

deepboof.forward.BatchNorm, DBatchNorm<T>, deepboof.DFunction<T>, deepboof.Function<T>

All Known Implementing Classes:

DFunctionBatchNorm_F64

public interface DFunctionBatchNorm<T extends deepboof.Tensor<T>>
extends DBatchNorm<T>

Implementation of Batch Normalization for training networks. This has distinctly different behavior from forward only implementations. In this learning implementation, statistics of the input parameters are recomputed every time forward(T, T) is invoked. While for the forward only implementation those statistics are known already and not recomputed.

The above described change in behavior also changes how parameters are specified. mean and variance are no longer input parameters but are computed dynamically in the forwards pass.

NOTES:

• Variance is computed the unbiased formulation, i.e. divide by N-1 instead of N

Method Summary

Modifier and Type	Method and Description
void	forward(T input, T output) Applies batch normalization to each variable in the input.
void	setParameters(java.util.List<T> parameters) See forward(T, T) for a description of parameters.

Methods inherited from interface deepboof.backward.DBatchNorm

getMean, getVariance

Methods inherited from interface deepboof.forward.BatchNorm

getEPS, hasGammaBeta, setEPS

Methods inherited from interface deepboof.DFunction

backwards, evaluating, isLearning, learning

Methods inherited from interface deepboof.Function

getOutputShape, getParameters, getParameterShapes, getTensorType, initialize

Method Detail

forward

void forward(T input,
             T output)

Applies batch normalization to each variable in the input.

There is only a parameter tensor if BatchNorm.hasGammaBeta() returns true. If true then gamma, and beta are encoded in a single tensor in an interleaved fashion (gamma, beta).

 Summary Table
 -------------------------------------------------
 Input   shape = (N, d[i], ... , d[k])
 Output  shape = (N, d[i], ... , d[k])
 Params  shape = (d[i], ... , d[k], 2)
 -------------------------------------------------
 N    = Size of mini-batch
 d[i] = length of a dimension
 

NOTE: Interleaving is used in the parameters instead of multiple tensors to improve memory locality, which reduces cache misses.

Specified by:

forward in interface deepboof.Function<T extends deepboof.Tensor<T>>

Parameters:

input - Input tensor. Tensor with a shape of (N, d[i], ... , d[k]), where N is mini-batch size

output - Output tensor. Same shape as input tensor Modified.

setParameters

void setParameters(java.util.List<T> parameters)

See forward(T, T) for a description of parameters.

Specified by:

setParameters in interface deepboof.Function<T extends deepboof.Tensor<T>>

Parameters:

parameters - Variable tensor. (d[i], ... , d[k], 2). Not modified.