Package deepboof.backward

Interface DFunctionBatchNorm<T extends deepboof.Tensor<T>>

All Superinterfaces:
deepboof.forward.BatchNorm, DBatchNorm<T>, deepboof.DFunction<T>, deepboof.Function<T>
All Known Implementing Classes:
DFunctionBatchNorm_F64
public interface DFunctionBatchNorm<T extends deepboof.Tensor<T>>
extends DBatchNorm<T>

Implementation of Batch Normalization for training networks. This has distinctly different behavior from forward only implementations. In this learning implementation, statistics of the input parameters are recomputed every time forward(T, T) is invoked. While for the forward only implementation those statistics are known already and not recomputed.

The above described change in behavior also changes how parameters are specified. mean and variance are no longer input parameters but are computed dynamically in the forwards pass.

NOTES:
  • Variance is computed the unbiased formulation, i.e. divide by N-1 instead of N

  • Method Summary

    Modifier and Type Method Description
    void forward​(T input, T output)
    Applies batch normalization to each variable in the input.
    void setParameters​(List<T> parameters)
    See forward(T, T) for a description of parameters.

    Methods inherited from interface deepboof.forward.BatchNorm

    getEPS, hasGammaBeta, setEPS

    Methods inherited from interface deepboof.backward.DBatchNorm

    getMean, getVariance

    Methods inherited from interface deepboof.DFunction

    backwards, evaluating, isLearning, learning

    Methods inherited from interface deepboof.Function

    getOutputShape, getParameters, getParameterShapes, getTensorType, initialize

  • Method Details

    • forward

      void forward​(T input, T output)

      Applies batch normalization to each variable in the input.

      There is only a parameter tensor if BatchNorm.hasGammaBeta() returns true. If true then gamma, and beta are encoded in a single tensor in an interleaved fashion (gamma, beta).

       Summary Table
 -------------------------------------------------
 Input   shape = (N, d[i], ... , d[k])
 Output  shape = (N, d[i], ... , d[k])
 Params  shape = (d[i], ... , d[k], 2)
 -------------------------------------------------
 N    = Size of mini-batch
 d[i] = length of a dimension

      NOTE: Interleaving is used in the parameters instead of multiple tensors to improve memory locality, which reduces cache misses.

      Specified by:
      forward in interface deepboof.Function<T extends deepboof.Tensor<T>>
      Parameters:
      input - Input tensor. Tensor with a shape of (N, d[i], ... , d[k]), where N is mini-batch size
      output - Output tensor. Same shape as input tensor Modified.

    • setParameters

      void setParameters​(List<T> parameters)
      See forward(T, T) for a description of parameters.
      Specified by:
      setParameters in interface deepboof.Function<T extends deepboof.Tensor<T>>
      Parameters:
      parameters - Variable tensor. (d[i], ... , d[k], 2). Not modified.