case class MsraFiller(varianceNormAverage: Boolean = true) extends InitializationMethod with Product with Serializable
A Filler based on the paper [He, Zhang, Ren and Sun 2015]: Specifically accounts for ReLU nonlinearities.
Aside: for another perspective on the scaling factor, see the derivation of [Saxe, McClelland, and Ganguli 2013 (v3)].
It fills the incoming matrix by randomly sampling Gaussian data with std = sqrt(2 / n) where n is the fanIn, fanOut, or their average, depending on the varianceNormAverage parameter.
- varianceNormAverage
VarianceNorm use average of (fanIn + fanOut) or just fanOut
- Alphabetic
- By Inheritance
- MsraFiller
- Serializable
- Serializable
- Product
- Equals
- InitializationMethod
- AnyRef
- Any
- Hide All
- Show All
- Public
- All
Instance Constructors
-
new
MsraFiller(varianceNormAverage: Boolean = true)
- varianceNormAverage
VarianceNorm use average of (fanIn + fanOut) or just fanOut
Type Members
-
type
Shape = Array[Int]
- Definition Classes
- InitializationMethod
Value Members
-
final
def
!=(arg0: Any): Boolean
- Definition Classes
- AnyRef → Any
-
final
def
##(): Int
- Definition Classes
- AnyRef → Any
-
final
def
==(arg0: Any): Boolean
- Definition Classes
- AnyRef → Any
-
final
def
asInstanceOf[T0]: T0
- Definition Classes
- Any
-
def
clone(): AnyRef
- Attributes
- protected[java.lang]
- Definition Classes
- AnyRef
- Annotations
- @native() @throws( ... )
-
final
def
eq(arg0: AnyRef): Boolean
- Definition Classes
- AnyRef
-
def
finalize(): Unit
- Attributes
- protected[java.lang]
- Definition Classes
- AnyRef
- Annotations
- @throws( classOf[java.lang.Throwable] )
-
final
def
getClass(): Class[_]
- Definition Classes
- AnyRef → Any
- Annotations
- @native()
-
def
init[T](variable: Tensor[T], dataFormat: VariableFormat)(implicit ev: TensorNumeric[T]): Unit
Initialize the given weight and bias.
Initialize the given weight and bias.
- variable
the weight to initialize
- dataFormat
the data format of weight indicating the dimension order of the weight. "output_first" means output is in the lower dimension "input_first" means input is in the lower dimension.
- Definition Classes
- MsraFiller → InitializationMethod
-
final
def
isInstanceOf[T0]: Boolean
- Definition Classes
- Any
-
final
def
ne(arg0: AnyRef): Boolean
- Definition Classes
- AnyRef
-
final
def
notify(): Unit
- Definition Classes
- AnyRef
- Annotations
- @native()
-
final
def
notifyAll(): Unit
- Definition Classes
- AnyRef
- Annotations
- @native()
-
final
def
synchronized[T0](arg0: ⇒ T0): T0
- Definition Classes
- AnyRef
- val varianceNormAverage: Boolean
-
final
def
wait(): Unit
- Definition Classes
- AnyRef
- Annotations
- @throws( ... )
-
final
def
wait(arg0: Long, arg1: Int): Unit
- Definition Classes
- AnyRef
- Annotations
- @throws( ... )
-
final
def
wait(arg0: Long): Unit
- Definition Classes
- AnyRef
- Annotations
- @native() @throws( ... )