Created Thursday 01 May 2014
"Pick the weights from a uniform distribution whose mean is zero and whose variance is chosen to make the standard deviation of the induced local fields of the neurons (See neural networks) lie at the transition between the linear and saturated parts of the activation function" (I have no idea what the emphasized parts means.)
Most researchers initialize the weights to random values between 0 and 1. I use a gaussian with mean 0.5 and standard deviation 1.
No backlinks to this page. comments powered by Disqus