You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
This repository has been archived by the owner on Oct 15, 2019. It is now read-only.
I tried to change L33 to: ret = npr.uniform(low=-var, high=var, size=shape)
But it caused another exception in MXNet, this might be another issue
Updated: Even the above is fixed, it is still wrong as the Xavier initialization depends on different activation functions. See clab/dynet#295 and clab/dynet#348 for discussions.
The text was updated successfully, but these errors were encountered:
The Xavier init in https://github.com/dmlc/minpy/blob/master/minpy/nn/init.py#L33
is wrong. According to Glorot et al (2010) Eq.(1) and Eq.(16), the weights are sampled from a uniform dist, not a normal dist (I assume the npr.randn is a normal dist as numpy.random does).
I tried to change L33 to:
ret = npr.uniform(low=-var, high=var, size=shape)
But it caused another exception in MXNet, this might be another issue
Updated: Even the above is fixed, it is still wrong as the Xavier initialization depends on different activation functions. See clab/dynet#295 and clab/dynet#348 for discussions.
The text was updated successfully, but these errors were encountered: