Papers of the day   All papers

RNNs Evolving in Equilibrium: A Solution to the Vanishing and Exploding Gradients

Comments

Aug 27 2019 Daisuke Okanohara

For RNN, they proposed to evolve the hidden state on the equilibrium manifold of ODE defined by the current hidden state and input. This solves gradient vanishing/explosion and makes state transition significantly stable; a big breakthrough. https://arxiv.org/abs/1908.08574
0 replies, 42 likes


Aug 27 2019 Daisuke Okanohara

IMO, this computation looks closer to actual brain computation than current RNN. Combined with backward-propagation free methods (e.g., equilibrium propagation) we may obtain BPTT-free learning algorithm which is effective as BPTT.
1 replies, 23 likes


Aug 26 2019 arxiv

RNNs Evolving in Equilibrium: A Solution to the Vanishing and Exploding Gradients. http://arxiv.org/abs/1908.08574 https://t.co/XH0PTEfgOY
0 replies, 8 likes


Aug 26 2019 Statistics Papers

RNNs Evolving in Equilibrium: A Solution to the Vanishing and Exploding Gradients. http://arxiv.org/abs/1908.08574
0 replies, 3 likes


Aug 26 2019 arXiv in review

#NeurIPS2019 RNNs Evolving in Equilibrium: A Solution to the Vanishing and Exploding Gradients. (arXiv:1908.08574v1 [cs\.LG]) http://arxiv.org/abs/1908.08574
0 replies, 2 likes


Aug 26 2019 午後のarXiv

"RNNs Evolving in Equilibrium: A Solution to the Vanishing and Exploding Gradients", Anil Kag, Ziming Zhang, Venkat… https://arxiv.org/abs/1908.08574
0 replies, 1 likes


Nov 20 2019 HotComputerScience

Most popular computer science paper of the day: "RNNs Evolving in Equilibrium: A Solution to the Vanishing and Exploding Gradients" https://hotcomputerscience.com/paper/rnns-evolving-in-equilibrium-a-solution-to-the-vanishing-and-exploding-gradients https://twitter.com/hillbig/status/1166491936212967425
0 replies, 1 likes


Content