论文标题
耦合振荡复发性神经网络（CORNN）：一种精确且（梯度）的稳定体系结构，用于学习长时间的依赖关系
Coupled Oscillatory Recurrent Neural Network (coRNN): An accurate and (gradient) stable architecture for learning long time dependencies
论文作者
论文摘要
生物神经元的电路，例如在大脑的功能部位中，可以建模为耦合振荡器网络。受这些系统在保持状态变量界定的（梯度）界定的能力的启发下，我们为复发性神经网络提出了一种新颖的体系结构。我们提出的RNN基于二阶普通微分方程系统的时间限制，对受控非线性振荡器的网络进行建模。我们证明了隐藏状态梯度的精确界限，从而减轻了该RNN的爆炸和消失的梯度问题。实验表明，所提出的RNN在各种基准上的性能与艺术状态相当，这表明该体系结构的潜力提供了用于处理复杂顺序数据的稳定且准确的RNN。
Circuits of biological neurons, such as in the functional parts of the brain can be modeled as networks of coupled oscillators. Inspired by the ability of these systems to express a rich set of outputs while keeping (gradients of) state variables bounded, we propose a novel architecture for recurrent neural networks. Our proposed RNN is based on a time-discretization of a system of second-order ordinary differential equations, modeling networks of controlled nonlinear oscillators. We prove precise bounds on the gradients of the hidden states, leading to the mitigation of the exploding and vanishing gradient problem for this RNN. Experiments show that the proposed RNN is comparable in performance to the state of the art on a variety of benchmarks, demonstrating the potential of this architecture to provide stable and accurate RNNs for processing complex sequential data.
