论文标题
反复学习规则的分类学
A Taxonomy of Recurrent Learning Rules
论文作者
论文摘要
通过时间（BPTT）的反向传播是训练复发性神经网络（RNN）的事实上的标准，但它是非毒性和非本地的。实时循环学习是一种因果替代方法，但效率很低。最近，E-Prop被提出为这些算法的因果，局部和有效的实用替代方法，通过从根本上修剪随时间携带的经常性依赖性来提供确切梯度的近似值。在这里，我们使用详细的符号从BPTT得出RTRL，从而为它们如何连接带来了直觉和澄清。此外，我们在图片中内部构架电子螺旋桨，使其近似。最后，我们得出了一种特殊案例的算法系列。
Backpropagation through time (BPTT) is the de facto standard for training recurrent neural networks (RNNs), but it is non-causal and non-local. Real-time recurrent learning is a causal alternative, but it is highly inefficient. Recently, e-prop was proposed as a causal, local, and efficient practical alternative to these algorithms, providing an approximation of the exact gradient by radically pruning the recurrent dependencies carried over time. Here, we derive RTRL from BPTT using a detailed notation bringing intuition and clarification to how they are connected. Furthermore, we frame e-prop within in the picture, formalising what it approximates. Finally, we derive a family of algorithms of which e-prop is a special case.
