论文标题
液体时剂网络
Liquid Time-constant Networks
论文作者
论文摘要
我们介绍了新的一类时连续的经常性神经网络模型。我们没有通过隐式非线性声明学习系统的动力学，而是通过非线性互链的门来构建线性一阶动力系统网络。最终的模型代表具有变化（即液体）时体耦合到其隐藏状态的动态系统，其输出由数值微分方程求解器计算。这些神经网络表现出稳定且有界的行为，在神经普通微分方程家族中产生较高的表达性，并在时间序列预测任务上提高了性能。为了证明这些属性，我们首先采用一种理论方法来找到其动力学的界限，并通过潜在轨迹空间中的轨迹长度度量计算其表达能力。然后，我们进行了一系列的时间序列预测实验，以表现出与经典和现代RNN相比，液体时恒定网络（LTC）的近似能力。代码和数据可从https://github.com/raminmh/liquid_time_constant_networks获得
We introduce a new class of time-continuous recurrent neural network models. Instead of declaring a learning system's dynamics by implicit nonlinearities, we construct networks of linear first-order dynamical systems modulated via nonlinear interlinked gates. The resulting models represent dynamical systems with varying (i.e., liquid) time-constants coupled to their hidden state, with outputs being computed by numerical differential equation solvers. These neural networks exhibit stable and bounded behavior, yield superior expressivity within the family of neural ordinary differential equations, and give rise to improved performance on time-series prediction tasks. To demonstrate these properties, we first take a theoretical approach to find bounds over their dynamics and compute their expressive power by the trajectory length measure in latent trajectory space. We then conduct a series of time-series prediction experiments to manifest the approximation capability of Liquid Time-Constant Networks (LTCs) compared to classical and modern RNNs. Code and data are available at https://github.com/raminmh/liquid_time_constant_networks
