论文标题
将基于RELU的复发神经网络从离散时间转换为连续时间
Transformation of ReLU-based recurrent neural networks from discrete-time to continuous-time
论文作者
论文摘要
机器学习中使用的复发性神经网络（RNN）通常在离散时间（即作为递归图片）中提出。这为数据上的培训模型带来了许多优势，例如出于时间序列预测或动态系统识别的目的，作为离散时间系统存在强大有效的推理算法，并且无需进行微分方程的数值集成。另一方面，从数据推断出的动态系统的数学分析通常更方便，如果在连续时间进行表述，即作为普通（或部分）微分方程（ODE）的系统，则可以提供其他见解。在这里，我们展示了如何为基于RELU的RNN进行从离散到连续时间的翻译。我们证明了三个定理在各种条件下离散和连续时间公式之间的数学等效性，并说明了如何在不同的机器学习和非线性动态系统示例上使用我们的数学结果。
Recurrent neural networks (RNN) as used in machine learning are commonly formulated in discrete time, i.e. as recursive maps. This brings a lot of advantages for training models on data, e.g. for the purpose of time series prediction or dynamical systems identification, as powerful and efficient inference algorithms exist for discrete time systems and numerical integration of differential equations is not necessary. On the other hand, mathematical analysis of dynamical systems inferred from data is often more convenient and enables additional insights if these are formulated in continuous time, i.e. as systems of ordinary (or partial) differential equations (ODE). Here we show how to perform such a translation from discrete to continuous time for a particular class of ReLU-based RNN. We prove three theorems on the mathematical equivalence between the discrete and continuous time formulations under a variety of conditions, and illustrate how to use our mathematical results on different machine learning and nonlinear dynamical systems examples.
