论文标题
学习不稳定的动态系统，并随时间加权的对数损失
Learning Unstable Dynamical Systems with Time-Weighted Logarithmic Loss
论文作者
论文摘要
当训练线性动力学模型的参数时，如果将平方错误损失用作训练损失函数，则梯度下降算法可能会收敛。将参数空间限制为较小的子集并在此子集中运行梯度下降算法可以允许学习稳定的动态系统，但是此策略对不稳定的系统不起作用。在这项工作中，我们研究了梯度下降算法的动力学，并查明是什么导致学习不稳定系统的困难。我们表明，在从系统的不同时间进行的观察以学习影响梯度下降算法的动力学，其程度大大不同。我们引入了一个时间加权的对数损失函数，以解决这种失衡，并证明其在学习不稳定系统中的有效性。
When training the parameters of a linear dynamical model, the gradient descent algorithm is likely to fail to converge if the squared-error loss is used as the training loss function. Restricting the parameter space to a smaller subset and running the gradient descent algorithm within this subset can allow learning stable dynamical systems, but this strategy does not work for unstable systems. In this work, we look into the dynamics of the gradient descent algorithm and pinpoint what causes the difficulty of learning unstable systems. We show that observations taken at different times from the system to be learned influence the dynamics of the gradient descent algorithm in substantially different degrees. We introduce a time-weighted logarithmic loss function to fix this imbalance and demonstrate its effectiveness in learning unstable systems.
