论文标题
如何训练您的神经颂歌：雅各布和动力学正则化的世界
How to train your neural ODE: the world of Jacobian and kinetic regularization
论文作者
论文摘要
大型数据集上的训练神经odes由于必须允许自适应数值求解器将其步骤大小提高到非常小的值，因此训练神经ODES尚不可处理。在实践中，这导致动态等同于数百甚至数千个层。在本文中，我们通过引入最佳运输和稳定性正规化的理论上结合来克服这一明显的困难，鼓励神经ODE希望从解决问题的所有动力学中更喜欢简单的动态。简单的动态导致更快的收敛性和更少的求解器离散化，从而大大降低了壁式时间的时间而不会损失性能。我们的方法使我们能够训练基于神经ODE的生成模型的性能与未注册的动态相同，并大大减少了训练时间。这会使神经OD在大规模应用中更接近实际相关性。
Training neural ODEs on large datasets has not been tractable due to the necessity of allowing the adaptive numerical ODE solver to refine its step size to very small values. In practice this leads to dynamics equivalent to many hundreds or even thousands of layers. In this paper, we overcome this apparent difficulty by introducing a theoretically-grounded combination of both optimal transport and stability regularizations which encourage neural ODEs to prefer simpler dynamics out of all the dynamics that solve a problem well. Simpler dynamics lead to faster convergence and to fewer discretizations of the solver, considerably decreasing wall-clock time without loss in performance. Our approach allows us to train neural ODE-based generative models to the same performance as the unregularized dynamics, with significant reductions in training time. This brings neural ODEs closer to practical relevance in large-scale applications.
