论文标题
鞭打梯度下降动力学
Whiplash Gradient Descent Dynamics
论文作者
论文摘要
在本文中，我们提出了鞭打惯性梯度动力学，这是一种利用梯度信息的闭环优化方法，以在有限维设置中找到成本函数的最小值。我们介绍了用于凸功能的旋转系统的互隔渐进收敛分析。我们还介绍了放松序列，以解释算法的非古典性质和旋转算法的探索启发式变体，以确定性地逃脱鞍点。我们研究该算法的各种成本性能，并提供了使用积分约束界和新型Lyapunov速率方法来分析收敛率的实用方法。我们的结果表明，二次成本函数的收敛性多项式和指数率。
In this paper, we propose the Whiplash Inertial Gradient dynamics, a closed-loop optimization method that utilises gradient information, to find the minima of a cost function in finite-dimensional settings. We introduce the symplectic asymptotic convergence analysis for the Whiplash system for convex functions. We also introduce relaxation sequences to explain the non-classical nature of the algorithm and an exploring heuristic variant of the Whiplash algorithm to escape saddle points, deterministically. We study the algorithm's performance for various costs and provide a practical methodology for analyzing convergence rates using integral constraint bounds and a novel Lyapunov rate method. Our results demonstrate polynomial and exponential rates of convergence for quadratic cost functions.
