论文标题
通过近端方法了解Nesterov的加速度
Understanding Nesterov's Acceleration via Proximal Point Method
论文作者
论文摘要
近端方法（PPM）是优化的基本方法，通常用作设计优化算法的构件。在这项工作中，我们使用PPM方法提供概念上的简单推导以及对Nesterov加速梯度方法（AGM）的不同版本的收敛分析。关键观察结果是，AGM是PPM的简单近似值，这导致了更新方程和股东周期尺寸的基本推导。这种观点还通过使用PPM的分析对AGM的收敛进行了透明且概念上的简单分析。这些推导也自然扩展到强烈的凸状情况。最终，本文介绍的结果既具有教学价值又具有概念价值。他们统一并解释了AGM的现有变体，同时激发了其他相关设置的其他加速方法。
The proximal point method (PPM) is a fundamental method in optimization that is often used as a building block for designing optimization algorithms. In this work, we use the PPM method to provide conceptually simple derivations along with convergence analyses of different versions of Nesterov's accelerated gradient method (AGM). The key observation is that AGM is a simple approximation of PPM, which results in an elementary derivation of the update equations and stepsizes of AGM. This view also leads to a transparent and conceptually simple analysis of AGM's convergence by using the analysis of PPM. The derivations also naturally extend to the strongly convex case. Ultimately, the results presented in this paper are of both didactic and conceptual value; they unify and explain existing variants of AGM while motivating other accelerated methods for practically relevant settings.
