论文标题
加速梯度流：风险，稳定性和隐式正则化
Accelerated Gradient Flow: Risk, Stability, and Implicit Regularization
论文作者
论文摘要
加速度和动量是机器学习和优化现代应用中的事实上的标准，但是，隐式正规化的大部分工作集中在不加速的方法上。在本文中，我们研究了Nesterov加速梯度方法和Polyak的重球方法产生的统计风险，当应用于最小二乘回归时，会引起几个连接以明确的惩罚。我们在连续时间进行分析，使我们能够比先前的工作更清晰，并揭示早期停止，稳定性和损失函数曲率之间的复杂相互作用。
Acceleration and momentum are the de facto standard in modern applications of machine learning and optimization, yet the bulk of the work on implicit regularization focuses instead on unaccelerated methods. In this paper, we study the statistical risk of the iterates generated by Nesterov's accelerated gradient method and Polyak's heavy ball method, when applied to least squares regression, drawing several connections to explicit penalization. We carry out our analyses in continuous-time, allowing us to make sharper statements than in prior work, and revealing complex interactions between early stopping, stability, and the curvature of the loss function.
