论文标题
关于最小值优化的负动量的次要性
On the Suboptimality of Negative Momentum for Minimax Optimization
论文作者
论文摘要
流畅的游戏优化最近对机器学习引起了极大的兴趣，因为它概括了单目标优化范式。但是，由于不同玩家之间的相互作用，游戏动力学更为复杂，因此与最小化根本不同，对算法设计提出了新的挑战。值得注意的是，由于其能够减少游戏动力学振荡的能力，因此首选负动量。然而，负动量的融合率仅在简单的双线性游戏中建立。在本文中，我们将分析扩展到平滑而强烈的强烈符合最小值游戏，通过采用各种不平等公式。通过将动量方法与Chebyshev多项式连接起来，我们表明负动量会在本地加速游戏动力学的收敛性，尽管速率较高。据我们所知，这是\ emph {第一项工作}，在这种情况下为负动量提供了明显的收敛率。
Smooth game optimization has recently attracted great interest in machine learning as it generalizes the single-objective optimization paradigm. However, game dynamics is more complex due to the interaction between different players and is therefore fundamentally different from minimization, posing new challenges for algorithm design. Notably, it has been shown that negative momentum is preferred due to its ability to reduce oscillation in game dynamics. Nevertheless, the convergence rate of negative momentum was only established in simple bilinear games. In this paper, we extend the analysis to smooth and strongly-convex strongly-concave minimax games by taking the variational inequality formulation. By connecting momentum method with Chebyshev polynomials, we show that negative momentum accelerates convergence of game dynamics locally, though with a suboptimal rate. To the best of our knowledge, this is the \emph{first work} that provides an explicit convergence rate for negative momentum in this setting.
