论文标题
最佳时期随机梯度下降方法，用于最低最大优化
Optimal Epoch Stochastic Gradient Descent Ascent Methods for Min-Max Optimization
论文作者
论文摘要
Hazan和Kale（2011）提出的Epoch梯度下降法（又名Epoch-GD）被认为是随机凸的随机最小化的突破，它实现了$ o（1/t）$的最佳收敛速率，$ o（1/t）$ t $ t $ t $ t $ t $ t $ t $ it i i i i i it it iT IT IT IT IT IT IT IT IT IT IT ABSECIPPABSE gap}。但是，它扩展到解决具有强凸力和强凹度的随机最小问题问题仍然保持开放，目前尚不清楚{\ it duality Gap}的快速速率是否可以在强大的凸度和强大的凹入性下进行{\ it duality Gap}来实现。 Although some recent studies have proposed stochastic algorithms with fast convergence rates for min-max problems, they require additional assumptions about the problem, e.g., smoothness, bi-linear structure, etc. In this paper, we bridge this gap by providing a sharp analysis of epoch-wise stochastic gradient descent ascent method (referred to as Epoch-GDA) for solving strongly convex strongly concave (SCSC) min-max problems, without imposing any additional关于平滑度或函数结构的假设。据我们所知，我们的结果是第一个显示Epoch-GDA可以达到$ O（1/t）$的最佳速率，对于一般SCSC Min-Max问题的双重性差距。我们强调，对于SCSC Min-Max问题的Epoch-GDA的强烈凸出最小化问题的时期GD的这种概括是非平凡的，需要新颖的技术分析。此外，我们注意到，关键引理也可以用于证明弱凸率强烈concove min-max问题的时期GDA的收敛性，从而导致几乎最佳的复杂性而无需诉诸平滑度或其他结构条件。
Epoch gradient descent method (a.k.a. Epoch-GD) proposed by Hazan and Kale (2011) was deemed a breakthrough for stochastic strongly convex minimization, which achieves the optimal convergence rate of $O(1/T)$ with $T$ iterative updates for the {\it objective gap}. However, its extension to solving stochastic min-max problems with strong convexity and strong concavity still remains open, and it is still unclear whether a fast rate of $O(1/T)$ for the {\it duality gap} is achievable for stochastic min-max optimization under strong convexity and strong concavity. Although some recent studies have proposed stochastic algorithms with fast convergence rates for min-max problems, they require additional assumptions about the problem, e.g., smoothness, bi-linear structure, etc. In this paper, we bridge this gap by providing a sharp analysis of epoch-wise stochastic gradient descent ascent method (referred to as Epoch-GDA) for solving strongly convex strongly concave (SCSC) min-max problems, without imposing any additional assumption about smoothness or the function's structure. To the best of our knowledge, our result is the first one that shows Epoch-GDA can achieve the optimal rate of $O(1/T)$ for the duality gap of general SCSC min-max problems. We emphasize that such generalization of Epoch-GD for strongly convex minimization problems to Epoch-GDA for SCSC min-max problems is non-trivial and requires novel technical analysis. Moreover, we notice that the key lemma can also be used for proving the convergence of Epoch-GDA for weakly-convex strongly-concave min-max problems, leading to a nearly optimal complexity without resorting to smoothness or other structural conditions.
