论文标题
改进的凸孔concave minimax优化的算法
Improved Algorithms for Convex-Concave Minimax Optimization
论文作者
论文摘要
本文研究minimax优化问题$ \ min_x \ max_y f（x，y）$，其中$ f（x，y）$是$ m_x $ -stronglonglongly convex相对于$ x $，$ x $，$ m_y $ -stronglongly-stronglongly cove of $ y $ $ y $和$ y $和$ y $和$ y $ and $ y $ y $ y $ y $ y $ y y _x，l_x，l_ {xy}，l_ {xy}，l_y）$ - 平滑。张等。提供了任何一阶方法的梯度复杂度的以下下限：$ω\ bigl（\ sqrt {\ frac {\ frac {l_x} {m_x}+\ frac {l_ {l_ {xy}^2} {m_x {m_x {m_x m_y}+\frac{L_y}{m_y}}\ln(1/ε)\Bigr).$ This paper proposes a new algorithm with gradient complexity upper bound $\tilde{O}\Bigl(\sqrt{\frac{L_x}{m_x}+\frac{L\cdot l_ {xy}} {m_x m_y}+\ frac {l_y} {m_y}}} \ ln \ left（1/ε\ right）\ bigr），$ l = \ axax \ axmax \ = \ max \ {l_x，l_x，l_ {xy}，l_ {xy}，l_y \ y \} $。这可以改善最著名的上限$ \ tilde {o} \ left（\ sqrt {\ frac {\ frac {l^2} {m_x m_y}} \ ln^3 \ 3 \ left（1/ε\ right）\ right（1/ε\ right）\ right）。我们的界限实现了线性收敛速率和对条件号的依赖性，尤其是当$ l_ {xy} \ ll l $（即，当$ x $和$ y $之间的相互作用较弱时）。通过减少，我们的新结合还意味着改善了强烈凸出孔孔和凸孔concave minimax优化问题的界限。当$ f $是二次的时，我们可以进一步改善上限，该界限与下界匹配到一个小的亚物质因子。
This paper studies minimax optimization problems $\min_x \max_y f(x,y)$, where $f(x,y)$ is $m_x$-strongly convex with respect to $x$, $m_y$-strongly concave with respect to $y$ and $(L_x,L_{xy},L_y)$-smooth. Zhang et al. provided the following lower bound of the gradient complexity for any first-order method: $Ω\Bigl(\sqrt{\frac{L_x}{m_x}+\frac{L_{xy}^2}{m_x m_y}+\frac{L_y}{m_y}}\ln(1/ε)\Bigr).$ This paper proposes a new algorithm with gradient complexity upper bound $\tilde{O}\Bigl(\sqrt{\frac{L_x}{m_x}+\frac{L\cdot L_{xy}}{m_x m_y}+\frac{L_y}{m_y}}\ln\left(1/ε\right)\Bigr),$ where $L=\max\{L_x,L_{xy},L_y\}$. This improves over the best known upper bound $\tilde{O}\left(\sqrt{\frac{L^2}{m_x m_y}} \ln^3\left(1/ε\right)\right)$ by Lin et al. Our bound achieves linear convergence rate and tighter dependency on condition numbers, especially when $L_{xy}\ll L$ (i.e., when the interaction between $x$ and $y$ is weak). Via reduction, our new bound also implies improved bounds for strongly convex-concave and convex-concave minimax optimization problems. When $f$ is quadratic, we can further improve the upper bound, which matches the lower bound up to a small sub-polynomial factor.
