论文标题
通过使用凸双重性来区分价值函数
Differentiating the Value Function by using Convex Duality
论文作者
论文摘要
我们考虑参数优化问题的价值函数的差异化。此类问题在机器学习应用中无处不在，例如结构化支持向量机，矩阵分解和最小或最小问题。计算衍生物的现有方法取决于参数函数的强有力假设。因此，在几种情况下，没有理论上的证据表明给定算法分化策略可以计算值函数的真实梯度信息。我们利用凸双重性理论的众所周知的结果来放松条件，并得出机器学习中几类参数优化问题的衍生近似的收敛速率。我们在几个实验中证明了方法的多功能性，包括非平滑参数函数。即使在适用其他方法的设置中，我们的基于二元性的策略也会表现出良好的表现。
We consider the differentiation of the value function for parametric optimization problems. Such problems are ubiquitous in Machine Learning applications such as structured support vector machines, matrix factorization and min-min or minimax problems in general. Existing approaches for computing the derivative rely on strong assumptions of the parametric function. Therefore, in several scenarios there is no theoretical evidence that a given algorithmic differentiation strategy computes the true gradient information of the value function. We leverage a well known result from convex duality theory to relax the conditions and to derive convergence rates of the derivative approximation for several classes of parametric optimization problems in Machine Learning. We demonstrate the versatility of our approach in several experiments, including non-smooth parametric functions. Even in settings where other approaches are applicable, our duality based strategy shows a favorable performance.
