论文标题
学习具有展开和双层优化的变分模型
Learning Variational Models with Unrolling and Bilevel Optimization
论文作者
论文摘要
在本文中，我们考虑了通过风险最小化监督学习中学习变分模型的问题。我们的目标是通过双重优化和通过算法展开对学习变异模型的两种方法进行更深入的了解。前者将变异模型视为低于风险最小化问题的较低级别优化问题，而后者则将较低级别优化问题替换为解决上述问题的算法。两种方法都在实践中使用，但是从计算的角度来看，展开要简单得多。为了分析和比较两种方法，我们考虑了一个简单的玩具模型，并明确计算所有风险和各自的估计器。我们表明，展开可能比双重优化方法更好，但是展开的性能可以在很大程度上取决于进一步的参数，有时会以意外的方式：虽然展开的算法的步骤大大很重要（并且学习得出的步骤大幅度提高了），而展开的迭代数量则表现出较小的角色。
In this paper we consider the problem of learning variational models in the context of supervised learning via risk minimization. Our goal is to provide a deeper understanding of the two approaches of learning of variational models via bilevel optimization and via algorithm unrolling. The former considers the variational model as a lower level optimization problem below the risk minimization problem, while the latter replaces the lower level optimization problem by an algorithm that solves said problem approximately. Both approaches are used in practice, but unrolling is much simpler from a computational point of view. To analyze and compare the two approaches, we consider a simple toy model, and compute all risks and the respective estimators explicitly. We show that unrolling can be better than the bilevel optimization approach, but also that the performance of unrolling can depend significantly on further parameters, sometimes in unexpected ways: While the stepsize of the unrolled algorithm matters a lot (and learning the stepsize gives a significant improvement), the number of unrolled iterations plays a minor role.
