论文标题
任务刺激模型 - 反应元学习
Task-Robust Model-Agnostic Meta-Learning
论文作者
论文摘要
元学习方法表现出令人印象深刻的训练模型的能力，这些模型迅速学习了新任务。但是，这些方法仅旨在在对某些特定分布的任务中表现出色，这些任务通常在元训练和元测试中等效，而不是考虑最坏的任务绩效。在这项工作中，我们通过重新提出流行的模型 - 静态元学习（MAML）目标来介绍“任务常做”的概念[Finn等。 [2017年]这样的目标是最大程度地减少观察到的元训练任务的最大损失。解决这个新颖配方的解决方案是任务努力，因为它在最困难和/或罕见的任务上也将其赋予同等的重要性。这也意味着它在观察到的任务的所有分布中都表现良好，从而使元训练和元测试之间的任务分布的变化变得强大。我们提出了一种算法来解决拟议的最低 - 最大问题，并表明它以$ \ Mathcal {o}（1/ε^2）$的最佳速度收敛到$ \ MATHCAL {O}（1/ε^2）$，并以$（三δ）$ sentary点为$ \ MATHCAL CAL CAL CAL CAL CAL CALCAL CALCAL CALTAINTAL IN 1/δ^5 \}）$在非convex设置中。我们还为新的任务概括误差提供了上限，该误差捕获了最小化最差的任务损失的优势，并在正弦回归和图像分类实验中证明了这一优势。
Meta-learning methods have shown an impressive ability to train models that rapidly learn new tasks. However, these methods only aim to perform well in expectation over tasks coming from some particular distribution that is typically equivalent across meta-training and meta-testing, rather than considering worst-case task performance. In this work we introduce the notion of "task-robustness" by reformulating the popular Model-Agnostic Meta-Learning (MAML) objective [Finn et al. 2017] such that the goal is to minimize the maximum loss over the observed meta-training tasks. The solution to this novel formulation is task-robust in the sense that it places equal importance on even the most difficult and/or rare tasks. This also means that it performs well over all distributions of the observed tasks, making it robust to shifts in the task distribution between meta-training and meta-testing. We present an algorithm to solve the proposed min-max problem, and show that it converges to an $ε$-accurate point at the optimal rate of $\mathcal{O}(1/ε^2)$ in the convex setting and to an $(ε, δ)$-stationary point at the rate of $\mathcal{O}(\max\{1/ε^5, 1/δ^5\})$ in nonconvex settings. We also provide an upper bound on the new task generalization error that captures the advantage of minimizing the worst-case task loss, and demonstrate this advantage in sinusoid regression and image classification experiments.
