论文标题
尖锐的MAML：清晰感 - 默认元素元学习
Sharp-MAML: Sharpness-Aware Model-Agnostic Meta Learning
论文作者
论文摘要
目前，模型不合时宜的元学习（MAML）是几乎没有元学习的主要方法之一。尽管它具有有效性，但由于先天的二线问题结构，MAML的优化可能具有挑战性。具体而言，MAML的损失格局比其经验风险最小化的对应物更为复杂，可能的鞍点和局部最小化可能更复杂。为了应对这一挑战，我们利用了最近发明的清晰度最小化的最小化，并开发出一种敏锐的MAML方法，我们称其为Sharp MAML。我们从经验上证明，Sharp-MAML及其计算有效的变体可以胜过普通的MAML基线（例如，Mini-ImageNet上的$+3 \％$ $精度）。我们通过收敛速率分析和尖锐MAML的概括结合了实证研究。据我们所知，这是在双层学习背景下对清晰度最小化的首次经验和理论研究。该代码可在https://github.com/mominabbass/sharp-maml上找到。
Model-agnostic meta learning (MAML) is currently one of the dominating approaches for few-shot meta-learning. Albeit its effectiveness, the optimization of MAML can be challenging due to the innate bilevel problem structure. Specifically, the loss landscape of MAML is much more complex with possibly more saddle points and local minimizers than its empirical risk minimization counterpart. To address this challenge, we leverage the recently invented sharpness-aware minimization and develop a sharpness-aware MAML approach that we term Sharp-MAML. We empirically demonstrate that Sharp-MAML and its computation-efficient variant can outperform the plain-vanilla MAML baseline (e.g., $+3\%$ accuracy on Mini-Imagenet). We complement the empirical study with the convergence rate analysis and the generalization bound of Sharp-MAML. To the best of our knowledge, this is the first empirical and theoretical study on sharpness-aware minimization in the context of bilevel learning. The code is available at https://github.com/mominabbass/Sharp-MAML.
