论文标题
具有自适应超参数的元学习
Meta-Learning with Adaptive Hyperparameters
论文作者
论文摘要
尽管它很受欢迎，但最近的一些作品质疑测试任务与培训任务不同时MAML的有效性，从而提出了各种任务条件的方法来改善初始化。我们专注于MAML框架，内环优化（或快速适应）中的补充因素，而不是寻找更好的任务感知初始化。因此，我们提出了一项新的重量更新规则，可大大增强快速适应过程。具体而言，我们引入了一个小型的元网络，该网络可以适应性地生成每步超级参数：学习率和权重衰减系数。实验结果验证了快速适应超级参数的适应性学习（ALFA）是同样重要的成分，在最近的几次学习方法中经常被忽略。令人惊讶的是，从随机初始化使用ALFA的快速适应已经超过了MAML。
Despite its popularity, several recent works question the effectiveness of MAML when test tasks are different from training tasks, thus suggesting various task-conditioned methodology to improve the initialization. Instead of searching for better task-aware initialization, we focus on a complementary factor in MAML framework, inner-loop optimization (or fast adaptation). Consequently, we propose a new weight update rule that greatly enhances the fast adaptation process. Specifically, we introduce a small meta-network that can adaptively generate per-step hyperparameters: learning rate and weight decay coefficients. The experimental results validate that the Adaptive Learning of hyperparameters for Fast Adaptation (ALFA) is the equally important ingredient that was often neglected in the recent few-shot learning approaches. Surprisingly, fast adaptation from random initialization with ALFA can already outperform MAML.
