论文标题
通过忽略，应用于域适应
Learning by Ignoring, with Application to Domain Adaptation
论文作者
论文摘要
通过忽略的学习是在人类的学习中广泛实践的，它忽略了较少的重要事物并将其排除在学习过程之外，并且已经显示出无处不在的有效性。已经有心理研究表明，学习忽略某些事物是帮助人们集中精力的有力工具。在本文中，我们探讨了是否可以借用这种有用的人类学习方法来改善机器学习。我们提出了一个新颖的机器学习框架，即通过忽略（LBI）称为学习。我们的框架自动确定了通过学习每个示例的忽略变量并将其排除在训练过程中，从而从目标分布转移了较大的域名示例。我们将LBI作为一个三级优化框架，其中涉及三个学习阶段：通过忽略变量来最大程度地减少权衡损失的预处理；微调;通过最小化验证损失来更新忽略变量。开发了一种基于梯度的算法，以有效地解决LBI中的三级优化问题。各种数据集的实验证明了我们框架的有效性。
Learning by ignoring, which identifies less important things and excludes them from the learning process, is broadly practiced in human learning and has shown ubiquitous effectiveness. There has been psychological studies showing that learning to ignore certain things is a powerful tool for helping people focus. In this paper, we explore whether this useful human learning methodology can be borrowed to improve machine learning. We propose a novel machine learning framework referred to as learning by ignoring (LBI). Our framework automatically identifies pretraining data examples that have large domain shift from the target distribution by learning an ignoring variable for each example and excludes them from the pretraining process. We formulate LBI as a three-level optimization framework where three learning stages are involved: pretraining by minimizing the losses weighed by ignoring variables; finetuning; updating the ignoring variables by minimizing the validation loss. A gradient-based algorithm is developed to efficiently solve the three-level optimization problem in LBI. Experiments on various datasets demonstrate the effectiveness of our framework.
