论文标题
Maslow的锤子锤子遗忘：节点重复使用与节点激活
Maslow's Hammer for Catastrophic Forgetting: Node Re-Use vs Node Activation
论文作者
论文摘要
持续学习 - 按顺序学习新任务，同时保持旧任务的绩效 - 对于人工神经网络仍然尤其具有挑战性。令人惊讶的是，遗忘的数量并没有随着学习任务之间的差异而增加，但在中间相似性方面似乎是最坏的。 在本文中，我们从理论上分析了合成的教师学生框架和真实的数据设置，以提供这种现象的解释，即我们将Maslow的锤子假设命名为。我们的分析揭示了节点激活与节点重复使用之间的权衡，从而导致中间制度中最严重的遗忘。利用这种理解，我们将流行的算法干预措施重新解释了这种折衷的灾难性干扰，并确定它们最有效的制度。
Continual learning - learning new tasks in sequence while maintaining performance on old tasks - remains particularly challenging for artificial neural networks. Surprisingly, the amount of forgetting does not increase with the dissimilarity between the learned tasks, but appears to be worst in an intermediate similarity regime. In this paper we theoretically analyse both a synthetic teacher-student framework and a real data setup to provide an explanation of this phenomenon that we name Maslow's hammer hypothesis. Our analysis reveals the presence of a trade-off between node activation and node re-use that results in worst forgetting in the intermediate regime. Using this understanding we reinterpret popular algorithmic interventions for catastrophic interference in terms of this trade-off, and identify the regimes in which they are most effective.
