论文标题
学习身份关系的权重先验
Weight Priors for Learning Identity Relations
论文作者
论文摘要
30多年来，学习抽象和系统关系一直是神经网络学习中的一个开放问题。最近已经显示，神经网络没有基于身份学习关系，并且无法很好地概括到看不见的数据。基于关系的模式（RBP）方法已被提出作为解决此问题的解决方案。在这项工作中，我们通过将RBP意识到网络权重上的贝叶斯先验来扩展RBP，以模拟身份关系。此重量先验导致在其他标准网络学习中修改了正则术语。在我们的实验中，我们表明，贝叶斯体重先验在学习基于身份的关系并且不会阻碍一般的神经网络学习时导致完美的概括。我们认为，可以轻松地扩展到其他形式的关系，对其他形式的关系延长，对许多其他学习任务有益。
Learning abstract and systematic relations has been an open issue in neural network learning for over 30 years. It has been shown recently that neural networks do not learn relations based on identity and are unable to generalize well to unseen data. The Relation Based Pattern (RBP) approach has been proposed as a solution for this problem. In this work, we extend RBP by realizing it as a Bayesian prior on network weights to model the identity relations. This weight prior leads to a modified regularization term in otherwise standard network learning. In our experiments, we show that the Bayesian weight priors lead to perfect generalization when learning identity based relations and do not impede general neural network learning. We believe that the approach of creating an inductive bias with weight priors can be extended easily to other forms of relations and will be beneficial for many other learning tasks.
