论文标题
用形式语言的基准构图
Benchmarking Compositionality with Formal Languages
论文作者
论文摘要
将已知的原始概念重组为更大的新型组合是一种典型的人类认知能力。 NLP中的大型神经模型是否可以在从数据中学习的同时获得此能力是一个悬而未决的问题。在本文中，我们从形式语言的角度研究了这个问题。我们使用确定性有限状态传感器来制作具有控制组合性的可控属性的无限数量的数据集。通过对许多传感器进行随机采样，我们探讨了它们的哪些特性有助于通过神经网络的组成关系可学习。我们发现模型要么完全学习关系。关键是过渡覆盖范围，以每个过渡为400个示例设置软可学习性限制。
Recombining known primitive concepts into larger novel combinations is a quintessentially human cognitive capability. Whether large neural models in NLP can acquire this ability while learning from data is an open question. In this paper, we investigate this problem from the perspective of formal languages. We use deterministic finite-state transducers to make an unbounded number of datasets with controllable properties governing compositionality. By randomly sampling over many transducers, we explore which of their properties contribute to learnability of a compositional relation by a neural network. We find that the models either learn the relations completely or not at all. The key is transition coverage, setting a soft learnability limit at 400 examples per transition.
