论文标题
我们如何加速发展类似人类的语言概括？
How Can We Accelerate Progress Towards Human-like Linguistic Generalization?
论文作者
论文摘要
该立场论文描述并批评了训练预先分布的分布（付费）评估范式，该范式已成为衡量自然语言理解进度的核心工具。该范式由三个阶段组成：（1）在任意大小的语料库上对单词预测模型进行预训练； （2）代表分类任务的培训集中的微调（转移学习）； （3）对与该训练集相同的分布提取的测试集评估。该范式有利于简单，低偏差的体系结构，首先可以缩放以处理大量数据，其次，可以捕获特定数据集的细粒统计属性，无论这些属性是否可能会推广到数据集之外的任务示例。这与人类形成鲜明对比，他们从几个数量级的数据中学习语言的数据比该评估范式偏爱的系统要少，并以一致的方式推广到新任务。我们倡导补充或取代用范式付出的范式，这些范式奖励像人类一样快速而坚固的体系结构。
This position paper describes and critiques the Pretraining-Agnostic Identically Distributed (PAID) evaluation paradigm, which has become a central tool for measuring progress in natural language understanding. This paradigm consists of three stages: (1) pre-training of a word prediction model on a corpus of arbitrary size; (2) fine-tuning (transfer learning) on a training set representing a classification task; (3) evaluation on a test set drawn from the same distribution as that training set. This paradigm favors simple, low-bias architectures, which, first, can be scaled to process vast amounts of data, and second, can capture the fine-grained statistical properties of a particular data set, regardless of whether those properties are likely to generalize to examples of the task outside the data set. This contrasts with humans, who learn language from several orders of magnitude less data than the systems favored by this evaluation paradigm, and generalize to new tasks in a consistent way. We advocate for supplementing or replacing PAID with paradigms that reward architectures that generalize as quickly and robustly as humans.
