论文标题
Bert-knn：在预验证的语言模型中添加KNN搜索组件，以获得更好的质量请访问
BERT-kNN: Adding a kNN Search Component to Pretrained Language Models for Better QA
论文作者
论文摘要
Khandelwal等。 （2020）使用K-Nearest-neighbor（KNN）组件来提高语言模型性能。我们表明，这个想法对开放域问题回答（QA）是有益的。为了改善培训期间遇到的事实的回忆，我们将Bert（Devlin等，2019）与传统的信息检索步骤（IR）结合在一起，并在嵌入式文本集合的大型数据存储中进行了KNN搜索。我们的贡献如下：i）Bert-Knn在没有任何进一步培训的情况下，优于berts bert在固定风格的质量上。 ii）我们表明，伯特经常确定正确的响应类别（例如，美国城市），但只有KNN恢复了事实正确的答案（例如，“迈阿密”）。 iii）与伯特（Bert）相比，伯特·斯诺（Bert-Knn）擅长罕见事实。 iv）Bert-Knn可以轻松处理伯特（Bert）训练集（例如最近事件）所涵盖的事实。
Khandelwal et al. (2020) use a k-nearest-neighbor (kNN) component to improve language model performance. We show that this idea is beneficial for open-domain question answering (QA). To improve the recall of facts encountered during training, we combine BERT (Devlin et al., 2019) with a traditional information retrieval step (IR) and a kNN search over a large datastore of an embedded text collection. Our contributions are as follows: i) BERT-kNN outperforms BERT on cloze-style QA by large margins without any further training. ii) We show that BERT often identifies the correct response category (e.g., US city), but only kNN recovers the factually correct answer (e.g., "Miami"). iii) Compared to BERT, BERT-kNN excels for rare facts. iv) BERT-kNN can easily handle facts not covered by BERT's training set, e.g., recent events.
