论文标题
使用校准器来改善机器阅读理解的鲁棒性
Using calibrator to improve robustness in Machine Reading Comprehension
论文作者
论文摘要
机器阅读理解（MRC）取得了显着的结果，因为提出了一些强大的模型，例如BERT。但是，这些模型不够强大，并且容易受到对抗输入扰动和概括示例的影响。一些作品试图通过将一些相关示例添加到培训数据中，同时导致原始数据集降级，以提高特定数据类型的性能，因为数据分布的移位使得根据模型不可靠的软掌项概率使得答案排名。在本文中，我们提出了一种通过使用校准器作为事后重读者来改善鲁棒性的方法，该方法是基于XGBoost模型实现的。校准器将手动特征和表示学习功能结合在一起，并将其与候选结果相结合。对抗数据集的实验结果表明，我们的模型可以提高10 \％的性能，并改善原始和概括数据集。
Machine Reading Comprehension(MRC) has achieved a remarkable result since some powerful models, such as BERT, are proposed. However, these models are not robust enough and vulnerable to adversarial input perturbation and generalization examples. Some works tried to improve the performance on specific types of data by adding some related examples into training data while it leads to degradation on the original dataset, because the shift of data distribution makes the answer ranking based on the softmax probability of model unreliable. In this paper, we propose a method to improve the robustness by using a calibrator as the post-hoc reranker, which is implemented based on XGBoost model. The calibrator combines both manual features and representation learning features to rerank candidate results. Experimental results on adversarial datasets show that our model can achieve performance improvement by more than 10\% and also make improvement on the original and generalization datasets.
