论文标题
关于基于BERT的关系提取的鲁棒性和偏差分析
On Robustness and Bias Analysis of BERT-based Relation Extraction
论文作者
论文摘要
微调预训练的模型在标准的自然语言处理基准上取得了令人印象深刻的性能。但是，由此产生的模型推广性仍然鲜为人知。例如，我们不知道出色的性能如何导致概括模型的完美。在这项研究中，我们使用关系提取从不同的角度分析了一个微调的BERT模型。我们还根据提议的改进来表征泛化技术的差异。从经验实验中，我们发现，伯特通过随机化，对抗和反事实检验以及偏见（即选择和语义）的方式侵略了瓶颈。这些发现突出了未来改进的机会。我们的开源测试床诊断可在\ url {https://github.com/zjunlp/diagnosere}中获得。
Fine-tuning pre-trained models have achieved impressive performance on standard natural language processing benchmarks. However, the resultant model generalizability remains poorly understood. We do not know, for example, how excellent performance can lead to the perfection of generalization models. In this study, we analyze a fine-tuned BERT model from different perspectives using relation extraction. We also characterize the differences in generalization techniques according to our proposed improvements. From empirical experimentation, we find that BERT suffers a bottleneck in terms of robustness by way of randomizations, adversarial and counterfactual tests, and biases (i.e., selection and semantic). These findings highlight opportunities for future improvements. Our open-sourced testbed DiagnoseRE is available in \url{https://github.com/zjunlp/DiagnoseRE}.
