论文标题
弹性重量巩固以获得更好的偏差接种
Elastic weight consolidation for better bias inoculation
论文作者
论文摘要
培训数据集中存在的偏见已显示出影响句子对分类任务的模型，例如自然语言推断（NLI）和事实验证。尽管已使用其他数据的微调模型来减轻它们，但一个常见的问题是灾难性忘记原始培训数据集的问题。在本文中，我们表明弹性重量合并（EWC）允许对模型进行微调来减轻偏见，同时又不太容易受到灾难性遗忘的影响。在我们对事实验证和NLI应力测试的评估中，我们表明，通过EWC进行微调主导了标准的微调，产生了模型，在原始（偏见）数据集上遗忘了较低的遗忘，以在微调（无偏见）数据集的准确性上获得同等的提高。
The biases present in training datasets have been shown to affect models for sentence pair classification tasks such as natural language inference (NLI) and fact verification. While fine-tuning models on additional data has been used to mitigate them, a common issue is that of catastrophic forgetting of the original training dataset. In this paper, we show that elastic weight consolidation (EWC) allows fine-tuning of models to mitigate biases while being less susceptible to catastrophic forgetting. In our evaluation on fact verification and NLI stress tests, we show that fine-tuning with EWC dominates standard fine-tuning, yielding models with lower levels of forgetting on the original (biased) dataset for equivalent gains in accuracy on the fine-tuning (unbiased) dataset.
