论文标题
自然语言理解中大型语言模型的快捷方式学习
Shortcut Learning of Large Language Models in Natural Language Understanding
论文作者
论文摘要
大型语言模型（LLM）已在一系列自然语言理解任务上实现了最先进的表现。但是，这些LLM可能依靠数据集偏差和文物作为预测的快捷方式。这严重影响了他们的普遍性和对抗性鲁棒性。在本文中，我们对最近的发展方案进行了回顾，这些发展涉及LLMS的快捷方式学习和鲁棒性挑战。我们首先介绍了语言模型的快捷方式学习的概念。然后，我们介绍了在语言模型中识别快捷方式学习行为的方法，表征快捷方式学习的原因以及引入缓解解决方案。最后，我们讨论了关键的研究挑战和潜在的研究方向，以推动LLM的领域。
Large language models (LLMs) have achieved state-of-the-art performance on a series of natural language understanding tasks. However, these LLMs might rely on dataset bias and artifacts as shortcuts for prediction. This has significantly affected their generalizability and adversarial robustness. In this paper, we provide a review of recent developments that address the shortcut learning and robustness challenge of LLMs. We first introduce the concepts of shortcut learning of language models. We then introduce methods to identify shortcut learning behavior in language models, characterize the reasons for shortcut learning, as well as introduce mitigation solutions. Finally, we discuss key research challenges and potential research directions in order to advance the field of LLMs.
