论文标题
预训练的语言模型是否意识到短语？简单但坚固的语法诱导基线
Are Pre-trained Language Models Aware of Phrases? Simple but Strong Baselines for Grammar Induction
论文作者
论文摘要
随着预先训练的语言模型（LMS）在自然语言处理中的最新成功和普及，努力了解其内部运作的努力已经有所增加。我们提出了一种新型方法，可以帮助我们研究预培训的LMS捕获选区的句法概念的程度。我们的方法提供了一种有效的方法，可以在未经培训的情况下从预训练的LM中提取选区树。此外，我们报告了诱发树木中有趣的发现，包括预先训练的LMS在句子中正确划定副词短语的其他方法都超过了其他方法。
With the recent success and popularity of pre-trained language models (LMs) in natural language processing, there has been a rise in efforts to understand their inner workings. In line with such interest, we propose a novel method that assists us in investigating the extent to which pre-trained LMs capture the syntactic notion of constituency. Our method provides an effective way of extracting constituency trees from the pre-trained LMs without training. In addition, we report intriguing findings in the induced trees, including the fact that pre-trained LMs outperform other approaches in correctly demarcating adverb phrases in sentences.
