论文标题
通过序列到序列预训练改善AMR解析
Improving AMR Parsing with Sequence-to-Sequence Pre-training
论文作者
论文摘要
在文献中，关于抽象意义表示（AMR）解析的研究受到人类策划数据集的规模的限制，这对于建立具有良好性能的AMR解析器至关重要。为了减轻此类数据尺寸限制，预先训练的模型在AMR解析中引起了越来越多的关注。但是，以前的预培训模型（如BERT）是出于通用目的而实施的，这些模型可能无法按预期的是AMR解析的特定任务。在本文中，我们着重于序列到序列（SEQ2SEQ）AMR解析，并提出了一种SEQ2SEQ预训练方法，以在三个相关任务（即机器翻译，句法解析和AMR解析本身）上以单个和联合方式构建预训练的模型。此外，我们将香草微调方法扩展到多任务学习微调方法，该方法为AMR解析的性能优化，同时努力保留预训练模型的响应。两个英语基准数据集的广泛实验结果表明，单个和联合预训练的模型都显着提高了性能（例如，从AMR 2.0上的71.5到80.2），该模型达到了最新状态。结果非常令人鼓舞，因为我们使用SEQ2SEQ模型而不是复杂的模型实现了这一目标。我们在https://github.com/xdqkid/s2s-amr-parser上提供代码和模型。
In the literature, the research on abstract meaning representation (AMR) parsing is much restricted by the size of human-curated dataset which is critical to build an AMR parser with good performance. To alleviate such data size restriction, pre-trained models have been drawing more and more attention in AMR parsing. However, previous pre-trained models, like BERT, are implemented for general purpose which may not work as expected for the specific task of AMR parsing. In this paper, we focus on sequence-to-sequence (seq2seq) AMR parsing and propose a seq2seq pre-training approach to build pre-trained models in both single and joint way on three relevant tasks, i.e., machine translation, syntactic parsing, and AMR parsing itself. Moreover, we extend the vanilla fine-tuning method to a multi-task learning fine-tuning method that optimizes for the performance of AMR parsing while endeavors to preserve the response of pre-trained models. Extensive experimental results on two English benchmark datasets show that both the single and joint pre-trained models significantly improve the performance (e.g., from 71.5 to 80.2 on AMR 2.0), which reaches the state of the art. The result is very encouraging since we achieve this with seq2seq models rather than complex models. We make our code and model available at https://github.com/xdqkid/S2S-AMR-Parser.
