论文标题
跨语性摘要的混合语言预培训
Mixed-Lingual Pre-training for Cross-lingual Summarization
论文作者
论文摘要
跨语性摘要（CLS）旨在用源语言的文章进行目标语言摘要。传统解决方案采用两步方法，即翻译，然后总结或总结一下，然后翻译。最近，端到端模型取得了更好的结果，但是这些方法大多受其对大规模标记数据的依赖的限制。我们提出了一种基于混合语言预训练的解决方案，该解决方案利用了跨语性任务，例如翻译和单语言任务，例如蒙版语言模型。因此，我们的模型可以利用大量的单语言数据来增强其语言建模。此外，体系结构没有特定于任务的组件，可以节省内存并提高优化效率。我们在实验中表明，这种预训练方案可以有效地提高跨语性摘要的性能。在神经跨语性摘要（NCLS）数据集中，我们的模型可取得2.82（英语至中文）和1.15（英语至英文）roge-1的成绩，而不是最先进的结果。
Cross-lingual Summarization (CLS) aims at producing a summary in the target language for an article in the source language. Traditional solutions employ a two-step approach, i.e. translate then summarize or summarize then translate. Recently, end-to-end models have achieved better results, but these approaches are mostly limited by their dependence on large-scale labeled data. We propose a solution based on mixed-lingual pre-training that leverages both cross-lingual tasks such as translation and monolingual tasks like masked language models. Thus, our model can leverage the massive monolingual data to enhance its modeling of language. Moreover, the architecture has no task-specific components, which saves memory and increases optimization efficiency. We show in experiments that this pre-training scheme can effectively boost the performance of cross-lingual summarization. In Neural Cross-Lingual Summarization (NCLS) dataset, our model achieves an improvement of 2.82 (English to Chinese) and 1.15 (Chinese to English) ROUGE-1 scores over state-of-the-art results.
