论文标题
对话：通过学习恢复和排名话语的话语感知的响应产生
DialogBERT: Discourse-Aware Response Generation via Learning to Recover and Rank Utterances
论文作者
论文摘要
预训练的语言模型的最新进展已大大改善了神经反应的产生。但是，现有方法通常将对话上下文视为令牌的线性序列，并学会通过令牌级别的自我注意来生成下一个单词。这样的代币层面编码的层次阻碍了话语之间的话语级连贯性的探索。本文介绍了Dialogbert，这是一种新型的对话响应生成模型，可增强以前的基于PLM的对话模型。 Dialogbert采用了分层变压器体系结构。为了有效地捕捉话语之间的话语级别的连贯性，我们提出了两个训练目标，包括掩盖的话语回归和分布式的话语顺序与原始BERT培训类似。在三个多转交谈数据集上进行的实验表明，我们的方法在定量评估方面非常优于基本线，例如Bart和Dialogpt。人类评估表明，对话框比具有显着余量的基准产生更连贯，信息性和类似人类的反应。
Recent advances in pre-trained language models have significantly improved neural response generation. However, existing methods usually view the dialogue context as a linear sequence of tokens and learn to generate the next word through token-level self-attention. Such token-level encoding hinders the exploration of discourse-level coherence among utterances. This paper presents DialogBERT, a novel conversational response generation model that enhances previous PLM-based dialogue models. DialogBERT employs a hierarchical Transformer architecture. To efficiently capture the discourse-level coherence among utterances, we propose two training objectives, including masked utterance regression and distributed utterance order ranking in analogy to the original BERT training. Experiments on three multi-turn conversation datasets show that our approach remarkably outperforms the baselines, such as BART and DialoGPT, in terms of quantitative evaluation. The human evaluation suggests that DialogBERT generates more coherent, informative, and human-like responses than the baselines with significant margins.
