论文标题
评估句子和段落级别的文本连贯性
Evaluating Text Coherence at Sentence and Paragraph Levels
论文作者
论文摘要
在本文中，为了评估文本连贯性，我们提出了段落订购任务以及进行句子顺序的段落。我们从不同的域中收集了四个不同的语料库，我们研究了现有句子排序方法对段落订购任务的改编。我们还通过人工创建迷你数据集和嘈杂的数据集并在这种情况下验证已建立模型的效率，从而比较了现有模型的可学习性和鲁棒性。此外，我们对两个竞争模型的重新排列段落进行了人体评估，并确认WLCS-L是一种更好的指标，其与人类评级的相关性明显高于Tau，这是以前使用的最普遍的度量。这些评估的结果表明，除了某些极端条件外，基于图形神经网络的复发模型是连贯建模的最佳选择。
In this paper, to evaluate text coherence, we propose the paragraph ordering task as well as conducting sentence ordering. We collected four distinct corpora from different domains on which we investigate the adaptation of existing sentence ordering methods to a paragraph ordering task. We also compare the learnability and robustness of existing models by artificially creating mini datasets and noisy datasets respectively and verifying the efficiency of established models under these circumstances. Furthermore, we carry out human evaluation on the rearranged passages from two competitive models and confirm that WLCS-l is a better metric performing significantly higher correlations with human rating than tau, the most prevalent metric used before. Results from these evaluations show that except for certain extreme conditions, the recurrent graph neural network-based model is an optimal choice for coherence modeling.
