论文标题
supert：在无监督的评估指标中朝着新的边界进行多文件摘要
SUPERT: Towards New Frontiers in Unsupervised Evaluation Metrics for Multi-Document Summarization
论文作者
论文摘要
我们研究了无监督的多文章摘要评估指标，这些指标既不需要人写的参考摘要，也不需要人类注释（例如偏好，评分等）。我们提出了SUPERT，该Supert使用上下文化的嵌入式和软令牌比对技术来评估其语义相似性，即从源文档中选择的显着句子，即从源文档中选定的显着句子。与最先进的无监督评估指标相比，Supert与人类评分的相关性增长了18-39％。此外，我们使用Supert作为奖励来指导基于神经的增强学习摘要，与最先进的无监督摘要相比，表现出色。所有源代码均可在https://github.com/yg211/acl20-ref-free-eval上找到。
We study unsupervised multi-document summarization evaluation metrics, which require neither human-written reference summaries nor human annotations (e.g. preferences, ratings, etc.). We propose SUPERT, which rates the quality of a summary by measuring its semantic similarity with a pseudo reference summary, i.e. selected salient sentences from the source documents, using contextualized embeddings and soft token alignment techniques. Compared to the state-of-the-art unsupervised evaluation metrics, SUPERT correlates better with human ratings by 18-39%. Furthermore, we use SUPERT as rewards to guide a neural-based reinforcement learning summarizer, yielding favorable performance compared to the state-of-the-art unsupervised summarizers. All source code is available at https://github.com/yg211/acl20-ref-free-eval.
