论文标题

参数挖掘驱动的同行评审分析

Argument Mining Driven Analysis of Peer-Reviews

论文作者

Fromm, Michael, Faerman, Evgeniy, Berrendorf, Max, Bhargava, Siddharth, Qi, Ruoxia, Zhang, Yao, Dennert, Lukas, Selle, Sophia, Mao, Yang, Seidl, Thomas

论文摘要

同行审查是现代研究中的一个中心过程,对于确保已发表工作的高质量和可靠性至关重要。同时,这是一个耗时的过程,对新兴领域的兴趣日益增加,通常会导致审查工作量很高,尤其是对于该领域的高级研究人员而言。如何解决这个问题是一个悬而未决的问题,在所有主要会议上都会生动地讨论它。在这项工作中,我们提出了一种基于论点挖掘的方法,以帮助编辑,元评论者和审阅者。我们证明,科学出版物领域的决策过程是由论点驱动的,自动参数识别对各种用例有帮助。我们的发现之一是,同行评审过程中使用的参数不同于其他域中的参数,使预训练模型的转移变得困难。因此,我们从不同的计算机科学会议上为社区提供了带有注释论点的新的同行评审数据集。在我们广泛的经验评估中,我们表明可以使用论证挖掘来有效地从评论中提取最相关的部分,这对于出版决策至关重要。该过程仍然可以解释,因为提取的论点可以在审查中突出显示而不将其从上下文中分离出来。

Peer reviewing is a central process in modern research and essential for ensuring high quality and reliability of published work. At the same time, it is a time-consuming process and increasing interest in emerging fields often results in a high review workload, especially for senior researchers in this area. How to cope with this problem is an open question and it is vividly discussed across all major conferences. In this work, we propose an Argument Mining based approach for the assistance of editors, meta-reviewers, and reviewers. We demonstrate that the decision process in the field of scientific publications is driven by arguments and automatic argument identification is helpful in various use-cases. One of our findings is that arguments used in the peer-review process differ from arguments in other domains making the transfer of pre-trained models difficult. Therefore, we provide the community with a new peer-review dataset from different computer science conferences with annotated arguments. In our extensive empirical evaluation, we show that Argument Mining can be used to efficiently extract the most relevant parts from reviews, which are paramount for the publication decision. The process remains interpretable since the extracted arguments can be highlighted in a review without detaching them from their context.

扫码加入交流群

加入微信交流群

微信交流群二维码

扫码加入学术交流群,获取更多资源