论文标题
Maven：大型通用域事件检测数据集
MAVEN: A Massive General Domain Event Detection Dataset
论文作者
论文摘要
事件检测（ED），这意味着识别事件触发单词和分类事件类型，是从纯文本中提取事件知识的第一个也是最基本的步骤。大多数现有的数据集都表现出以下问题，这些问题限制了ED的进一步发展：（1）数据稀缺。现有的小规模数据集不足以训练和稳定基准测试越来越复杂的现代神经方法。 （2）低覆盖范围。现有数据集的有限事件类型不能很好地涵盖限制ED模型应用的通用域事件。为了减轻这些问题，我们提出了一个大规模的事件检测数据集（MAVEN），其中包含4,480个Wikipedia文档，118,732事件提及实例和168个事件类型。 Maven减轻了数据稀缺问题，并涵盖了更多一般的事件类型。我们重现了最新的最新模型，并对Maven进行了彻底的评估。实验结果表明，现有的ED方法无法像小型数据集一样在Maven上获得有希望的结果，这表明现实世界中的ED仍然是一项艰巨的任务，需要进一步的研究工作。我们还通过经验分析讨论了通用领域的进一步方向。可以从https://github.com/thu-keg/maven-dataset获得源代码和数据集。
Event detection (ED), which means identifying event trigger words and classifying event types, is the first and most fundamental step for extracting event knowledge from plain text. Most existing datasets exhibit the following issues that limit further development of ED: (1) Data scarcity. Existing small-scale datasets are not sufficient for training and stably benchmarking increasingly sophisticated modern neural methods. (2) Low coverage. Limited event types of existing datasets cannot well cover general-domain events, which restricts the applications of ED models. To alleviate these problems, we present a MAssive eVENt detection dataset (MAVEN), which contains 4,480 Wikipedia documents, 118,732 event mention instances, and 168 event types. MAVEN alleviates the data scarcity problem and covers much more general event types. We reproduce the recent state-of-the-art ED models and conduct a thorough evaluation on MAVEN. The experimental results show that existing ED methods cannot achieve promising results on MAVEN as on the small datasets, which suggests that ED in the real world remains a challenging task and requires further research efforts. We also discuss further directions for general domain ED with empirical analyses. The source code and dataset can be obtained from https://github.com/THU-KEG/MAVEN-dataset.
