论文标题
HIT比率：标签建议的评估指标
Hit ratio: An Evaluation Metric for Hashtag Recommendation
论文作者
论文摘要
主题标签建议是一项至关重要的任务，尤其是在过去十年中使用Twitter等社交媒体平台的兴趣增加。标签建议系统在编写推文时自动向用户建议主题标签。主题标签建议领域的大多数研究都使用了经典指标，例如命中率，精度，召回和F1得分来衡量主题标签推荐系统的准确性。这些指标基于推荐的主题标签与相应的地面真相的确切匹配。但是，尚不清楚这些指标如何评估标签建议。我们有兴趣寻求答案的研究问题是：当推文中的地面真相主题标签的数量高度可变时，这些指标是否足以评估标签建议系统？在本文中，我们提出了一个新的指标，我们称其为主题标签建议的HIT比率。通过假设的示例和在一系列主题标签建议模型中的现实应用程序的广泛评估表明，HIT比是一个有用的指标。命中率与经典评估指标的比较揭示了它们的局限性。
Hashtag recommendation is a crucial task, especially with an increase of interest in using social media platforms such as Twitter in the last decade. Hashtag recommendation systems automatically suggest hashtags to a user while writing a tweet. Most of the research in the area of hashtag recommendation have used classical metrics such as hit rate, precision, recall, and F1-score to measure the accuracy of hashtag recommendation systems. These metrics are based on the exact match of the recommended hashtags with their corresponding ground truth. However, it is not clear how adequate these metrics to evaluate hashtag recommendation. The research question that we are interested in seeking an answer is: are these metrics adequate for evaluating hashtag recommendation systems when the numbers of ground truth hashtags in tweets are highly variable? In this paper, we propose a new metric which we call hit ratio for hashtag recommendation. Extensive evaluation through hypothetical examples and real-world application across a range of hashtag recommendation models indicate that the hit ratio is a useful metric. A comparison of hit ratio with the classical evaluation metrics reveals their limitations.
