论文标题
convai3挑战中NTES_ALONG的澄清问题选择系统
A Clarifying Question Selection System from NTES_ALONG in Convai3 Challenge
论文作者
论文摘要
本文介绍了2020年在面向搜索的对话AI（SCAI）EMNLP研讨会上参加Clariq挑战的NetEase游戏AI实验室团队的参与。挑战要求建立一个完整的对话信息检索系统，以理解和生成澄清问题。我们提出了一个澄清的问题选择系统，该系统包括回答理解，候选问题回忆和澄清问题排名。我们微调了罗伯塔（Roberta）模型，以了解用户的响应，并使用增强的BM25模型来回顾候选问题。在澄清问题排名阶段时，我们重建了训练数据集，并提出了基于Electra的两个模型。最后，我们通过总结其输出概率并以最高概率作为澄清问题来结合模型。实验表明，我们的整体排名模型在相关任务中优于文档中的表现，并实现了最佳召回@[20,30]相关任务的指标。在阶段2中的多转交谈评估中，我们的系统达到了所有文档相关指标的最高分数。
This paper presents the participation of NetEase Game AI Lab team for the ClariQ challenge at Search-oriented Conversational AI (SCAI) EMNLP workshop in 2020. The challenge asks for a complete conversational information retrieval system that can understanding and generating clarification questions. We propose a clarifying question selection system which consists of response understanding, candidate question recalling and clarifying question ranking. We fine-tune a RoBERTa model to understand user's responses and use an enhanced BM25 model to recall the candidate questions. In clarifying question ranking stage, we reconstruct the training dataset and propose two models based on ELECTRA. Finally we ensemble the models by summing up their output probabilities and choose the question with the highest probability as the clarification question. Experiments show that our ensemble ranking model outperforms in the document relevance task and achieves the best recall@[20,30] metrics in question relevance task. And in multi-turn conversation evaluation in stage2, our system achieve the top score of all document relevance metrics.
