论文标题
课程学习，以进行大量检索蒸馏
Curriculum Learning for Dense Retrieval Distillation
论文作者
论文摘要
最近的工作表明，可以通过从现有基础重新排列模型中提取排名知识来获得更有效的密集检索模型。在本文中，我们提出了一个名为CL-DRD的基于通用课程的优化框架，该框架控制了由重新排列（教师）模型产生的培训数据的难度。 CL-DRD迭代通过增加可用的知识蒸馏数据的难度来优化密集检索（学生）模型。更详细地，我们最初提供了学生模型的粗粒偏好对，教师排名中的文档之间，并逐步朝着更细粒度的成对文档订购要求迈进。在我们的实验中，我们采用了简单的CL-DRD框架实现，以增强两个最先进的密集检索模型。在三个公共通道检索数据集上进行的实验证明了我们提出的框架的有效性。
Recent work has shown that more effective dense retrieval models can be obtained by distilling ranking knowledge from an existing base re-ranking model. In this paper, we propose a generic curriculum learning based optimization framework called CL-DRD that controls the difficulty level of training data produced by the re-ranking (teacher) model. CL-DRD iteratively optimizes the dense retrieval (student) model by increasing the difficulty of the knowledge distillation data made available to it. In more detail, we initially provide the student model coarse-grained preference pairs between documents in the teacher's ranking and progressively move towards finer-grained pairwise document ordering requirements. In our experiments, we apply a simple implementation of the CL-DRD framework to enhance two state-of-the-art dense retrieval models. Experiments on three public passage retrieval datasets demonstrate the effectiveness of our proposed framework.
