论文标题
学习最近的邻居神经机器翻译的解耦检索表示
Learning Decoupled Retrieval Representation for Nearest Neighbour Neural Machine Translation
论文作者
论文摘要
K-Nearest邻居神经机器翻译（KNN-MT）通过在测试时间检索单词级表示，成功地纳入了外部语料库。通常，KNN-MT在翻译任务中借用了现成的上下文表示，例如最后一个解码器层的输出，作为检索任务的查询向量。在这项工作中，我们强调说，将这两个任务的表示形式结合起来是优质检索的最佳选择。为了减轻它，我们利用受监督的对比学习来学习从原始上下文表示获得的独特检索表示。我们还提出了一种快速有效的方法来构建硬性样本。在五个领域的实验结果表明，与香草KNN-MT相比，我们的方法提高了检索准确性和BLEU评分。
K-Nearest Neighbor Neural Machine Translation (kNN-MT) successfully incorporates external corpus by retrieving word-level representations at test time. Generally, kNN-MT borrows the off-the-shelf context representation in the translation task, e.g., the output of the last decoder layer, as the query vector of the retrieval task. In this work, we highlight that coupling the representations of these two tasks is sub-optimal for fine-grained retrieval. To alleviate it, we leverage supervised contrastive learning to learn the distinctive retrieval representation derived from the original context representation. We also propose a fast and effective approach to constructing hard negative samples. Experimental results on five domains show that our approach improves the retrieval accuracy and BLEU score compared to vanilla kNN-MT.
