论文标题
多阶段影响函数
Multi-Stage Influence Function
论文作者
论文摘要
多阶段培训和知识转移，从大规模的预处理任务到各种填充任务，都彻底改变了自然语言处理和计算机视觉，从而改善了最新的绩效。在本文中，我们开发一个多阶段影响力得分，以跟踪从填充模型一直回到预处理数据的预测。有了这个分数，我们可以在预训练任务中确定训练示例，这些示例最大程度地促进了填充任务的预测。所提出的多阶段影响功能概括了（Koh＆Liang，2017）中单个模型的原始影响函数，从而通过预审预测的模型和填充模型来启用影响计算。我们研究了两种不同的方案，并在填充任务中固定或更新了验证的嵌入。我们在各种实验中测试了我们提出的方法，以显示其有效性和潜在应用。
Multi-stage training and knowledge transfer, from a large-scale pretraining task to various finetuning tasks, have revolutionized natural language processing and computer vision resulting in state-of-the-art performance improvements. In this paper, we develop a multi-stage influence function score to track predictions from a finetuned model all the way back to the pretraining data. With this score, we can identify the pretraining examples in the pretraining task that contribute most to a prediction in the finetuning task. The proposed multi-stage influence function generalizes the original influence function for a single model in (Koh & Liang, 2017), thereby enabling influence computation through both pretrained and finetuned models. We study two different scenarios with the pretrained embeddings fixed or updated in the finetuning tasks. We test our proposed method in various experiments to show its effectiveness and potential applications.
