论文标题
Depara：深层知识转移性的深度归因图
DEPARA: Deep Attribution Graph for Deep Knowledge Transferability
论文作者
论文摘要
探索在训练的深层神经网络（PR-DNN）中编码的异质任务的知识之间的内在互连，这阐明了它们的相互转移性，因此可以使知识从一项任务转移到另一个任务，以减少后者的训练工作。在本文中，我们提出了深层归因图（DEPARA）来研究从PR-DNNS中学到的知识的可传递性。在Depara中，节点对应于输入，并由其矢量归因图表示，以pr-dnn的输出为代表。边缘表示输入之间的相关性，并通过从PR-DNN中提取的特征的相似性来衡量。两个PR-DNN的知识转移性是通过其相应Deparas的相似性来衡量的。我们将depara应用于传输学习中的两个重要但研究不足的问题：预训练的模型选择和层选择。进行了广泛的实验，以证明拟议方法解决这两个问题的有效性和优越性。代码，数据和模型在本文中重现了结果，请参见\ url {https://github.com/zju-vipa/depara}。
Exploring the intrinsic interconnections between the knowledge encoded in PRe-trained Deep Neural Networks (PR-DNNs) of heterogeneous tasks sheds light on their mutual transferability, and consequently enables knowledge transfer from one task to another so as to reduce the training effort of the latter. In this paper, we propose the DEeP Attribution gRAph (DEPARA) to investigate the transferability of knowledge learned from PR-DNNs. In DEPARA, nodes correspond to the inputs and are represented by their vectorized attribution maps with regards to the outputs of the PR-DNN. Edges denote the relatedness between inputs and are measured by the similarity of their features extracted from the PR-DNN. The knowledge transferability of two PR-DNNs is measured by the similarity of their corresponding DEPARAs. We apply DEPARA to two important yet under-studied problems in transfer learning: pre-trained model selection and layer selection. Extensive experiments are conducted to demonstrate the effectiveness and superiority of the proposed method in solving both these problems. Code, data and models reproducing the results in this paper are available at \url{https://github.com/zju-vipa/DEPARA}.
