论文标题
打破架构障碍：一种有效的知识转移的方法
Breaking the Architecture Barrier: A Method for Efficient Knowledge Transfer Across Networks
论文作者
论文摘要
转移学习是改善神经网络性能的流行技术。但是，现有方法仅限于在具有相同体系结构的网络之间传输参数。我们提出了一种在具有不同体系结构的神经网络之间传输参数的方法。我们的方法称为DPIAT，使用动态编程来有效地匹配体系结构和传输参数之间的块和层。与现有的参数预测和随机初始化方法相比，它显着提高了训练效率和验证精度。在ImageNet的实验中，我们的方法在50个训练时期后平均提高了验证精度1.6倍。 DPIAT允许研究人员和神经体系结构搜索系统修改训练有素的网络并重用知识，从而避免了从头开始进行重新训练的需求。我们还引入了网络体系结构相似性度量，使用户能够在没有任何培训的情况下选择最佳的源网络。
Transfer learning is a popular technique for improving the performance of neural networks. However, existing methods are limited to transferring parameters between networks with same architectures. We present a method for transferring parameters between neural networks with different architectures. Our method, called DPIAT, uses dynamic programming to match blocks and layers between architectures and transfer parameters efficiently. Compared to existing parameter prediction and random initialization methods, it significantly improves training efficiency and validation accuracy. In experiments on ImageNet, our method improved validation accuracy by an average of 1.6 times after 50 epochs of training. DPIAT allows both researchers and neural architecture search systems to modify trained networks and reuse knowledge, avoiding the need for retraining from scratch. We also introduce a network architecture similarity measure, enabling users to choose the best source network without any training.
