论文标题
对监测神经网络训练进度的重量空间的调查
An Investigation of the Weight Space to Monitor the Training Progress of Neural Networks
论文作者
论文摘要
安全使用深神经网络（DNN）需要仔细测试。但是，通常对部署的模型进行进一步培训以提高性能。由于严格的测试和评估很昂贵，因此需要触发器来确定模型的变化程度。在本文中，我们研究了可以利用到该目的的结构的DNN模型的重量空间。我们的结果表明，DNN模型在重量空间中的独特，平滑轨迹上发展，可用于跟踪DNN训练进度。我们假设轨迹的曲率和平稳性以及沿其长度的步长可能包含有关训练状态以及潜在域移动的信息。我们表明，模型轨迹可以分开，并且在恢复的轨迹上的检查点的顺序可以作为DNN模型版本的第一步。
Safe use of Deep Neural Networks (DNNs) requires careful testing. However, deployed models are often trained further to improve in performance. As rigorous testing and evaluation is expensive, triggers are in need to determine the degree of change of a model. In this paper we investigate the weight space of DNN models for structure that can be exploited to that end. Our results show that DNN models evolve on unique, smooth trajectories in weight space which can be used to track DNN training progress. We hypothesize that curvature and smoothness of the trajectories as well as step length along it may contain information on the state of training as well as potential domain shifts. We show that the model trajectories can be separated and the order of checkpoints on the trajectories recovered, which may serve as a first step towards DNN model versioning.
