论文标题
自动编码器在培训和初始化中的特征值
Eigenvalues of Autoencoders in Training and at Initialization
论文作者
论文摘要
在本文中，我们研究了自动编码器在初始化附近的演变。特别是，我们研究了训练过程的早期，在MNIST数据集的培训中，研究了自动编码器的雅各布矩阵的特征值的分布。我们发现，尚未接受过培训的自动编码器具有与经过长期培训的特征值分布（$> $ 100时代）。此外，我们发现，即使在早期时期，这些特征值分布也与完全训练的自动编码器的特征值迅速相似。我们还将初始化时的特征值与随机矩阵和此类矩阵产物的特征值相关的理论工作进行了比较。
In this paper, we investigate the evolution of autoencoders near their initialization. In particular, we study the distribution of the eigenvalues of the Jacobian matrices of autoencoders early in the training process, training on the MNIST data set. We find that autoencoders that have not been trained have eigenvalue distributions that are qualitatively different from those which have been trained for a long time ($>$100 epochs). Additionally, we find that even at early epochs, these eigenvalue distributions rapidly become qualitatively similar to those of the fully trained autoencoders. We also compare the eigenvalues at initialization to pertinent theoretical work on the eigenvalues of random matrices and the products of such matrices.
