论文标题
深线性神经网络中的冗余
Redundancy in Deep Linear Neural Networks
论文作者
论文摘要
传统观念指出，深层线性神经网络受益于单个线性层的表现力和优化优势。本文表明，在实践中，使用常规优化器的深线性完全连接网络的训练过程与单个线性完全连接的层相同。本文旨在解释这一主张并证明这一点。即使卷积网络与此描述不符，但这项工作旨在获得对完全连接的线性网络的新概念理解，该网络可能会阐明卷积设置和非线性体系结构的可能约束。
Conventional wisdom states that deep linear neural networks benefit from expressiveness and optimization advantages over a single linear layer. This paper suggests that, in practice, the training process of deep linear fully-connected networks using conventional optimizers is convex in the same manner as a single linear fully-connected layer. This paper aims to explain this claim and demonstrate it. Even though convolutional networks are not aligned with this description, this work aims to attain a new conceptual understanding of fully-connected linear networks that might shed light on the possible constraints of convolutional settings and non-linear architectures.
