论文标题
理论分析对神经网络加深的优势
Theoretical Analysis of the Advantage of Deepening Neural Networks
论文作者
论文摘要
我们提出了两个新标准，以了解加深神经网络的优势。重要的是要了解深层神经网络可计算的功能的表现力，以了解加深神经网络的优势。除非深层神经网络具有足够的表现力，否则即使学习成功，它们也无法具有良好的表现。在这种情况下，提出的标准有助于理解加深神经网络的优势，因为它们可以独立于学习效率来评估表达能力。第一个标准显示了深神经网络与目标函数的近似准确性。该标准的背景是深度学习的目标是通过深神经网络近似目标功能。第二个标准显示了通过深神经网络可计算的函数线性区域的特性。该标准的背景是，激活函数是分段线性的深神经网络也是分段线性的。此外，根据这两个标准，我们表明，增加层比在提高深层神经网络的表达性方面增加单位更有效。
We propose two new criteria to understand the advantage of deepening neural networks. It is important to know the expressivity of functions computable by deep neural networks in order to understand the advantage of deepening neural networks. Unless deep neural networks have enough expressivity, they cannot have good performance even though learning is successful. In this situation, the proposed criteria contribute to understanding the advantage of deepening neural networks since they can evaluate the expressivity independently from the efficiency of learning. The first criterion shows the approximation accuracy of deep neural networks to the target function. This criterion has the background that the goal of deep learning is approximating the target function by deep neural networks. The second criterion shows the property of linear regions of functions computable by deep neural networks. This criterion has the background that deep neural networks whose activation functions are piecewise linear are also piecewise linear. Furthermore, by the two criteria, we show that to increase layers is more effective than to increase units at each layer on improving the expressivity of deep neural networks.
