论文标题
神经网络的全球景观：概述
The Global Landscape of Neural Networks: An Overview
论文作者
论文摘要
神经网络培训的主要问题之一是，相关损失功能的非跨性别性可能会导致不良景观。神经网络的最新成功表明，他们的损失格局还不错，但是我们对景观有什么具体结果？在本文中，我们回顾了有关神经网络全球景观的最新发现和结果。首先，我们指出，在某些假设下，广泛的神经网可能具有最佳的局部最小值。其次，我们讨论了有关宽网络的几何特性的一些严格结果，例如“不糟糕的盆地”，以及一些消除了亚最佳局部最小值和/或降低无穷大的路径的修改。第三，我们讨论了实用神经网的景观的可视化和经验探索。最后，我们简要讨论了一些融合结果及其与景观结果的关系。
One of the major concerns for neural network training is that the non-convexity of the associated loss functions may cause bad landscape. The recent success of neural networks suggests that their loss landscape is not too bad, but what specific results do we know about the landscape? In this article, we review recent findings and results on the global landscape of neural networks. First, we point out that wide neural nets may have sub-optimal local minima under certain assumptions. Second, we discuss a few rigorous results on the geometric properties of wide networks such as "no bad basin", and some modifications that eliminate sub-optimal local minima and/or decreasing paths to infinity. Third, we discuss visualization and empirical explorations of the landscape for practical neural nets. Finally, we briefly discuss some convergence results and their relation to landscape results.
