论文标题
稀疏学习的统计保证
Statistical guarantees for sparse deep learning
论文作者
论文摘要
神经网络在应用中越来越流行，但是我们对其潜力和局限性的数学理解仍然有限。在本文中，我们通过为稀疏深度学习开发统计保证来进一步理解。与以前的工作相反，我们考虑了不同类型的稀疏性，例如很少的活动连接，很少的活动节点和其他基于规范的稀疏性。此外，我们的理论涵盖了以前理论所忽略的重要方面，例如多个输出，正则化和L2损失。保证对网络宽度和深度有轻微的依赖，这意味着它们从统计的角度支持稀疏但广泛的网络的应用。我们在派生中使用的一些概念和工具在深度学习中并不常见，因此可能会引起更多兴趣。
Neural networks are becoming increasingly popular in applications, but our mathematical understanding of their potential and limitations is still limited. In this paper, we further this understanding by developing statistical guarantees for sparse deep learning. In contrast to previous work, we consider different types of sparsity, such as few active connections, few active nodes, and other norm-based types of sparsity. Moreover, our theories cover important aspects that previous theories have neglected, such as multiple outputs, regularization, and l2-loss. The guarantees have a mild dependence on network widths and depths, which means that they support the application of sparse but wide and deep networks from a statistical perspective. Some of the concepts and tools that we use in our derivations are uncommon in deep learning and, hence, might be of additional interest.
