论文标题
深层网络优化算法的收敛保证的基本方法
An Elementary Approach to Convergence Guarantees of Optimization Algorithms for Deep Networks
论文作者
论文摘要
我们提出了一种基于基本参数和计算的深网络优化算法的收敛保证的方法。融合分析围绕着优化的分析和计算结构，这是在机器学习软件中实现深网的核心。我们提供了一种系统的方式来计算控制用于训练深网络的一阶优化算法的融合行为的平滑度常数的估计。在现代深网中产生的一组示例组件和体系结构插入了展示方法，以说明这种方法。
We present an approach to obtain convergence guarantees of optimization algorithms for deep networks based on elementary arguments and computations. The convergence analysis revolves around the analytical and computational structures of optimization oracles central to the implementation of deep networks in machine learning software. We provide a systematic way to compute estimates of the smoothness constants that govern the convergence behavior of first-order optimization algorithms used to train deep networks. A diverse set of example components and architectures arising in modern deep networks intersperse the exposition to illustrate the approach.
