论文标题
惯性随机棕榈（ISPALM）和机器学习中的应用
Inertial Stochastic PALM (iSPALM) and Applications in Machine Learning
论文作者
论文摘要
用于最小化非滑动和非凸的惯性算法作为惯性近端交替线性化最小化算法（IPALM）的功能表明，它们在计算时间上的优越性超过了非惯性变体。在成像和机器学习方面的许多问题中，目标功能具有特殊的形式，涉及大量数据，该数据鼓励使用随机算法。尽管基于随机梯度下降的算法仍用于大多数应用中，但最近提出了最小化非滑动和非凸功能的随机算法。在本文中，我们得出了一种随机棕榈算法的惯性变体，其降低了梯度估计量，称为ISPALM，并在某些假设下证明了该算法的线性收敛。我们的惯性方法可以看作是广泛使用的动量方法的概括，以加快和稳定优化算法，尤其是在机器学习中，以至于非平滑问题。学习所谓近端神经网络的权重和Student-T混合物模型的参数的数值实验表明，我们的新算法的表现都优于随机棕榈及其确定性对应物。
Inertial algorithms for minimizing nonsmooth and nonconvex functions as the inertial proximal alternating linearized minimization algorithm (iPALM) have demonstrated their superiority with respect to computation time over their non inertial variants. In many problems in imaging and machine learning, the objective functions have a special form involving huge data which encourage the application of stochastic algorithms. While algorithms based on stochastic gradient descent are still used in the majority of applications, recently also stochastic algorithms for minimizing nonsmooth and nonconvex functions were proposed. In this paper, we derive an inertial variant of a stochastic PALM algorithm with variance-reduced gradient estimator, called iSPALM, and prove linear convergence of the algorithm under certain assumptions. Our inertial approach can be seen as generalization of momentum methods widely used to speed up and stabilize optimization algorithms, in particular in machine learning, to nonsmooth problems. Numerical experiments for learning the weights of a so-called proximal neural network and the parameters of Student-t mixture models show that our new algorithm outperforms both stochastic PALM and its deterministic counterparts.
