论文标题
随机复合材料非凸优化的最佳混合方差降低算法
An Optimal Hybrid Variance-Reduced Algorithm for Stochastic Composite Nonconvex Optimization
论文作者
论文摘要
在本说明中，我们提出了[7]中杂交方差降低近端方法的新变体，以在标准假设下求解常见的随机复合非凸优化问题。我们只需在[7]引入[7]引入的杂交sarah估计量中的独立无偏估计量，在同一样品上评估的随机梯度，从而导致[2]中引入的相同动量 - 萨拉估计器。与[7]相比，这使我们可以节省一个随机梯度，并且每个迭代只需要两个样本。我们的算法非常简单，并且可以在随机梯度评估（最多达到恒定因素）方面达到最佳随机甲骨文复杂性。我们的分析本质上是受[7]的启发，但我们不使用两个不同的步骤尺寸。
In this note we propose a new variant of the hybrid variance-reduced proximal gradient method in [7] to solve a common stochastic composite nonconvex optimization problem under standard assumptions. We simply replace the independent unbiased estimator in our hybrid- SARAH estimator introduced in [7] by the stochastic gradient evaluated at the same sample, leading to the identical momentum-SARAH estimator introduced in [2]. This allows us to save one stochastic gradient per iteration compared to [7], and only requires two samples per iteration. Our algorithm is very simple and achieves optimal stochastic oracle complexity bound in terms of stochastic gradient evaluations (up to a constant factor). Our analysis is essentially inspired by [7], but we do not use two different step-sizes.
