论文标题
降低方差的随机梯度跟踪算法，用于分散优化，并具有正交性约束
A Variance-Reduced Stochastic Gradient Tracking Algorithm for Decentralized Optimization with Orthogonality Constraints
论文作者
论文摘要
在科学计算和数据科学中广泛发现了具有正交性约束的分散优化。由于正交性的限制是非convex，因此设计有效的算法非常具有挑战性。现有方法利用从Riemannian优化的几何工具以高样本和通信复杂性为代价来解决此问题。为了减轻这一难度，基于两种可以放弃正交性约束的新技术，我们提出了一个方差减少的随机梯度跟踪（VRSGT）算法，其收敛速率为$ O（1 / K）$。据我们所知，VRSGT是分散优化的第一种算法，具有正交性约束，同时降低了采样和通信复杂性。在数值实验中，VRSGT在现实世界自动驾驶应用程序中具有有希望的性能。
Decentralized optimization with orthogonality constraints is found widely in scientific computing and data science. Since the orthogonality constraints are nonconvex, it is quite challenging to design efficient algorithms. Existing approaches leverage the geometric tools from Riemannian optimization to solve this problem at the cost of high sample and communication complexities. To relieve this difficulty, based on two novel techniques that can waive the orthogonality constraints, we propose a variance-reduced stochastic gradient tracking (VRSGT) algorithm with the convergence rate of $O(1 / k)$ to a stationary point. To the best of our knowledge, VRSGT is the first algorithm for decentralized optimization with orthogonality constraints that reduces both sampling and communication complexities simultaneously. In the numerical experiments, VRSGT has a promising performance in a real-world autonomous driving application.
