论文标题
通过本地电源迭代进行沟通效率的分布式SVD
Communication-Efficient Distributed SVD via Local Power Iterations
论文作者
论文摘要
我们研究了截短的奇异值分解问题的分布计算。我们开发了一种称为\ texttt {localpower}的算法，以提高通信效率。具体而言，我们将数据集均匀地分配在$ m $节点之间，并在多个（确切的$ p $）本地电源迭代和一个全球聚合之间进行替代。在汇总中，我们建议用正交procrustes转换（OPT）加权每个局部特征向量矩阵。作为OPT，符号固定的实际替代物，它使用具有$ \ pm 1 $条目的对角线矩阵作为权重，在实验中具有更好的计算复杂性和稳定性。从理论上讲，我们表明，在某些假设下\ texttt {localpower}将所需的通信数降低了$ p $，以达到恒定的精度。我们还表明，定期衰减$ p $的策略有助于获得高精度解决方案。我们进行实验以证明\ texttt {localpower}的有效性。
We study distributed computing of the truncated singular value decomposition problem. We develop an algorithm that we call \texttt{LocalPower} for improving communication efficiency. Specifically, we uniformly partition the dataset among $m$ nodes and alternate between multiple (precisely $p$) local power iterations and one global aggregation. In the aggregation, we propose to weight each local eigenvector matrix with orthogonal Procrustes transformation (OPT). As a practical surrogate of OPT, sign-fixing, which uses a diagonal matrix with $\pm 1$ entries as weights, has better computation complexity and stability in experiments. We theoretically show that under certain assumptions \texttt{LocalPower} lowers the required number of communications by a factor of $p$ to reach a constant accuracy. We also show that the strategy of periodically decaying $p$ helps obtain high-precision solutions. We conduct experiments to demonstrate the effectiveness of \texttt{LocalPower}.
