论文标题
ABC：沟通前的聚合，用于分布式图神经网络培训的沟通减少框架和有效分区
ABC: Aggregation before Communication, a Communication Reduction Framework for Distributed Graph Neural Network Training and Effective Partition
论文作者
论文摘要
图神经网络（GNN）是针对图形结构数据量身定制的神经模型家族，在图形结构数据的学习表示方面表现出了出色的性能。但是，大图上的训练GNN仍然具有挑战性，并且有希望的方向是分布的GNN训练，这是分配输入图并在多个计算机上分配工作负载。现有的分布式GNNS培训框架的关键瓶颈是依赖于GNNS的图形数据和聚合操作员引起的跨金融通信。在本文中，我们研究了分布式GNNS培训期间的沟通复杂性，并提出了一种简单的无损通信方法，称为沟通前的聚集（ABC）方法。 ABC方法利用了GNNS层的置换式属性，并导致一个范式，该范式被证明是比当前流行的范式（Edge-Cut）所接受的较高的沟通性能。此外，我们表明，在动态图的情况下，新的分区范式特别理想，因为由于图形变化过程的未知随机性而导致的边缘放置是不可行的。
Graph Neural Networks(GNNs) are a family of neural models tailored for graph-structure data and have shown superior performance in learning representations for graph-structured data. However, training GNNs on large graphs remains challenging and a promising direction is distributed GNN training, which is to partition the input graph and distribute the workload across multiple machines. The key bottleneck of the existing distributed GNNs training framework is the across-machine communication induced by the dependency on the graph data and aggregation operator of GNNs. In this paper, we study the communication complexity during distributed GNNs training and propose a simple lossless communication reduction method, termed the Aggregation before Communication (ABC) method. ABC method exploits the permutation-invariant property of the GNNs layer and leads to a paradigm where vertex-cut is proved to admit a superior communication performance than the currently popular paradigm (edge-cut). In addition, we show that the new partition paradigm is particularly ideal in the case of dynamic graphs where it is infeasible to control the edge placement due to the unknown stochastic of the graph-changing process.
