论文标题
TornadoAggregate：通过基于环的架构进行准确且可扩展的联合学习
TornadoAggregate: Accurate and Scalable Federated Learning via the Ring-Based Architecture
论文作者
论文摘要
联合学习已成为协作机器学习的新范式。但是，许多先前的研究已经沿着星形拓扑使用了全球聚合，而没有太多考虑沟通可伸缩性或依赖客户当地时代品种的昼夜属性。相比之下，环形架构可以通过迭代节点而无需聚合来解决义务问题，甚至可以满足昼夜性能。然而，这种基于环的算法本质上会遭受高变化问题。为此，我们提出了一种称为TornadoAggregate的新型算法，该算法通过促进环构建来提高准确性和可扩展性。特别是，为了提高准确性，我们将损失最小化为减少方差问题，并建立了三个降低方差的原则：环形感知分组，小环和环链链。实验结果表明，TornadoAggregate提高了测试准确性高达26.7％，并达到了接近线性的可扩展性。
Federated learning has emerged as a new paradigm of collaborative machine learning; however, many prior studies have used global aggregation along a star topology without much consideration of the communication scalability or the diurnal property relied on clients' local time variety. In contrast, ring architecture can resolve the scalability issue and even satisfy the diurnal property by iterating nodes without an aggregation. Nevertheless, such ring-based algorithms can inherently suffer from the high-variance problem. To this end, we propose a novel algorithm called TornadoAggregate that improves both accuracy and scalability by facilitating the ring architecture. In particular, to improve the accuracy, we reformulate the loss minimization into a variance reduction problem and establish three principles to reduce variance: Ring-Aware Grouping, Small Ring, and Ring Chaining. Experimental results show that TornadoAggregate improved the test accuracy by up to 26.7% and achieved near-linear scalability.
