论文标题
联合学习的上行链路和下行链路通信的设计和分析
Design and Analysis of Uplink and Downlink Communications for Federated Learning
论文作者
论文摘要
众所周知，沟通是联邦学习（FL）的主要瓶颈之一，但现有的研究尚未解决高效的通信设计，尤其是在无线FL中，必须考虑上行链路和下行链路通信。在本文中，我们专注于无线FL的物理层量化和传输方法的设计和分析。我们回答有关客户与参数服务器之间的交流以及如何交流的问题，并评估更新模型的各种量化和传输选项对学习性能的影响。我们在非i.i.d下提供了众所周知的FedAvg的新收敛分析。在上行链路和下行链路通信中，数据集分布，部分客户的参与以及有限的量化量化。这些分析表明，为了实现O（1/T）收敛速率，使用量化率，可以以对数速率以对数速率提高量化水平，同时传输重量差异可以保持恒定的量化水平。对各种现实世界数据集进行的全面数值评估表明，FL-Tail-Tail-Tail-Tail-Tail-Tail-Tail-Tail-Tail-Tail-Tail-Tail-Tail-link沟通设计的好处是巨大的 - 精心设计的量化和传输实现了浮点数基线精度的98％以上，而基线频带少于10％的浮点数基线精度，而两位基线带宽的实验率是I.I.D的主要实验。和non-i.i.d。数据集。特别是，1位量化（占浮点基线带宽的3.1％）以几乎相同的MNIST收敛速率实现了99.8％的浮点基线准确性，这代表了最著名的带宽准确性权衡取舍，这是作者所知的最佳知识。
Communication has been known to be one of the primary bottlenecks of federated learning (FL), and yet existing studies have not addressed the efficient communication design, particularly in wireless FL where both uplink and downlink communications have to be considered. In this paper, we focus on the design and analysis of physical layer quantization and transmission methods for wireless FL. We answer the question of what and how to communicate between clients and the parameter server and evaluate the impact of the various quantization and transmission options of the updated model on the learning performance. We provide new convergence analysis of the well-known FedAvg under non-i.i.d. dataset distributions, partial clients participation, and finite-precision quantization in uplink and downlink communications. These analyses reveal that, in order to achieve an O(1/T) convergence rate with quantization, transmitting the weight requires increasing the quantization level at a logarithmic rate, while transmitting the weight differential can keep a constant quantization level. Comprehensive numerical evaluation on various real-world datasets reveals that the benefit of a FL-tailored uplink and downlink communication design is enormous - a carefully designed quantization and transmission achieves more than 98% of the floating-point baseline accuracy with fewer than 10% of the baseline bandwidth, for majority of the experiments on both i.i.d. and non-i.i.d. datasets. In particular, 1-bit quantization (3.1% of the floating-point baseline bandwidth) achieves 99.8% of the floating-point baseline accuracy at almost the same convergence rate on MNIST, representing the best known bandwidth-accuracy tradeoff to the best of the authors' knowledge.
