论文标题
FLCD：编码分布式计算的灵活的低复杂性设计
FLCD: A Flexible Low Complexity Design of Coded Distributed Computing
论文作者
论文摘要
我们提出了在亚马逊弹性计算云上进行经验评估（Amazon EC2）的编码分布式计算（CDC）的灵活的低复杂性设计（FLCD）。 CDC可以通过交易增加地图计算来加快MAPREDUCE，例如计算，以减少通信负载和洗牌时间。 FLCD的主要新颖性是利用设计自由来定义映射并减少功能以开发渐近均匀的系统，以在一般的MapReduce框架下支持不同的中间值（IV）尺寸。与现有IV尺寸的现有设计相比，FLCD在适应网络参数方面具有更大的灵活性，并通过更少的输入文件和混音组来大大降低实现复杂性。 FLCD方案是第一个提出的低复杂性CDC设计，可以在具有任意数量的节点和计算负载的网络上运行。我们通过在Amazon EC2群集上执行Terasort算法对FLCD进行经验评估。这是第一次通过经验评估来验证CDC洗牌时间的理论预测。与常规的未编码MAPREDUCE相比，评估表明2.0至4.24倍的速度，总时间减少了12％至52％，与现有CDC方案相比，操作网络参数的范围更大。
We propose a flexible low complexity design (FLCD) of coded distributed computing (CDC) with empirical evaluation on Amazon Elastic Compute Cloud (Amazon EC2). CDC can expedite MapReduce like computation by trading increased map computations to reduce communication load and shuffle time. A main novelty of FLCD is to utilize the design freedom in defining map and reduce functions to develop asymptotic homogeneous systems to support varying intermediate values (IV) sizes under a general MapReduce framework. Compared to existing designs with constant IV sizes, FLCD offers greater flexibility in adapting to network parameters and significantly reduces the implementation complexity by requiring fewer input files and shuffle groups. The FLCD scheme is the first proposed low-complexity CDC design that can operate on a network with an arbitrary number of nodes and computation load. We perform empirical evaluations of the FLCD by executing the TeraSort algorithm on an Amazon EC2 cluster. This is the first time that theoretical predictions of the CDC shuffle time are validated by empirical evaluations. The evaluations demonstrate a 2.0 to 4.24x speedup compared to conventional uncoded MapReduce, a 12% to 52% reduction in total time, and a wider range of operating network parameters compared to existing CDC schemes.
