论文标题
MPC友好的软磁更换的有效性
Effectiveness of MPC-friendly Softmax Replacement
论文作者
论文摘要
SoftMax广泛用于深度学习，以将某些表示形式映射到概率分布。由于Mohassel和Zhang（2017）基于在多方计算中相对昂贵的EXP/日志函数，因此提出了一个基于Relu的更简单的替代品，以用于安全计算中。但是，我们无法复制他们报告的对MNIST培训的准确性，并具有三个完全连接的层。后来的作品（例如Wagh等人，2019年和2021年）使用了SoftMax替换，而不是计算输出概率分布，而是用于近似后传播中的梯度。在这项工作中，我们分析了替换的两种用途，并将它们与SoftMax进行了比较，无论是在多方计算中的准确性和成本方面。我们发现，替换只为单层网络提供了显着的加速，而它总是会降低准确性，有时会大大降低。因此，我们得出的结论是，它的实用性受到限制，应该使用原始的SoftMax功能。
Softmax is widely used in deep learning to map some representation to a probability distribution. As it is based on exp/log functions that are relatively expensive in multi-party computation, Mohassel and Zhang (2017) proposed a simpler replacement based on ReLU to be used in secure computation. However, we could not reproduce the accuracy they reported for training on MNIST with three fully connected layers. Later works (e.g., Wagh et al., 2019 and 2021) used the softmax replacement not for computing the output probability distribution but for approximating the gradient in back-propagation. In this work, we analyze the two uses of the replacement and compare them to softmax, both in terms of accuracy and cost in multi-party computation. We found that the replacement only provides a significant speed-up for a one-layer network while it always reduces accuracy, sometimes significantly. Thus we conclude that its usefulness is limited and one should use the original softmax function instead.
