论文标题
辅助：学习高斯模式的对称混合物，以改善定量量化
SYMOG: learning symmetric mixture of Gaussian modes for improved fixed-point quantization
论文作者
论文摘要
深度神经网络（DNNS）已被证明可以优于几种机器学习基准的经典方法。但是，它们具有很高的计算复杂性，需要强大的处理单元。尤其是在嵌入式系统上部署时，必须大大减少模型大小和推理时间。我们提出了Symog（高斯模式的对称混合物），这显着降低了DNN的复杂性，通过低位固定点量化。 Symog是一种新颖的软量化方法，因此可以同时解决学习任务和量化。在训练过程中，重量分布从单峰高斯分布变为高斯人的对称混合物，在该混合物中，每个平均值均属于特定的固定点模式。我们使用常见基准数据集（MNIST，CIFAR-10，CIFAR-100）上的不同体系结构（LENET5，VGG7，VGG11，Densenet）评估我们的方法，我们与最先进的量化方法进行了比较。我们取得了出色的成绩，胜过2位最先进的性能，CIFAR-10的错误率仅为5.71％，而CIFAR-100的错误率仅为27.65％。
Deep neural networks (DNNs) have been proven to outperform classical methods on several machine learning benchmarks. However, they have high computational complexity and require powerful processing units. Especially when deployed on embedded systems, model size and inference time must be significantly reduced. We propose SYMOG (symmetric mixture of Gaussian modes), which significantly decreases the complexity of DNNs through low-bit fixed-point quantization. SYMOG is a novel soft quantization method such that the learning task and the quantization are solved simultaneously. During training the weight distribution changes from an unimodal Gaussian distribution to a symmetric mixture of Gaussians, where each mean value belongs to a particular fixed-point mode. We evaluate our approach with different architectures (LeNet5, VGG7, VGG11, DenseNet) on common benchmark data sets (MNIST, CIFAR-10, CIFAR-100) and we compare with state-of-the-art quantization approaches. We achieve excellent results and outperform 2-bit state-of-the-art performance with an error rate of only 5.71% on CIFAR-10 and 27.65% on CIFAR-100.
