论文标题
超级损失感知的三元量化
Hyperspherical Loss-Aware Ternary Quantization
论文作者
论文摘要
大多数现有作品都将投影功能用于离散空间中的三元量化。在某些情况下，使用缩放因子和阈值来提高模型的准确性。但是，用于优化的梯度是不准确的，并在完整的精度和三元模型之间存在明显的准确性差距。为了获得更准确的梯度，有些作品逐渐增加了正向传播通行证中完整精度权重的离散部分，例如，使用基于温度的Sigmoid函数。我们没有在离散空间中直接执行三元量化，而是在三元量化之前通过正则化项将完全的精度权重接近三元重量。此外，受到基于温度的方法的启发，我们引入了一个重新缩放因子，以通过模拟Sigmoid函数的衍生物来获得更准确的梯度。实验结果表明，我们的方法可以显着提高图像分类和对象检测任务中三元量化的准确性。
Most of the existing works use projection functions for ternary quantization in discrete space. Scaling factors and thresholds are used in some cases to improve the model accuracy. However, the gradients used for optimization are inaccurate and result in a notable accuracy gap between the full precision and ternary models. To get more accurate gradients, some works gradually increase the discrete portion of the full precision weights in the forward propagation pass, e.g., using temperature-based Sigmoid function. Instead of directly performing ternary quantization in discrete space, we push full precision weights close to ternary ones through regularization term prior to ternary quantization. In addition, inspired by the temperature-based method, we introduce a re-scaling factor to obtain more accurate gradients by simulating the derivatives of Sigmoid function. The experimental results show that our method can significantly improve the accuracy of ternary quantization in both image classification and object detection tasks.
