论文标题
Insta-BNN：具有实例感知阈值的二进制神经网络
INSTA-BNN: Binary Neural Network with INSTAnce-aware Threshold
论文作者
论文摘要
二进制神经网络（BNN）已成为减少记忆足迹和计算深神经网络成本的有前途解决方案，但是由于缺乏自由，由于激活和重量的自由缺乏限制，它们会受到质量退化的损失。为了补偿准确性下降，我们提出了一种具有实例感知阈值（Insta-bnn）的新型BNN设计，称为二进制神经网络，该设计以输入依赖性或实例感知方式动态控制量化阈值。根据我们的观察，高阶统计数据可以是估计输入分布特征的代表性指标。 Insta-BNN旨在考虑各种信息（包括高阶统计信息）动态调整阈值，但也明智地优化了它，以实现真实设备上的最小开销。我们的广泛研究表明，Insta-BNN在ImageNet分类任务上以可比的计算成本优于基线3.0％和2.8％，在Resnet-18和MobilenetV1的模型上分别达到68.5％和72.2％的TOP-1准确性。
Binary Neural Networks (BNNs) have emerged as a promising solution for reducing the memory footprint and compute costs of deep neural networks, but they suffer from quality degradation due to the lack of freedom as activations and weights are constrained to the binary values. To compensate for the accuracy drop, we propose a novel BNN design called Binary Neural Network with INSTAnce-aware threshold (INSTA-BNN), which controls the quantization threshold dynamically in an input-dependent or instance-aware manner. According to our observation, higher-order statistics can be a representative metric to estimate the characteristics of the input distribution. INSTA-BNN is designed to adjust the threshold dynamically considering various information, including higher-order statistics, but it is also optimized judiciously to realize minimal overhead on a real device. Our extensive study shows that INSTA-BNN outperforms the baseline by 3.0% and 2.8% on the ImageNet classification task with comparable computing cost, achieving 68.5% and 72.2% top-1 accuracy on ResNet-18 and MobileNetV1 based models, respectively.
