论文标题
如何训练准确的BNN用于嵌入式系统？
How to train accurate BNNs for embedded systems?
论文作者
论文摘要
在资源受限的嵌入式系统上部署卷积神经网络的关键推动力是二进制神经网络（BNN）。 BNNS通过将功能和权重进行二进制来保存内存并简化计算。不幸的是，二进制不可避免地伴随着准确性的严重降低。为了减少二进制和完整精确网络之间的准确性差距，最近提出了许多维修方法，我们已经将其分类并列入本章中的一个概述。维修方法分为两个主要分支，培训技术和网络拓扑变化，可以进一步分为较小的类别。后一种类别为嵌入式系统引入了额外的成本（能源消耗或额外的面积），而前者则没有。从我们的概述中，我们可以观察到在降低准确性差距方面取得了进展，但是BNN论文并未对应使用哪种修复方法来获得高度准确的BNN。因此，本章包含一项经验综述，该综述评估了许多维修方法的好处，而不是Resnet-20 \＆Cifar10和Resnet-18 \＆Cifar100基准。我们发现了三个最有益的维修类别：功能二进制器，功能归一化和双重残留。基于这篇评论，我们讨论未来的方向和研究机会。我们在嵌入式系统上勾勒出与BNN相关的收益和成本，因为BNN是否能够缩小准确性差距，同时在资源受限的嵌入式系统上保持高能效率还有待观察。
A key enabler of deploying convolutional neural networks on resource-constrained embedded systems is the binary neural network (BNN). BNNs save on memory and simplify computation by binarizing both features and weights. Unfortunately, binarization is inevitably accompanied by a severe decrease in accuracy. To reduce the accuracy gap between binary and full-precision networks, many repair methods have been proposed in the recent past, which we have classified and put into a single overview in this chapter. The repair methods are divided into two main branches, training techniques and network topology changes, which can further be split into smaller categories. The latter category introduces additional cost (energy consumption or additional area) for an embedded system, while the former does not. From our overview, we observe that progress has been made in reducing the accuracy gap, but BNN papers are not aligned on what repair methods should be used to get highly accurate BNNs. Therefore, this chapter contains an empirical review that evaluates the benefits of many repair methods in isolation over the ResNet-20\&CIFAR10 and ResNet-18\&CIFAR100 benchmarks. We found three repair categories most beneficial: feature binarizer, feature normalization, and double residual. Based on this review we discuss future directions and research opportunities. We sketch the benefit and costs associated with BNNs on embedded systems because it remains to be seen whether BNNs will be able to close the accuracy gap while staying highly energy-efficient on resource-constrained embedded systems.
