论文标题
SOFAR：基于快捷方式的二元卷积神经网络的分形体系结构
SoFAr: Shortcut-based Fractal Architectures for Binary Convolutional Neural Networks
论文作者
论文摘要
二进制卷积神经网络（BCNN）可以显着提高深卷积神经网络（DCNN）在资源受限平台上的部署（例如移动和嵌入式系统）的效率。但是，与其完整的精确性对应物相比，BCNN的准确性降解仍然相当大，从而阻碍了他们的实际部署。由于向后传播中正向传播和梯度不匹配问题的不可避免的二进制误差，因此训练BCNNs以达到令人满意的精度是不平凡的。为了缓解训练的困难，基于捷径的BCNN（例如基于剩余的基于连接的双重重新连接和基于密集的连接的二进制二元）还引入了其他快捷方式，除了在其完整的精确性对应物中已经存在的快捷键。此外，由于分形结构触发类似于深度监督和横向学生教师信息流的影响，因此还使用了分形架构来改善全精度DCNN的训练过程。受捷径和分形体系结构的启发，我们提出了两个基于快捷方式的分形体系结构（SOFAR），专为BCNNS设计：1。用于二进制重新NET的基于残留的基于连接的分形体系结构，以及2个。二进制二进制二进制基于二进制的基于密集的基于连接的分形架构。我们提出的Sofar结合了一种统一模型的捷径和分形体系结构，这有助于对BCNN的培训。结果表明，与基于快捷键的BCNN相比，我们提出的Sofar可以提高准确性。具体而言，Imagenet上提议的RF-C4D8 Resnet37（41）和DRF-C2D2 Densenet51（53）的前1位准确性优于Bi-Real Resnet18（64）和BinaryDensenet51（32）（32）（32），分别均在同一计算上均分别为3.29％和1.41％。
Binary Convolutional Neural Networks (BCNNs) can significantly improve the efficiency of Deep Convolutional Neural Networks (DCNNs) for their deployment on resource-constrained platforms, such as mobile and embedded systems. However, the accuracy degradation of BCNNs is still considerable compared with their full precision counterpart, impeding their practical deployment. Because of the inevitable binarization error in the forward propagation and gradient mismatch problem in the backward propagation, it is nontrivial to train BCNNs to achieve satisfactory accuracy. To ease the difficulty of training, the shortcut-based BCNNs, such as residual connection-based Bi-real ResNet and dense connection-based BinaryDenseNet, introduce additional shortcuts in addition to the shortcuts already present in their full precision counterparts. Furthermore, fractal architectures have been also been used to improve the training process of full-precision DCNNs since the fractal structure triggers effects akin to deep supervision and lateral student-teacher information flow. Inspired by the shortcuts and fractal architectures, we propose two Shortcut-based Fractal Architectures (SoFAr) specifically designed for BCNNs: 1. residual connection-based fractal architectures for binary ResNet, and 2. dense connection-based fractal architectures for binary DenseNet. Our proposed SoFAr combines the adoption of shortcuts and the fractal architectures in one unified model, which is helpful in the training of BCNNs. Results show that our proposed SoFAr achieves better accuracy compared with shortcut-based BCNNs. Specifically, the Top-1 accuracy of our proposed RF-c4d8 ResNet37(41) and DRF-c2d2 DenseNet51(53) on ImageNet outperforms Bi-real ResNet18(64) and BinaryDenseNet51(32) by 3.29% and 1.41%, respectively, with the same computational complexity overhead.
