论文标题
高容量专家二进制网络
High-Capacity Expert Binary Networks
论文作者
论文摘要
网络二进制化是创建有效的深层模型的有希望的硬件感知方向。尽管它具有记忆力和计算优势，但减少二进制模型与其实价对应物之间的准确性差距仍然是一个尚未解决的挑战性研究问题。为此，我们做出以下3个贡献：（a）为了提高模型容量，我们提出了专家二进制卷积，这是通过学习在输入特征条件下选择一个特定于数据的专家二进制滤波器来定制有条件计算的二进制网络。 （b）为了提高表示能力，我们建议通过引入有效的宽度扩展机制来解决二进制网络中固有的信息瓶颈，该机制将二进制操作保持在相同的预算之内。 （c）为了改善网络设计，我们提出了一种原则性的二进制网络增长机制，该机制揭示了一组有利属性的网络拓扑。总体而言，我们的方法在先前的工作中有所改善，而计算成本没有增加$ \ sim6 \％$，在Imagenet分类中达到了开创性的$ \ sim 71 \％$。代码将可用$ \ href {https://www.adrianbulat.com/binary-networks} {there} $。
Network binarization is a promising hardware-aware direction for creating efficient deep models. Despite its memory and computational advantages, reducing the accuracy gap between binary models and their real-valued counterparts remains an unsolved challenging research problem. To this end, we make the following 3 contributions: (a) To increase model capacity, we propose Expert Binary Convolution, which, for the first time, tailors conditional computing to binary networks by learning to select one data-specific expert binary filter at a time conditioned on input features. (b) To increase representation capacity, we propose to address the inherent information bottleneck in binary networks by introducing an efficient width expansion mechanism which keeps the binary operations within the same budget. (c) To improve network design, we propose a principled binary network growth mechanism that unveils a set of network topologies of favorable properties. Overall, our method improves upon prior work, with no increase in computational cost, by $\sim6 \%$, reaching a groundbreaking $\sim 71\%$ on ImageNet classification. Code will be made available $\href{https://www.adrianbulat.com/binary-networks}{here}$.
