论文标题
重新访问残余网络以进行对抗性鲁棒性：建筑观点
Revisiting Residual Networks for Adversarial Robustness: An Architectural Perspective
论文作者
论文摘要
改善卷积神经网络的对抗性鲁棒性的努力主要集中在开发更有效的对抗性训练方法上。相比之下，很少注意分析建筑元素（例如拓扑，深度和宽度）对对抗性鲁棒性的作用。本文旨在弥合这一差距，并就建筑设计对对抗性鲁棒性的影响进行整体研究。我们专注于剩余网络，并考虑块级别的体系结构设计，即拓扑，内核大小，激活和归一化，以及网络缩放级别，即网络中每个块的深度和宽度。在这两种情况下，我们都首先通过系统的烧烤实验得出见解。然后，我们设计了一个强大的残留块，称为Robustresblock，并设计了一个称为鲁棒尺度的复合缩放规则，以在所需的flop计数下分布深度和宽度。最后，我们结合了Robustresblock和Robustscaling，并提供了对抗性稳健的残留网络，Robustresnets的组合，跨越了广泛的模型能力。 Experimental validation across multiple datasets and adversarial attacks demonstrate that RobustResNets consistently outperform both the standard WRNs and other existing robust architectures, achieving state-of-the-art AutoAttack robust accuracy of 61.1% without additional data and 63.7% with 500K external data while being $2\times$ more compact in terms of parameters.代码可在\ url {https://github.com/zhichao-lu/robust-isidual-network}中获得。
Efforts to improve the adversarial robustness of convolutional neural networks have primarily focused on developing more effective adversarial training methods. In contrast, little attention was devoted to analyzing the role of architectural elements (such as topology, depth, and width) on adversarial robustness. This paper seeks to bridge this gap and present a holistic study on the impact of architectural design on adversarial robustness. We focus on residual networks and consider architecture design at the block level, i.e., topology, kernel size, activation, and normalization, as well as at the network scaling level, i.e., depth and width of each block in the network. In both cases, we first derive insights through systematic ablative experiments. Then we design a robust residual block, dubbed RobustResBlock, and a compound scaling rule, dubbed RobustScaling, to distribute depth and width at the desired FLOP count. Finally, we combine RobustResBlock and RobustScaling and present a portfolio of adversarially robust residual networks, RobustResNets, spanning a broad spectrum of model capacities. Experimental validation across multiple datasets and adversarial attacks demonstrate that RobustResNets consistently outperform both the standard WRNs and other existing robust architectures, achieving state-of-the-art AutoAttack robust accuracy of 61.1% without additional data and 63.7% with 500K external data while being $2\times$ more compact in terms of parameters. Code is available at \url{ https://github.com/zhichao-lu/robust-residual-network}
