论文标题
稀疏符合鲁棒性：Feynman-kac形式主义的渠道修剪原则性强大的深神经网
Sparsity Meets Robustness: Channel Pruning for the Feynman-Kac Formalism Principled Robust Deep Neural Nets
论文作者
论文摘要
深神经网（DNN）压缩对于适应移动设备至关重要。尽管存在许多成功的算法来压缩自然训练的DNN，但为训练有素的DNN开发有效且稳定的压缩算法仍然广泛开放。在本文中，我们专注于有效的DNN压缩算法和稀疏神经体系结构的共同设计，以进行鲁棒和准确的深度学习。这样的共同设计使我们能够促进适应稀疏性和健壮性的目标。考虑到这一目标，我们利用了放松的增强拉格朗日算法来修剪结构化和非结构化级别的受对抗训练的DNN的权重。使用Feynman-kac形式主义原则上有力且稀疏的DNNS，我们至少可以将经过对抗训练的RESNET20进行CIFAR10分类的频道稀疏增加一倍，同时，将自然准确性提高了8.69美元的$ 8.69 $ \％，而在基准标准的$ 20 $ ifgsm infgsm attererations the Ifgsm the ifgsm tearterations $ 5.42 $ 5.42 $ 5.42 $ 5.42 $ 5.42 $ \ f.542 $ \ f.5.42 $ f.fressive中。该代码可在\ url {https://github.com/baowangmath/rvsm-rgsm-admm}中获得。
Deep neural nets (DNNs) compression is crucial for adaptation to mobile devices. Though many successful algorithms exist to compress naturally trained DNNs, developing efficient and stable compression algorithms for robustly trained DNNs remains widely open. In this paper, we focus on a co-design of efficient DNN compression algorithms and sparse neural architectures for robust and accurate deep learning. Such a co-design enables us to advance the goal of accommodating both sparsity and robustness. With this objective in mind, we leverage the relaxed augmented Lagrangian based algorithms to prune the weights of adversarially trained DNNs, at both structured and unstructured levels. Using a Feynman-Kac formalism principled robust and sparse DNNs, we can at least double the channel sparsity of the adversarially trained ResNet20 for CIFAR10 classification, meanwhile, improve the natural accuracy by $8.69$\% and the robust accuracy under the benchmark $20$ iterations of IFGSM attack by $5.42$\%. The code is available at \url{https://github.com/BaoWangMath/rvsm-rgsm-admm}.
