论文标题
在最佳发展的训练扰动下调查神经网络中的概括
Investigating Generalization in Neural Networks under Optimally Evolved Training Perturbations
论文作者
论文摘要
在本文中，我们研究了输入扰动下神经网络的概括性能，并表明训练数据损坏最少会导致极大的过度拟合。我们提出了一种进化算法，以搜索最佳的像素扰动，使用来自域适应性文献启发的新成本功能，该功能明确地最大化了清洁图像和损坏的图像之间的通用差距和域差异。我们的方法在最先进的卷积神经网络（CNNS）体系结构上优于先前基于像素的数据分布移动方法。有趣的是，我们发现优化的选择在概括性鲁棒性中起着重要作用，这是因为经验观察到，与自适应优化技术不同（ADAM），SGD对此类训练数据腐败具有弹性。我们的源代码可在https://github.com/subhajitchaudhury/evo-shift上找到。
In this paper, we study the generalization properties of neural networks under input perturbations and show that minimal training data corruption by a few pixel modifications can cause drastic overfitting. We propose an evolutionary algorithm to search for optimal pixel perturbations using novel cost function inspired from literature in domain adaptation that explicitly maximizes the generalization gap and domain divergence between clean and corrupted images. Our method outperforms previous pixel-based data distribution shift methods on state-of-the-art Convolutional Neural Networks (CNNs) architectures. Interestingly, we find that the choice of optimization plays an important role in generalization robustness due to the empirical observation that SGD is resilient to such training data corruption unlike adaptive optimization techniques (ADAM). Our source code is available at https://github.com/subhajitchaudhury/evo-shift.
