论文标题
测量图像分类中自然分布转移的鲁棒性
Measuring Robustness to Natural Distribution Shifts in Image Classification
论文作者
论文摘要
我们研究了由数据集自然变化引起的分布变化的鲁棒当前影像网模型。 Most research on robustness focuses on synthetic image perturbations (noise, simulated weather artifacts, adversarial examples, etc.), which leaves open how robustness on synthetic distribution shift relates to distribution shift arising in real data.通过对213种不同测试条件下204个Imagenet模型的评估，我们发现鲁棒性从当前合成到自然分布转移通常几乎没有转移。此外，大多数当前技术对我们的测试床的自然分布变化没有鲁棒性。 The main exception is training on larger and more diverse datasets, which in multiple cases increases robustness, but is still far from closing the performance gaps.我们的结果表明，实际数据中出现的分布变化目前是一个开放的研究问题。我们在https://modestyachts.github.io/imagenet-testbed/上提供测试床和数据作为未来工作的资源。
We study how robust current ImageNet models are to distribution shifts arising from natural variations in datasets. Most research on robustness focuses on synthetic image perturbations (noise, simulated weather artifacts, adversarial examples, etc.), which leaves open how robustness on synthetic distribution shift relates to distribution shift arising in real data. Informed by an evaluation of 204 ImageNet models in 213 different test conditions, we find that there is often little to no transfer of robustness from current synthetic to natural distribution shift. Moreover, most current techniques provide no robustness to the natural distribution shifts in our testbed. The main exception is training on larger and more diverse datasets, which in multiple cases increases robustness, but is still far from closing the performance gaps. Our results indicate that distribution shifts arising in real data are currently an open research problem. We provide our testbed and data as a resource for future work at https://modestyachts.github.io/imagenet-testbed/ .
