论文标题
极其简单的激活形状以进行分布外检测
Extremely Simple Activation Shaping for Out-of-Distribution Detection
论文作者
论文摘要
机器学习模型的培训和部署之间的分离意味着，在培训期间，并非所有部署中遇到的场景都可以预期，因此仅依靠培训的进步都有其限制。分布式（OOD）检测是一个重要领域，它强调模型处理看不见的情况的能力：模型知道何时不知道吗？现有的OOD检测方法要么引起额外的培训步骤，其他数据或对训练的网络进行非平凡的修改。相比之下，在这项工作中，我们提出了一种非常简单的事后，即时激活塑形方法，灰分，其中除去了大量（例如90％）在后层层的样本激活中的激活，其余（例如10％）简化了或进行了轻微调整。塑形在推理时进行应用，并且不需要根据培训数据计算出的任何统计数据。实验表明，这种简单的治疗可以增强分布和分布外的区别，从而允许在图像网上进行最新的OOD检测，并且不会明显恶化分布精度。视频，动画和代码可以在以下网址找到：https：//andrijazz.github.io/ash
The separation between training and deployment of machine learning models implies that not all scenarios encountered in deployment can be anticipated during training, and therefore relying solely on advancements in training has its limits. Out-of-distribution (OOD) detection is an important area that stress-tests a model's ability to handle unseen situations: Do models know when they don't know? Existing OOD detection methods either incur extra training steps, additional data or make nontrivial modifications to the trained network. In contrast, in this work, we propose an extremely simple, post-hoc, on-the-fly activation shaping method, ASH, where a large portion (e.g. 90%) of a sample's activation at a late layer is removed, and the rest (e.g. 10%) simplified or lightly adjusted. The shaping is applied at inference time, and does not require any statistics calculated from training data. Experiments show that such a simple treatment enhances in-distribution and out-of-distribution distinction so as to allow state-of-the-art OOD detection on ImageNet, and does not noticeably deteriorate the in-distribution accuracy. Video, animation and code can be found at: https://andrijazz.github.io/ash
