论文标题
监督对比学习
Supervised Contrastive Learning
论文作者
论文摘要
近年来，应用于自我监督的代表学习的对比学习已经复兴，导致了对深度图像模型的无监督培训的状态表现。现代批次对比方法集合或显着优于传统的对比损失，例如三胞胎，最大额度和n对损失。在这项工作中，我们将自我监督的批处理对比方法扩展到了完全监督的设置，从而使我们能够有效利用标签信息。属于同一类的积分集簇在嵌入空间中拉在一起，同时将不同类别的样本簇推开。我们分析了有监督的对比度（SUPCON）损失的两个可能版本，从而确定了损失的表现最佳。在RESNET-200上，我们在Imagenet数据集上实现了81.4％的前1个精度，该数据集比该体系结构报告的最佳数量高0.8％。我们在其他数据集和两个Resnet变体上表现出对跨凝性的一致性。损失显示了对自然腐败的鲁棒性的好处，并且在优化器和数据增强等高参数设置中更稳定。我们的损失函数易于实现，并且参考tensorflow代码在https://t.ly/supcon上发布。
Contrastive learning applied to self-supervised representation learning has seen a resurgence in recent years, leading to state of the art performance in the unsupervised training of deep image models. Modern batch contrastive approaches subsume or significantly outperform traditional contrastive losses such as triplet, max-margin and the N-pairs loss. In this work, we extend the self-supervised batch contrastive approach to the fully-supervised setting, allowing us to effectively leverage label information. Clusters of points belonging to the same class are pulled together in embedding space, while simultaneously pushing apart clusters of samples from different classes. We analyze two possible versions of the supervised contrastive (SupCon) loss, identifying the best-performing formulation of the loss. On ResNet-200, we achieve top-1 accuracy of 81.4% on the ImageNet dataset, which is 0.8% above the best number reported for this architecture. We show consistent outperformance over cross-entropy on other datasets and two ResNet variants. The loss shows benefits for robustness to natural corruptions and is more stable to hyperparameter settings such as optimizers and data augmentations. Our loss function is simple to implement, and reference TensorFlow code is released at https://t.ly/supcon.
