论文标题
公正的风险估计器可能会误导：用互补标签进行学习的案例研究
Unbiased Risk Estimators Can Mislead: A Case Study of Learning with Complementary Labels
论文作者
论文摘要
在弱监督的学习中，无偏见的风险估计器（URE）是从不同分布中汲取培训和测试数据时训练分类器的强大工具。然而，当模型像深网一样复杂时，在许多问题设置中，URE会导致过度适应。在本文中，我们通过研究一个名为“学习互补标签的学习”的弱监督问题，研究了这种过度拟合的原因。我们认为，梯度估计的质量在最小化的风险最小化方面更重要。从理论上讲，我们表明URE给出了公正的梯度估计器（UGE）。但是，实际上，UGE可能会遭受巨大的差异，这会导致经验梯度通常在最小化过程中远离真实梯度。为此，我们提出了一种新型的替代互补损失（SCL）框架，该框架以降低的差异为零偏见，并使经验梯度与方向上的真实梯度更加一致。由于这种特征，SCL成功地减轻了过度拟合问题并改善了基于URE的方法。
In weakly supervised learning, unbiased risk estimator(URE) is a powerful tool for training classifiers when training and test data are drawn from different distributions. Nevertheless, UREs lead to overfitting in many problem settings when the models are complex like deep networks. In this paper, we investigate reasons for such overfitting by studying a weakly supervised problem called learning with complementary labels. We argue the quality of gradient estimation matters more in risk minimization. Theoretically, we show that a URE gives an unbiased gradient estimator(UGE). Practically, however, UGEs may suffer from huge variance, which causes empirical gradients to be usually far away from true gradients during minimization. To this end, we propose a novel surrogate complementary loss(SCL) framework that trades zero bias with reduced variance and makes empirical gradients more aligned with true gradients in the direction. Thanks to this characteristic, SCL successfully mitigates the overfitting issue and improves URE-based methods.
