论文标题
标签平滑的标签会减轻标签噪音吗？
Does label smoothing mitigate label noise?
论文作者
论文摘要
标签平滑度通常用于训练深度学习模型中，其中一列训练标签与均匀标签向量混合在一起。从经验上讲，平滑表明可以改善预测性能和模型校准。在本文中，我们研究标签平滑是否也有效作为应对标签噪声的一种手段。虽然标签平滑显然会放大这个问题---相当于将对称噪声注入标签 - 我们显示了它与标签噪声文献中的一般损失校正技术的关系。在此方面，我们表明标签平滑性具有竞争力，在标签噪声下具有损失校正。此外，我们表明，当从嘈杂的数据中提取模型时，老师的标签平滑是有益的。这与最新无噪声问题的发现相反，并进一步阐明了标签平滑有益的设置。
Label smoothing is commonly used in training deep learning models, wherein one-hot training labels are mixed with uniform label vectors. Empirically, smoothing has been shown to improve both predictive performance and model calibration. In this paper, we study whether label smoothing is also effective as a means of coping with label noise. While label smoothing apparently amplifies this problem --- being equivalent to injecting symmetric noise to the labels --- we show how it relates to a general family of loss-correction techniques from the label noise literature. Building on this connection, we show that label smoothing is competitive with loss-correction under label noise. Further, we show that when distilling models from noisy data, label smoothing of the teacher is beneficial; this is in contrast to recent findings for noise-free problems, and sheds further light on settings where label smoothing is beneficial.
