论文标题
与KCAL退后一步：用于深神经网络的多级内核校准
Taking a Step Back with KCal: Multi-Class Kernel-Based Calibration for Deep Neural Networks
论文作者
论文摘要
深度神经网络（DNN）分类器通常过于自信，从而产生错误的类概率。在医疗保健等高风险应用程序中，从业人员需要$ \ textit {完全校准} $概率预测决策。也就是说，以预测$ \ textit {vector} $，$ \ textit {每个} $ class'概率的条件应接近预测值。大多数现有的校准方法要么缺乏产生校准输出的理论保证，要么降低过程中的分类准确性，或者仅校准预测类。本文提出了一种新的基于内核的校准方法，称为KCAL。与现有的校准程序不同，KCAL不直接在DNN的逻辑或软磁输出上运行。取而代之的是，KCAL在倒数第二层潜在嵌入中学习了度量空间，并使用校准集对内核密度估计产生预测。我们首先从理论上分析KCAL，表明它享有可证明的$ \ textit {full} $校准保证。然后，通过跨多种数据集进行的广泛实验，我们表明，KCAL始终超过基本线，该基准通过校准误差和适当的评分规则（如Brier得分）来衡量。
Deep neural network (DNN) classifiers are often overconfident, producing miscalibrated class probabilities. In high-risk applications like healthcare, practitioners require $\textit{fully calibrated}$ probability predictions for decision-making. That is, conditioned on the prediction $\textit{vector}$, $\textit{every}$ class' probability should be close to the predicted value. Most existing calibration methods either lack theoretical guarantees for producing calibrated outputs, reduce classification accuracy in the process, or only calibrate the predicted class. This paper proposes a new Kernel-based calibration method called KCal. Unlike existing calibration procedures, KCal does not operate directly on the logits or softmax outputs of the DNN. Instead, KCal learns a metric space on the penultimate-layer latent embedding and generates predictions using kernel density estimates on a calibration set. We first analyze KCal theoretically, showing that it enjoys a provable $\textit{full}$ calibration guarantee. Then, through extensive experiments across a variety of datasets, we show that KCal consistently outperforms baselines as measured by the calibration error and by proper scoring rules like the Brier Score.
