论文标题
使用花键对神经网络的校准
Calibration of Neural Networks using Splines
论文作者
论文摘要
在下游决策取决于预测的概率的情况下，校准神经网络在安全至关重要的应用中时至关重要。测量校准误差等于比较两个经验分布。在这项工作中，我们介绍了一个受经典的Kolmogorov-Smirnov（KS）统计检验启发的无包校准度量，其中主要思想是比较各自的累积概率分布。由此，通过使用细条通过可区分函数近似经验累积分布，我们获得了重新校准函数，该功能将网络输出映射到实际（校准）类分配概率。使用固定校准集进行脊柱拟合，并在看不见的测试集上评估所获得的重新校准函数。我们针对各种图像分类数据集的现有校准方法测试了我们的方法，基于样条的重新校准方法始终优于KS错误以及其他常用的校准度量的现有方法。我们的代码可从https://github.com/kartikgupta-at-anu/spline-calibration获得。
Calibrating neural networks is of utmost importance when employing them in safety-critical applications where the downstream decision making depends on the predicted probabilities. Measuring calibration error amounts to comparing two empirical distributions. In this work, we introduce a binning-free calibration measure inspired by the classical Kolmogorov-Smirnov (KS) statistical test in which the main idea is to compare the respective cumulative probability distributions. From this, by approximating the empirical cumulative distribution using a differentiable function via splines, we obtain a recalibration function, which maps the network outputs to actual (calibrated) class assignment probabilities. The spine-fitting is performed using a held-out calibration set and the obtained recalibration function is evaluated on an unseen test set. We tested our method against existing calibration approaches on various image classification datasets and our spline-based recalibration approach consistently outperforms existing methods on KS error as well as other commonly used calibration measures. Our Code is available at https://github.com/kartikgupta-at-anu/spline-calibration.
