论文标题
无穷小折刀和模型的组合
The Infinitesimal Jackknife and Combinations of Models
论文作者
论文摘要
无穷小折刀是一种估计参数模型差异的通用方法，最近也用于某些集合方法。在本文中，我们扩展了无穷小折刀，以估计任意两种模型之间的协方差。这可用于量化模型组合的不确定性，或构建测试统计信息，以比较使用相同训练数据集拟合的模型的不同模型或组合。本文中的具体示例使用了随机森林和M估计剂等模型的增强组合。我们还研究了其在XGBoost模型的神经网络和集合上的应用。我们通过广泛的模拟及其在北京住房数据中的应用来说明差异估计的功效，并证明了无穷小折刀协方差估算的理论一致性。
The Infinitesimal Jackknife is a general method for estimating variances of parametric models, and more recently also for some ensemble methods. In this paper we extend the Infinitesimal Jackknife to estimate the covariance between any two models. This can be used to quantify uncertainty for combinations of models, or to construct test statistics for comparing different models or ensembles of models fitted using the same training dataset. Specific examples in this paper use boosted combinations of models like random forests and M-estimators. We also investigate its application on neural networks and ensembles of XGBoost models. We illustrate the efficacy of variance estimates through extensive simulations and its application to the Beijing Housing data, and demonstrate the theoretical consistency of the Infinitesimal Jackknife covariance estimate.
