论文标题
超高维稀疏机器学习的骨干方法
The Backbone Method for Ultra-High Dimensional Sparse Machine Learning
论文作者
论文摘要
我们提出了骨干方法，这是一个通用框架，可以使稀疏且可解释的监督机器学习方法扩展到超高维问题。我们以$ 10^7 $的功能在几分钟内使用$ 10^7 $的功能解决稀疏回归问题，并在几分钟内以$ 10^5 $功能为单位的决策树问题。拟议的方法分为两个阶段：我们首先确定由潜在相关功能组成的骨干组，通过求解许多诱使的子问题；然后，我们仅考虑骨干特征，解决了一个减少的问题。对于稀疏回归问题，我们的理论分析表明，在某些假设和高概率下，骨干集由真正相关的特征组成。关于合成和现实世界数据集的数值实验表明，我们的方法在超高维问题中优于或与最先进的方法竞争，并且在精确方法尺度上与最佳解决方案竞争，既可以恢复真正相关的特征及其在样本中的预测性能。
We present the backbone method, a generic framework that enables sparse and interpretable supervised machine learning methods to scale to ultra-high dimensional problems. We solve sparse regression problems with $10^7$ features in minutes and $10^8$ features in hours, as well as decision tree problems with $10^5$ features in minutes.The proposed method operates in two phases: we first determine the backbone set, consisting of potentially relevant features, by solving a number of tractable subproblems; then, we solve a reduced problem, considering only the backbone features. For the sparse regression problem, our theoretical analysis shows that, under certain assumptions and with high probability, the backbone set consists of the truly relevant features. Numerical experiments on both synthetic and real-world datasets demonstrate that our method outperforms or competes with state-of-the-art methods in ultra-high dimensional problems, and competes with optimal solutions in problems where exact methods scale, both in terms of recovering the truly relevant features and in its out-of-sample predictive performance.
