论文标题
随机特征模型的隐式正规化
Implicit Regularization of Random Feature Models
论文作者
论文摘要
随机特征（RF）模型用作内核方法的有效参数近似值。我们通过随机矩阵理论研究高斯RF模型与内核脊回归（KRR）之间的联系。对于具有$ p $功能，$ n $数据点和ridge $λ$的高斯RF型号，我们表明平均（即预期的）RF预测器接近具有有效山脊$ \tildeλ$的KRR预测变量。我们表明$ \tildeλ>λ$和$ \tildeλ\searrowλ$单调为$ p $生长，从而揭示了有限RF采样的隐式正则化效果。然后，我们将$ \tildeλ$ -KRR预测变量的风险（即测试误差）与$λ$ -RF预测变量的平均风险进行比较，并获得了其差异的精确和明确的限制。最后，我们从经验上发现平均$λ$ -RF预测器的测试错误与$ \tildeλ$ -KRR预测器之间的测试错误之间存在非常好的协议。
Random Feature (RF) models are used as efficient parametric approximations of kernel methods. We investigate, by means of random matrix theory, the connection between Gaussian RF models and Kernel Ridge Regression (KRR). For a Gaussian RF model with $P$ features, $N$ data points, and a ridge $λ$, we show that the average (i.e. expected) RF predictor is close to a KRR predictor with an effective ridge $\tildeλ$. We show that $\tildeλ > λ$ and $\tildeλ \searrow λ$ monotonically as $P$ grows, thus revealing the implicit regularization effect of finite RF sampling. We then compare the risk (i.e. test error) of the $\tildeλ$-KRR predictor with the average risk of the $λ$-RF predictor and obtain a precise and explicit bound on their difference. Finally, we empirically find an extremely good agreement between the test errors of the average $λ$-RF predictor and $\tildeλ$-KRR predictor.
