论文标题
过度参数化和泛化误差：加权三角插值
Overparameterization and generalization error: weighted trigonometric interpolation
论文作者
论文摘要
在过度参数化的场景和相关的双重下降现象中，学到的深神经网络的良好概括属性的激励，本文分析了过度参数化线性学习问题的平滑度与低概括误差之间的关系。我们研究了一个随机的傅立叶系列模型，其中的任务是估算等距样品中未知的傅立叶系数。我们得出了普通和加权最小二乘估计器的概括误差的精确表达式。我们确切地展示了与参数型不足的制度相比，以加权三角插值的形式对平滑插值剂的偏差如何导致过度参数化制度的概括误差较小。这提供了对过度参数化的力量的见解，这在现代机器学习中很常见。
Motivated by surprisingly good generalization properties of learned deep neural networks in overparameterized scenarios and by the related double descent phenomenon, this paper analyzes the relation between smoothness and low generalization error in an overparameterized linear learning problem. We study a random Fourier series model, where the task is to estimate the unknown Fourier coefficients from equidistant samples. We derive exact expressions for the generalization error of both plain and weighted least squares estimators. We show precisely how a bias towards smooth interpolants, in the form of weighted trigonometric interpolation, can lead to smaller generalization error in the overparameterized regime compared to the underparameterized regime. This provides insight into the power of overparameterization, which is common in modern machine learning.
