论文标题
单层神经网络共轭核的最大特征值
Largest Eigenvalues of the Conjugate Kernel of Single-Layered Neural Networks
论文作者
论文摘要
本文涉及来自神经网络研究的某些非线性随机矩阵合奏的最大特征值的渐近分布。更确切地说，我们考虑$ m = \ frac {1} {m} yy^\ top $，$ y = f（wx）$，其中$ w $和$ x $是随机的矩形矩阵，带有i.i.d.中心条目。这对单个分层随机前馈神经网络的数据协方差矩阵或共轭核进行了建模。函数$ f $进入进入，可以看作是神经网络的激活函数。我们表明，最大的特征值具有与某些众所周知的线性随机矩阵集合的极限（概率）相同的极限（概率）。特别是，我们将非线性模型最大特征值的渐近极限与信息加上噪声随机矩阵的渐近限制联系起来，并根据功能$ f $以及$ W $和$ x $的分布建立了可能的相变。对于机器学习的应用可能会很感兴趣。
This paper is concerned with the asymptotic distribution of the largest eigenvalues for some nonlinear random matrix ensemble stemming from the study of neural networks. More precisely we consider $M= \frac{1}{m} YY^\top$ with $Y=f(WX)$ where $W$ and $X$ are random rectangular matrices with i.i.d. centered entries. This models the data covariance matrix or the Conjugate Kernel of a single layered random Feed-Forward Neural Network. The function $f$ is applied entrywise and can be seen as the activation function of the neural network. We show that the largest eigenvalue has the same limit (in probability) as that of some well-known linear random matrix ensembles. In particular, we relate the asymptotic limit of the largest eigenvalue for the nonlinear model to that of an information-plus-noise random matrix, establishing a possible phase transition depending on the function $f$ and the distribution of $W$ and $X$. This may be of interest for applications to machine learning.
