论文标题
矩阵分解的景观重新审视
The Landscape of Matrix Factorization Revisited
论文作者
论文摘要
我们重新审视简单矩阵分解问题的景观。对于低级矩阵分解，先前的工作表明，存在无限的许多关键点，所有这些要么是全球最小值或严格的马鞍。严格的鞍座，Hessian的最低特征值为负。有趣的是，在所有严格的马鞍上，这种最低特征值是否在零以下均匀限制。为了回答这一点，我们考虑一般线性组下的关键点的轨道。对于每个轨道，我们确定一个代表点，称为规范点。如果规范点是严格的马鞍，那么轨道上的每个点也是如此。我们在每个规范严格的鞍座上得出了Hessian的最小特征值的表达式，并用它来表明，在一组严格的鞍座上，黑森的最小特征值并不均匀地限制在零以下。我们还表明，梯度流的已知不变性属性可确保梯度流的解决方案仅在不变的歧管$ \ Mathcal {M} _C $上遇到临界点，由初始条件确定。我们表明，与一般情况相反，在$ \ Mathcal {M} _ {0} $中，严格马鞍的最低特征值均匀地限制在零以下。我们根据分解矩阵的奇异值获得了这种结合的表达式。该结合取决于非零奇异值的大小以及矩阵的不同非零奇异值之间的分离。
We revisit the landscape of the simple matrix factorization problem. For low-rank matrix factorization, prior work has shown that there exist infinitely many critical points all of which are either global minima or strict saddles. At a strict saddle the minimum eigenvalue of the Hessian is negative. Of interest is whether this minimum eigenvalue is uniformly bounded below zero over all strict saddles. To answer this we consider orbits of critical points under the general linear group. For each orbit we identify a representative point, called a canonical point. If a canonical point is a strict saddle, so is every point on its orbit. We derive an expression for the minimum eigenvalue of the Hessian at each canonical strict saddle and use this to show that the minimum eigenvalue of the Hessian over the set of strict saddles is not uniformly bounded below zero. We also show that a known invariance property of gradient flow ensures the solution of gradient flow only encounters critical points on an invariant manifold $\mathcal{M}_C$ determined by the initial condition. We show that, in contrast to the general situation, the minimum eigenvalue of strict saddles in $\mathcal{M}_{0}$ is uniformly bounded below zero. We obtain an expression for this bound in terms of the singular values of the matrix being factorized. This bound depends on the size of the nonzero singular values and on the separation between distinct nonzero singular values of the matrix.
