论文标题
变异正交特征
Variational Orthogonal Features
论文作者
论文摘要
稀疏的随机变异推理允许将高斯过程模型应用于大型数据集。使用此方法推断的计算成本为$ \ MATHCAL {O}（\ tilde {n} m^2+m^3），$，其中$ \ tilde {n} $是minibatch中的点数，$ m $是``诱导variational家族''的数量。最近的几项作品表明，对于某些先验，可以定义删除$ \ MATHCAL {O}（M^3）的功能，计算计算MINIBATCH估算的成本下限（ELBO）。当$ m \ gg \ tilde {n} $时，这代表了一个明显的计算节省。我们为任何固定的先前内核提供了功能的结构，该功能允许使用$ \ Mathcal {o}（\ tilde {n} t+m^2t）$中的$ t $ Monte Carlo样品计算Elbo的无偏估计器，并以$ \ Mathcal {O}（O}（O}（\ tilde {\ tilde {\ tilde {n n} the）我们分析了这种额外近似对推理质量的影响。
Sparse stochastic variational inference allows Gaussian process models to be applied to large datasets. The per iteration computational cost of inference with this method is $\mathcal{O}(\tilde{N}M^2+M^3),$ where $\tilde{N}$ is the number of points in a minibatch and $M$ is the number of `inducing features', which determine the expressiveness of the variational family. Several recent works have shown that for certain priors, features can be defined that remove the $\mathcal{O}(M^3)$ cost of computing a minibatch estimate of an evidence lower bound (ELBO). This represents a significant computational savings when $M\gg \tilde{N}$. We present a construction of features for any stationary prior kernel that allow for computation of an unbiased estimator to the ELBO using $T$ Monte Carlo samples in $\mathcal{O}(\tilde{N}T+M^2T)$ and in $\mathcal{O}(\tilde{N}T+MT)$ with an additional approximation. We analyze the impact of this additional approximation on inference quality.
