论文标题
自由或深度：深贝叶斯神经网不需要复杂的重量后近似
Liberty or Depth: Deep Bayesian Neural Nets Do Not Need Complex Weight Posterior Approximations
论文作者
论文摘要
我们挑战了一个长期以来的假设，即贝叶斯神经网络中变异推断的平均场近似是严格的限制，并且在深层网络中并非如此。我们证明了几个结果，表明深平均变异重量后期可以在功能空间中诱导相似的分布，而较浅的网络具有复杂的重量后期。我们通过在小型模型中使用汉密尔顿蒙特卡洛（Hamiltonian Monte Carlo）的后验检查以及通过在大型环境中比较对角线和结构化稳定性来验证理论贡献。由于复杂的变分后代通常很昂贵且难以实施，因此我们的结果表明，在更深层次的模型中使用平均场变化推断是实用和理论上合理的结构化近似值替代方案。
We challenge the longstanding assumption that the mean-field approximation for variational inference in Bayesian neural networks is severely restrictive, and show this is not the case in deep networks. We prove several results indicating that deep mean-field variational weight posteriors can induce similar distributions in function-space to those induced by shallower networks with complex weight posteriors. We validate our theoretical contributions empirically, both through examination of the weight posterior using Hamiltonian Monte Carlo in small models and by comparing diagonal- to structured-covariance in large settings. Since complex variational posteriors are often expensive and cumbersome to implement, our results suggest that using mean-field variational inference in a deeper model is both a practical and theoretically justified alternative to structured approximations.
