论文标题
Bootstrap合奏的差异视图作为贝叶斯推断
A Variational View on Bootstrap Ensembles as Bayesian Inference
论文作者
论文摘要
在本文中，我们采用各种论点来建立神经网络的集合方法与贝叶斯推论之间的联系。我们考虑了一个基于合奏的方案，其中每个模型/粒子通过参数bootstrap和先验的扰动对应于数据的扰动。我们得出条件，在该条件下，粒子的任何优化步骤使关联的分布降低了其与模型参数的差异。这种条件不需要任何特定形式的近似形式，它们纯粹是几何形式，可以在许多有趣的模型（例如具有relu激活的神经网络）上进行有关集合的行为的见解。实验证实，合奏方法可以成为近似贝叶斯推断的有效替代方法。本文中的理论发展旨在解释这种行为。
In this paper, we employ variational arguments to establish a connection between ensemble methods for Neural Networks and Bayesian inference. We consider an ensemble-based scheme where each model/particle corresponds to a perturbation of the data by means of parametric bootstrap and a perturbation of the prior. We derive conditions under which any optimization steps of the particles makes the associated distribution reduce its divergence to the posterior over model parameters. Such conditions do not require any particular form for the approximation and they are purely geometrical, giving insights on the behavior of the ensemble on a number of interesting models such as Neural Networks with ReLU activations. Experiments confirm that ensemble methods can be a valid alternative to approximate Bayesian inference; the theoretical developments in the paper seek to explain this behavior.
