论文标题
MCMC相互作用的变异推断
MCMC-Interactive Variational Inference
论文作者
论文摘要
利用良好的MCMC策略，我们提出MCMC相互作用变异推理（MIVI）不仅以时间限制的方式估算后验，而且还促进了MCMC过渡的设计。 MIVI构建了一个有参数的差异分布，然后是一个短的马尔可夫链，它利用了变异推理和MCMC的互补特性来鼓励相互改进。一方面，由于变异分布定位高后密度区域，马尔可夫链在变异推理框架内进行了优化，尽管有少量过渡，但在变化推理框架内有效地靶向后部。另一方面，具有相当灵活性的优化马尔可夫链指导了向后的变异分布，并减轻了其不确定性的低估。此外，我们证明了MIVI中优化的马尔可夫链，可以推断出外推，这意味着其边际分布随着链的生长而靠近真实的后部。因此，马尔可夫链可单独用作有效的MCMC方案。实验表明，MIVI不仅准确有效地近似后代，而且还促进了随机梯度MCMC和Gibbs采样过渡的设计。
Leveraging well-established MCMC strategies, we propose MCMC-interactive variational inference (MIVI) to not only estimate the posterior in a time constrained manner, but also facilitate the design of MCMC transitions. Constructing a variational distribution followed by a short Markov chain that has parameters to learn, MIVI takes advantage of the complementary properties of variational inference and MCMC to encourage mutual improvement. On one hand, with the variational distribution locating high posterior density regions, the Markov chain is optimized within the variational inference framework to efficiently target the posterior despite a small number of transitions. On the other hand, the optimized Markov chain with considerable flexibility guides the variational distribution towards the posterior and alleviates its underestimation of uncertainty. Furthermore, we prove the optimized Markov chain in MIVI admits extrapolation, which means its marginal distribution gets closer to the true posterior as the chain grows. Therefore, the Markov chain can be used separately as an efficient MCMC scheme. Experiments show that MIVI not only accurately and efficiently approximates the posteriors but also facilitates designs of stochastic gradient MCMC and Gibbs sampling transitions.
