论文标题
功率球形分布
The Power Spherical distribution
论文作者
论文摘要
对在超球形空间中定义的概率模型的兴趣越来越大，无论是为了适应观察到的数据还是潜在的结构。 von Mises-fisher（VMF）的分布通常被视为超球的正态分布，是一种标准的建模选择：它是一个指数型家族，因此享有重要的统计结果，例如，已知的Kullback-Leibler（KL）与其他VMF分布的差异。但是，从VMF分布中采样需要一个拒绝采样程序，除了缓慢的姿势在随机反向传播的背景下，通过重新聚集技巧构成了困难。此外，对于某些VMF，例如，浓度高和/或在高维度的VMF上，此过程在数值上是不稳定的。我们提出了一个新颖的分布，功率球形分布，该分布保留了VMF的某些重要方面（例如，对超球的支持，对其平均方向参数的对称性，从其他VMF分布中知道的KL），同时解决其主要缺陷（即伸缩性和数值稳定性和数值稳定性）。我们通过数值实验证明了功率球形分布的稳定性，并将其进一步应用于接受MNIST训练的变异自动编码器。代码：https：//github.com/nicola-decao/power_spherical
There is a growing interest in probabilistic models defined in hyper-spherical spaces, be it to accommodate observed data or latent structure. The von Mises-Fisher (vMF) distribution, often regarded as the Normal distribution on the hyper-sphere, is a standard modeling choice: it is an exponential family and thus enjoys important statistical results, for example, known Kullback-Leibler (KL) divergence from other vMF distributions. Sampling from a vMF distribution, however, requires a rejection sampling procedure which besides being slow poses difficulties in the context of stochastic backpropagation via the reparameterization trick. Moreover, this procedure is numerically unstable for certain vMFs, e.g., those with high concentration and/or in high dimensions. We propose a novel distribution, the Power Spherical distribution, which retains some of the important aspects of the vMF (e.g., support on the hyper-sphere, symmetry about its mean direction parameter, known KL from other vMF distributions) while addressing its main drawbacks (i.e., scalability and numerical stability). We demonstrate the stability of Power Spherical distributions with a numerical experiment and further apply it to a variational auto-encoder trained on MNIST. Code at: https://github.com/nicola-decao/power_spherical
