论文标题
是什么使在自然世界中学习概率分布成为可能？
What makes it possible to learn probability distributions in the natural world?
论文作者
论文摘要
生物体和算法从先前观察结果中学习了概率分布，无论是在进化时间还是在即时观察中。在没有规律性的情况下，估算数据的基本分布将需要多次观察每个可能的结果。在这里，我们证明了两个条件使我们能够逃脱这一不可行的要求。首先，系统两半之间的相互信息应始终如一。其次，此共享信息应该是可压缩的，以便它可以由与信息成正比而不是熵代表。在这些条件下，可以用许多参数与系统大小线性生长。这些条件在自然图像和统计物理学的模型中都存在。
Organisms and algorithms learn probability distributions from previous observations, either over evolutionary time or on the fly. In the absence of regularities, estimating the underlying distribution from data would require observing each possible outcome many times. Here we show that two conditions allow us to escape this infeasible requirement. First, the mutual information between two halves of the system should be consistently sub-extensive. Second, this shared information should be compressible, so that it can be represented by a number of bits proportional to the information rather than to the entropy. Under these conditions, a distribution can be described with a number of parameters that grows linearly with system size. These conditions are borne out in natural images and in models from statistical physics, respectively.
