论文标题
置换不变代理的最小神经网络模型
Minimal Neural Network Models for Permutation Invariant Agents
论文作者
论文摘要
面对环境和/或自己的变化，自然界中的生物已经演变为表现出灵活性。人工神经网络（ANN）已被证明可用于控制在环境中作用的人工药物。但是，大多数用于增强学习类型任务的ANN模型具有刚性结构，不允许输入大小。此外，如果在优化期间未见的订购中显示输入，它们会灾难性地失败。我们发现，这两种ANN的屈曲度可以缓解，它们的解决方案很简单且高度相关。对于置换不变性，没有优化的参数可以与输入元素的特定索引相关。对于尺寸不变性，必须将输入投影到不随预测数量而增长的公共空间上。基于这些限制，我们构建了一个概念上简单的模型，该模型表现出大多数ANN缺乏的灵活性。我们在多个控制问题上演示了模型的属性，并表明它可以应对输入指数的非常快速的排列以及输入大小的变化。消融研究表明，可以通过简单的前馈结构来实现这些特性，但是优化复发结构要容易得多。
Organisms in nature have evolved to exhibit flexibility in face of changes to the environment and/or to themselves. Artificial neural networks (ANNs) have proven useful for controlling of artificial agents acting in environments. However, most ANN models used for reinforcement learning-type tasks have a rigid structure that does not allow for varying input sizes. Further, they fail catastrophically if inputs are presented in an ordering unseen during optimization. We find that these two ANN inflexibilities can be mitigated and their solutions are simple and highly related. For permutation invariance, no optimized parameters can be tied to a specific index of the input elements. For size invariance, inputs must be projected onto a common space that does not grow with the number of projections. Based on these restrictions, we construct a conceptually simple model that exhibit flexibility most ANNs lack. We demonstrate the model's properties on multiple control problems, and show that it can cope with even very rapid permutations of input indices, as well as changes in input size. Ablation studies show that is possible to achieve these properties with simple feedforward structures, but that it is much easier to optimize recurrent structures.
