论文标题
神经网络中的周期性外推概括
Periodic Extrapolative Generalisation in Neural Networks
论文作者
论文摘要
学习最简单的计算模式 - 周期性 - 是神经网络强烈概括的一个开放问题。我们正式化了针对周期性信号的外推概括的问题，并系统地研究了对一组基准测试任务的经典，基于人群和最近提议的周期性体系结构的概括能力。我们发现，无论其周期性参数的训练性如何，周期性和“蛇”激活功能在周期性外推时始终失败。此外，我们的结果表明，传统的顺序模型仍然胜过专门为推断设计的新型体系结构，而这些架构又通过基于人群的培训而胜过。我们使我们的基准测试和评估工具包，Perkit，可用且易于访问，以促进该地区的未来工作。
The learning of the simplest possible computational pattern -- periodicity -- is an open problem in the research of strong generalisation in neural networks. We formalise the problem of extrapolative generalisation for periodic signals and systematically investigate the generalisation abilities of classical, population-based, and recently proposed periodic architectures on a set of benchmarking tasks. We find that periodic and "snake" activation functions consistently fail at periodic extrapolation, regardless of the trainability of their periodicity parameters. Further, our results show that traditional sequential models still outperform the novel architectures designed specifically for extrapolation, and that these are in turn trumped by population-based training. We make our benchmarking and evaluation toolkit, PerKit, available and easily accessible to facilitate future work in the area.
