论文标题
要了解增强学习的神经网络中的模块化与性能之间的联系
Towards Understanding the Link Between Modularity and Performance in Neural Networks for Reinforcement Learning
论文作者
论文摘要
模块化已被广泛研究为通过手工制作的模块化体系结构和自动方法等各种技术来提高神经网络功能的机制。尽管这些方法有时显示出对概括能力，鲁棒性和效率的改善，但使模块化具有性能优势的机制尚不清楚。在本文中，我们研究了这个问题，发现最佳性能的网络模块化量可能纠缠在网络的许多其他功能与问题环境之间的复杂关系中。因此，在神经网络中直接优化或任意指定适当的模块化可能无益。我们使用了经典的神经进化算法，该算法能够对具有不同模块化水平的神经网络体系结构和权重的丰富，自动优化和探索。在三个强化学习任务上，评估了通过增强拓扑算法的神经进化而产生的网络的结构模块和性能，并且有或没有其他模块化目标。质量多样性优化算法，地图 - 精英的结果表明，模块化，性能和其他预定义的网络特征之间的复杂条件关系。
Modularity has been widely studied as a mechanism to improve the capabilities of neural networks through various techniques such as hand-crafted modular architectures and automatic approaches. While these methods have sometimes shown improvements towards generalisation ability, robustness, and efficiency, the mechanisms that enable modularity to give performance advantages are unclear. In this paper, we investigate this issue and find that the amount of network modularity for optimal performance is likely entangled in complex relationships between many other features of the network and problem environment. Therefore, direct optimisation or arbitrary designation of a suitable amount of modularity in neural networks may not be beneficial. We used a classic neuroevolutionary algorithm which enables rich, automatic optimisation and exploration of neural network architectures and weights with varying levels of modularity. The structural modularity and performance of networks generated by the NeuroEvolution of Augmenting Topologies algorithm was assessed on three reinforcement learning tasks, with and without an additional modularity objective. The results of the quality-diversity optimisation algorithm, MAP-Elites, suggest intricate conditional relationships between modularity, performance, and other predefined network features.
