论文标题
通过合作协调的进化增强学习否定性相关搜索
Evolutionary Reinforcement Learning via Cooperative Coevolutionary Negatively Correlated Search
论文作者
论文摘要
由于其勘探能力，已成功应用进化算法（EAS）来优化增强学习（RL）任务的政策。最近提出的负相关搜索（NCS）提供了独特的平行探索搜索行为，并有望更有效地促进RL。考虑到通常采用的神经政策通常涉及要优化数百万个参数，因此直接应用NC在RL上可能会面临大规模搜索空间的巨大挑战。为了解决这个问题，本文介绍了NCS友好的合作协调（CC）框架，以扩展NCS，同时在很大程度上保留其平行探索搜索行为。还讨论了可能恶化NCS的传统CC问题。对10个受欢迎的Atari游戏的实证研究表明，该方法可以通过有效探索170万维度的搜索空间来大大优于三种最先进的深度RL方法，而计算时间减少了50％。
Evolutionary algorithms (EAs) have been successfully applied to optimize the policies for Reinforcement Learning (RL) tasks due to their exploration ability. The recently proposed Negatively Correlated Search (NCS) provides a distinct parallel exploration search behavior and is expected to facilitate RL more effectively. Considering that the commonly adopted neural policies usually involves millions of parameters to be optimized, the direct application of NCS to RL may face a great challenge of the large-scale search space. To address this issue, this paper presents an NCS-friendly Cooperative Coevolution (CC) framework to scale-up NCS while largely preserving its parallel exploration search behavior. The issue of traditional CC that can deteriorate NCS is also discussed. Empirical studies on 10 popular Atari games show that the proposed method can significantly outperform three state-of-the-art deep RL methods with 50% less computational time by effectively exploring a 1.7 million-dimensional search space.
