论文标题
有效的分布式框架，用于协作多代理增强学习
Efficient Distributed Framework for Collaborative Multi-Agent Reinforcement Learning
论文作者
论文摘要
针对不完整信息环境的多机构增强学习吸引了研究人员的广泛关注。但是，由于样本收集缓慢和样本探索较差，在多机构增强学习中仍然存在一些问题，例如不稳定的模型迭代和低训练效率。此外，大多数现有的分布式框架都是用于单药加固学习的，并且不适合多代理。在本文中，我们根据Actor-Work-Learner Architecture设计了一个分布式的MARL框架。在此框架中，可以同时部署多个异步环境相互作用模块，从而大大提高了样本收集速度和样本多样性。同时，为了充分利用计算资源，我们将模型迭代与环境互动相结合，从而加速了政策迭代。最后，我们验证了在MACA军事模拟环境中提出框架和具有IMComplete信息特征的SMAC 3D实时策略游戏环境的有效性。
Multi-agent reinforcement learning for incomplete information environments has attracted extensive attention from researchers. However, due to the slow sample collection and poor sample exploration, there are still some problems in multi-agent reinforcement learning, such as unstable model iteration and low training efficiency. Moreover, most of the existing distributed framework are proposed for single-agent reinforcement learning and not suitable for multi-agent. In this paper, we design an distributed MARL framework based on the actor-work-learner architecture. In this framework, multiple asynchronous environment interaction modules can be deployed simultaneously, which greatly improves the sample collection speed and sample diversity. Meanwhile, to make full use of computing resources, we decouple the model iteration from environment interaction, and thus accelerate the policy iteration. Finally, we verified the effectiveness of propose framework in MaCA military simulation environment and the SMAC 3D realtime strategy gaming environment with imcomplete information characteristics.
