论文标题
罗马：多方强化学习以及紧急角色
ROMA: Multi-Agent Reinforcement Learning with Emergent Roles
论文作者
论文摘要
角色概念为设计和理解复杂的多代理系统提供了有用的工具，该系统允许具有相似角色的代理人共享相似的行为。但是，现有的基于角色的方法使用先前的领域知识和预定的角色结构和行为。相比之下，多机构增强学习（MARL）提供了灵活性和适应性，但在复杂任务中的效率较低。在本文中，我们协同了这两个范式，并提出了面向角色的MARL框架（ROMA）。在此框架中，角色是紧急的，具有相似角色的代理人倾向于共享他们的学习，并专门研究某些子任务。为此，我们通过引入两个新的正规化器并根据角色调节各个策略来构建一个随机角色嵌入空间。实验表明，我们的方法可以学习专业，动态和可识别的角色，这有助于我们的方法推动了Starcraft II微管理基准上的最新技术。可在https://sites.google.com/view/romarl/上找到演示性视频。
The role concept provides a useful tool to design and understand complex multi-agent systems, which allows agents with a similar role to share similar behaviors. However, existing role-based methods use prior domain knowledge and predefine role structures and behaviors. In contrast, multi-agent reinforcement learning (MARL) provides flexibility and adaptability, but less efficiency in complex tasks. In this paper, we synergize these two paradigms and propose a role-oriented MARL framework (ROMA). In this framework, roles are emergent, and agents with similar roles tend to share their learning and to be specialized on certain sub-tasks. To this end, we construct a stochastic role embedding space by introducing two novel regularizers and conditioning individual policies on roles. Experiments show that our method can learn specialized, dynamic, and identifiable roles, which help our method push forward the state of the art on the StarCraft II micromanagement benchmark. Demonstrative videos are available at https://sites.google.com/view/romarl/.
