论文标题
具有战略互补性的平均现场游戏的强化学习
Reinforcement Learning for Mean Field Games with Strategic Complementarities
论文作者
论文摘要
平均野外游戏（MFG）是具有大量代理商的游戏类别，标准平衡概念是平均场均衡（MFE）。一般来说，用于学习动态MFG中MFE的算法是未知的。我们的重点是具有称为“战略互补”（MFG-SC）的单调性能的重要子类。我们对平衡概念介绍了一种自然的改进，我们称我们称为颤抖的手，完美的MFE（T-MFE），该概念使代理人可以采用随机化度量，同时考虑到这种随机分配对他们的收益的影响。我们提出了一种用于在已知模型下计算T-MFE的简单算法。我们还引入了一种学习T-MFE的无模型和基于模型的方法，并提供两种算法的样本复杂性。我们还制定了一种完全在线学习方案，该方案避免了对模拟器的需求。最后，我们通过由现实世界应用激励的示例从经验上评估所提出的算法的性能。
Mean Field Games (MFG) are the class of games with a very large number of agents and the standard equilibrium concept is a Mean Field Equilibrium (MFE). Algorithms for learning MFE in dynamic MFGs are unknown in general. Our focus is on an important subclass that possess a monotonicity property called Strategic Complementarities (MFG-SC). We introduce a natural refinement to the equilibrium concept that we call Trembling-Hand-Perfect MFE (T-MFE), which allows agents to employ a measure of randomization while accounting for the impact of such randomization on their payoffs. We propose a simple algorithm for computing T-MFE under a known model. We also introduce a model-free and a model-based approach to learning T-MFE and provide sample complexities of both algorithms. We also develop a fully online learning scheme that obviates the need for a simulator. Finally, we empirically evaluate the performance of the proposed algorithms via examples motivated by real-world applications.
