论文标题
rllib流量：分布式增强学习是一个数据流问题
RLlib Flow: Distributed Reinforcement Learning is a Dataflow Problem
论文作者
论文摘要
增强学习领域（RL）领域的研究人员和从业人员经常利用并行计算，这在过去几年中导致了许多新算法和系统。在本文中，我们重新检查了分布式RL提出的挑战，并尝试通过一个旧想法的镜头进行查看：分布式数据流。我们表明将RL视为数据流问题会导致高度合成和性能的实现。我们提出了Rllib Flow，这是一种用于分布式RL的混合Actor-Dataflow编程模型，并通过在RLLIB（广泛采用的分布式RL库）中移植完整的算法套件来验证其实用性。具体而言，RLLIB流提供了2-9代码在实际生产代码中节省，并在最终用户不可能的多代理算法中启用了多代理算法的组成。开源代码可作为RLLIB的一部分获得，网址为https://github.com/ray-project/ray/ray/ray/tree/master/rllib。
Researchers and practitioners in the field of reinforcement learning (RL) frequently leverage parallel computation, which has led to a plethora of new algorithms and systems in the last few years. In this paper, we re-examine the challenges posed by distributed RL and try to view it through the lens of an old idea: distributed dataflow. We show that viewing RL as a dataflow problem leads to highly composable and performant implementations. We propose RLlib Flow, a hybrid actor-dataflow programming model for distributed RL, and validate its practicality by porting the full suite of algorithms in RLlib, a widely adopted distributed RL library. Concretely, RLlib Flow provides 2-9 code savings in real production code and enables the composition of multi-agent algorithms not possible by end users before. The open-source code is available as part of RLlib at https://github.com/ray-project/ray/tree/master/rllib.
