论文标题
分析机器人操纵任务中强化学习中SIM2REAL转移的随机影响
Analysis of Randomization Effects on Sim2Real Transfer in Reinforcement Learning for Robotic Manipulation Tasks
论文作者
论文摘要
当前,随机化是机器人技术中数据驱动的学习算法的SIM2REAL传输中广泛使用的方法。尽管如此,大多数SIM2REAL研究都报告了特定随机技术的结果,并且通常是在高度定制的机器人系统上,因此很难系统地评估不同的随机方法。为了解决这个问题,我们为机器人触及余量操纵器任务定义了易于生产的实验设置,该设置可以作为比较的基准。我们将四个随机策略与模拟和真实机器人中的三个随机参数进行比较。我们的结果表明,更多的随机化有助于SIM2REAL转移,但它也可能损害该算法在模拟中找到良好策略的能力。完全随机的模拟和微调显示出差异化的结果,并且比测试的其他方法更好地转化为实际机器人。
Randomization is currently a widely used approach in Sim2Real transfer for data-driven learning algorithms in robotics. Still, most Sim2Real studies report results for a specific randomization technique and often on a highly customized robotic system, making it difficult to evaluate different randomization approaches systematically. To address this problem, we define an easy-to-reproduce experimental setup for a robotic reach-and-balance manipulator task, which can serve as a benchmark for comparison. We compare four randomization strategies with three randomized parameters both in simulation and on a real robot. Our results show that more randomization helps in Sim2Real transfer, yet it can also harm the ability of the algorithm to find a good policy in simulation. Fully randomized simulations and fine-tuning show differentiated results and translate better to the real robot than the other approaches tested.