论文标题
通过可区分的归纳逻辑编程将关系背景知识纳入强化学习中
Incorporating Relational Background Knowledge into Reinforcement Learning via Differentiable Inductive Logic Programming
论文作者
论文摘要
关系增强学习（RRL）可以提供各种理想的功能。最重要的是，它允许将专家知识纳入学习中，从而与标准的深入强化学习相比，导致更快的学习和更好的概括。但是，大多数现有的RRL方法要么无法纳入专家背景知识（例如，以显式谓词语言的形式），要么无法直接从非关系数据（例如图像）中学习。在本文中，我们提出了一种基于可区分的归纳逻辑编程（ILP）的新型深度RRL，该RRL可以有效地从图像中学习关系信息，并将环境状态作为一阶逻辑谓词呈现。此外，它可以使用专家背景知识，并使用适当的谓词将其纳入学习问题。可区分的ILP允许对整个框架进行终点的优化，以学习RRL中的策略。我们使用Boxworld，Gridworld等环境以及CLEVR数据集的关系推理来显示这种新颖的RRL框架的功效。
Relational Reinforcement Learning (RRL) can offers various desirable features. Most importantly, it allows for incorporating expert knowledge into the learning, and hence leading to much faster learning and better generalization compared to the standard deep reinforcement learning. However, most of the existing RRL approaches are either incapable of incorporating expert background knowledge (e.g., in the form of explicit predicate language) or are not able to learn directly from non-relational data such as image. In this paper, we propose a novel deep RRL based on a differentiable Inductive Logic Programming (ILP) that can effectively learn relational information from image and present the state of the environment as first order logic predicates. Additionally, it can take the expert background knowledge and incorporate it into the learning problem using appropriate predicates. The differentiable ILP allows an end to end optimization of the entire framework for learning the policy in RRL. We show the efficacy of this novel RRL framework using environments such as BoxWorld, GridWorld as well as relational reasoning for the Sort-of-CLEVR dataset.
