论文标题
推理等计划执行人
Reasoning Like Program Executors
论文作者
论文摘要
关于自然语言的推理是研究界的长期目标。但是，研究表明，现有的语言模型在推理方面不足。为了解决这个问题，我们提出了诗人，这是一种新颖的推理预训练范式。通过具有程序及其执行结果的预培训语言模型，诗人使语言模型通过数据驱动的方法收获了程序执行者所拥有的推理知识。诗人在概念上很简单，可以由不同种类的程序执行者实例化。在本文中，我们展示了两个简单的实例，除了复杂的实例诗人-SQL之外，我们还展示了诗人的记录和诗人逻辑。六个基准的实验结果表明，诗人可以显着提高自然语言推理中的模型表现，例如数值推理，逻辑推理和多跳推理。诗人为推理增强预培训开了一个新的大门，我们希望我们的分析能够阐明未来对计划执行者等推理的研究。
Reasoning over natural language is a long-standing goal for the research community. However, studies have shown that existing language models are inadequate in reasoning. To address the issue, we present POET, a novel reasoning pre-training paradigm. Through pre-training language models with programs and their execution results, POET empowers language models to harvest the reasoning knowledge possessed by program executors via a data-driven approach. POET is conceptually simple and can be instantiated by different kinds of program executors. In this paper, we showcase two simple instances POET-Math and POET-Logic, in addition to a complex instance, POET-SQL. Experimental results on six benchmarks demonstrate that POET can significantly boost model performance in natural language reasoning, such as numerical reasoning, logical reasoning, and multi-hop reasoning. POET opens a new gate on reasoning-enhancement pre-training, and we hope our analysis would shed light on the future research of reasoning like program executors.
