Publication
Program Synthesis Guided Reinforcement Learning
AbstractA key challenge for reinforcement learning is solving long-horizon planning and control problems. Recent work has proposed leveraging programs to help guide the learning algorithm in these settings. However, these approaches impose a high manual burden on the user since they must provide a guiding program for every new task they seek to achieve. We propose an approach that leverages program synthesis to automatically generate the guiding program. A key challenge is how to handle partially observable environments. We propose model predictive program synthesis, which trains a generative model to predict the unobserved portions of the world, and then synthesizes a program based on samples from this model in a way that is robust to its uncertainty. We evaluate our approach on a set of challenging benchmarks, including a 2D Minecraft-inspired “craft” environment where the agent must perform a complex sequence of subtasks to achieve its goal, a box-world environment that requires abstract reasoning, and a variant of the craft environment where the agent is a MuJoCo Ant. Our approach significantly outperforms several baselines, and performs essentially as well as an oracle that is given an effective program.
Download publicationRelated Resources
See what’s new.
12/07/2022
Supercharging Trial-and-Error for Learning Complex Software Applications
Despite an abundance of carefully-crafted tutorials, trial-and-error…
01/01/2015
Automatic Extraction of Function Knowledge from Text
This paper presents a method to automatically extract function…
01/01/2021
Validating a Termite-Inspired Construction Coordination Mechanism Using an Autonomous Robot
Many species of termites build large, structurally complex mounds, and…
01/01/2018
Unsupervised Image to Sequence Translation with Canvas-Drawer Networks
Encoding images as a series of high-level constructs, such as brush…
Get in touch
Something pique your interest? Get in touch if you’d like to learn more about Autodesk Research, our projects, people, and potential collaboration opportunities.
Contact us