Curating Visual Information in Sequential Decision Making Problems

Abstract

Images often contain both too much and not enough information. When an image is cluttered and contains many irrelevant details, this extra information can make it challenging to learn which parts are necessary, and how to act accordingly. At the same time, an image is a narrow 2D slice of a 3D world. This means that factors such as object occlusions and a limited field of view necessarily limit the amount of information available at any given time. These two problems, informational clutter and partial observability fundamentally concern the agent’s access to information. When the agent has too much information, it must learn to discard that which is irrelevant. When it does not have enough, the agent must come up with information gathering actions to acquire the information that it lacks. We explore these issues in a variety of sequential decision making problems in which an agent must repeatedly interact with an environment in order to accomplish some objective. These settings include goal-conditioned object retrieval in a simulated kitchen, part-based assembly problems using LEGO bricks, and escaping giant monsters in the first-person video game DOOM. In all of these settings, we show that by carefully considering an agent’s access to information, we can significantly improve its performance.