Learning Generalizable Robot Policies with Targeted Generative Augmentation

Abstract

Visual Imitation Learning methods have the potential to generalize across diverse tasks and environments but often encounter challenges in acquiring large and diverse datasets due to the high costs of real-world data collection. This dissertation introduces Targeted Generative Augmentation, a systematic approach designed to generate data that closely mimics real-world distributions. This method aims to effectively diversify the initial dataset that robots are more likely to encounter, thereby bridging the data scarcity gap. In this thesis, I present realistic data generation techniques that help robots generalize to unseen scenarios. The dissertation outlines three main approaches: Static Targeted Augmentation for Visual Diversity, Dynamic Targeted Augmentation for Physical Realism, and Contextual Targeted Augmentation for Real-World Scene Generation. These methods generate data with visual realism and the complexity of the real world, which help to bootstrap the robot policy and improve generalization in unfamiliar environments.