Broad Generalization through Domain Transfer: Abstractions and Algorithms

Abstract

Deep learning and reinforcement learning have recently had a transformative impact on the fields of computer vision, NLP, and robotics. However, most of the recent progress have been in narrowly defined tasks where training and deployment happen under the same (or similar) conditions. This has resulted in brittle models that catastrophically fail when presented with conditions even moderately different from what they were trained on. Furthermore, current approaches are known to be data hungry, and thus may require prohibitively large datasets for many applications. How can we create intelligent agents that are data-efficient, robust, capable of broad generalization, and fast adaptation? In this thesis, we outline the importance of domain transfer as a key component to achieve the aforementioned capabilities. Domain transfer refers to the ability of an AI agent to draw upon experiences from related tasks and transfer inductive biases, enabling more efficient and proficient learning in downstream tasks. This thesis presents abstractions and algorithms to enable such domain transfer. In particular, we focus on domain transfer that arises in the context of simulation to reality transfer in robotics, learning from static offline datasets in reinforcement learning, and efficient adaptation to new tasks in the case of meta-learning. The algorithms we present enjoy rigorous theoretical guarantees and also demonstrate strong empirical results in a variety of benchmark tasks and real-world case studies spanning perception, control, and robotics.