Data-Driven Fine Manipulation

Abstract

We are driven to build robots that operate outside specialized factories and automate precision-critical tasks with sub-millimeter accuracy, such as assistive homecare, surgery, harvesting, and agile manufacturing. We ask, how to enable general-purpose robots to learn, adapt, and improve fine motor skills in uncontrolled environments? Current robotic systems tend to be either precise or general, but rarely both. Robots designed for specific use cases rely on controller environments and specific tooling to offer high precision, but are costly to deploy to novel tasks in less structured settings. Conversely, the learning methods for developing general systems have not demonstrated necessary granularity for sub-millimeter accuracy. Challenges for robots to generalize to high precision tasks can come from both the robot and the data: inaccuracy in execution, scarcity of data and suboptimality of demonstrations. We emphasize the need for general-purpose robots capable of fine manipulation in diverse and uncontrolled environments. This dissertation describes three steps that sought to make more practical and generalizable robotic fine manipulator by developing data-driven algorithms and systems.The first part introduces a general-purpose robot platform for fine manipulation and uses it as a testbed to characterize the challenge. The second project learns from expert data via imitation learning but concerns the compounding errors problem - small execution or prediction mistakes accumulate along trajectory, leading to increasingly significant deviations from expert data. The third project explores learning beyond expert data in a data-efficient manner to address difficult tasks whose complexity limits the data quantity and quality. An emergent theme through these works is that, while we aim to develop data-driven methods to address fine manipulation challenges in a general way, leveraging structures and priors, such as dynamics and constraints, significantly enhances the efficiency and robustness of the learning process. We conclude with a conceptual analysis of moving data-driven robotics forward - as models and data keep improving, what key challenges would likely persists and what structures can be useful.