End-to-End Programming Tools for Mobile Manipulator Robots

Abstract

Mobile manipulator robots have the potential to help people in a variety of unstructured scenarios, such as in households or in the service industry. However, with so many possible scenarios, roboticists cannot pre-program every task the robot needs to do. Instead, we need tools that make robot programming simpler, faster, and accessible to a wider audience of programmers. To this end, this dissertation presents research on two main approaches to robot programming for non-expert users: direct programming and programming by imitation. These approaches, and the technologies that support them, are situated in a conceptual framework comprising three key components: 1) perception, 2) motion specification, and 3) task scripting. In the realm of perception, we present a user-friendly system for specifying and locating task-relevant landmarks and a novel, state-of-the-art shelf detection algorithm. In the realm of motion specification, we enhanced existing programming by demonstration systems and created a new system for programming robots by visual imitation of a human demonstrator. Lastly, we developed a task scripting system that was deployed on a commercial robot. Throughout our work, we used system-level experiments, user studies, and case studies to show that non-roboticists could quickly learn to use our systems and program useful robot tasks. We conclude by describing possible extensions to our framework and envisioning directions for future work.