A Learning Approach for Extending Human-Robot Collaboration to Manufacturing-Specific Tasks

Abstract

This thesis presents the development and evaluation of methods for extending shared autonomy to limited-access manufacturing telerobotics. Shared teleoperation has potential to reduce strenuous working conditions and increase process efficiency in this application domain. However, current methods for shared autonomy in such applications are limited by: (Q1) difficulty handling pose errors that arise from uncertain placement of the manipulator; (Q2) fragility to off-nominal situations that have potential to degrade system performance; and (Q3) difficulty automating the physical tasks prevalent in limited-access operations. The main contribution of this thesis is an imitation learning method that produces dynamical models of a manufacturing task in order to address these limitations. The method (i) learns a structured model of the data, including positions, velocities, accelerations, and forces; (ii) performs a state-action decomposition of the model; and (iii) constructs dynamical models to describe the motion and forces for each action, as well as the sequence of actions. The resulting model of the task dynamics enables the following contributions. (C1) To address Q1, this work uses motion and force feedback data during human teleoperation to localize the target position for the task, and trades control between human and autonomy based on localization uncertainty. The challenges are how to produce a reliable estimate within the required tolerance to enable automation, and how to handle human-robot disagreements that arise due to estimate uncertainty. Use of the task dynamics model addresses the first challenge by providing a sufficient likelihood for observed positions and accelerations during task teleoperation, and a procedure is developed to address the second challenge. (C2) To address Q2, this work trades control to the human in off-nominal situations. The challenge is how to detect off-nominal situations. Use of the task dynamics model addresses this challenge by providing an expectation of interaction forces during task automation. (C3) To address Q3, this work uses imitation learning to develop a control policy for automation that mimics an expert operator. The challenge is how to imitate demonstrations that are not classical reaching movements, i.e. there is no clear target state. The developed task dynamics learning approach automates the isolation of target states to develop the sequence of actions in between them, thus providing reference motions and forces to which the system can be regulated during task automation. The final contribution (C4) experimentally evaluates the methods for an aerospace-manufacturing hole cleaning task. (E1) Use of control trading based on localization estimate uncertainty from C1 reduces completion times for a hole locating task by 50% as compared with teleoperation. (E2) Use of kinetics predictions to address off-nominal situations in assisted teleoperation from C2 reduces completion time by 17% and operator forces by 68% as compared with assistance without the method. (E3) Use of kinetics predictions from C3 as feedforward commands reduces tracking errors in position (61%), velocity (57%), and force (53%), as compared with feedback compensation alone during task automation. (E4) In concert, the contributions enable shared autonomy in a user study (n=8) to reduce completion time by 54.0%, operator energy expenditure by 80.5%, and operator forces by 44.0% as compared with teleoperation. These results illustrate the potential of the thesis contribution to improve process efficiency and mitigate strenuous work conditions for a class of manufacturing tasks.