Robot Rubik's Cube Solving: Pre-touch Sensing for Sequential Manipulation

Abstract

Robotics manipulation accuracy highly depends on the robot's belief about the relative pose between object and manipulator. While most robots achieve object pose estimation through computer vision via head-mounted cameras, robotics manipulation accuracy is limited by this perception method. A major limitation comes from our inability to perfectly calibrate both the cameras' parameters and the coordinate frames of robot arms. Moreover, when a task requires a robot to perform a long sequence of manipulation, manipulation errors could be accumulated a long time and eventually result in an irreversible failure to the task. The focus of this work is to examine how robots can achieve more robust sequential manipulation through the use of pre-touch sensors. The utility of close-range proximity sensing is evaluated through a robotic system that uses a new optical time-of-flight pre-touch sensor to complete a highly precise and sequential task - solving the Rubik's cube. The techniques used in this task are then extended to a more general framework in which ICP is used to match pre-touch data to a reference model, demonstrating that even simple pre-touch scans can be used to recover the pose of common objects that require sequential manipulation. Two pre-touch sensing region detection algorithms are introduced to detect object regions that contain distinctive geometric features. By focusing pre-touch sensing on these regions, the robot can more efficiently gather the information necessary to adjust its original pose estimate.