Evaluation of Haptic Virtual Fixtures with Real-Time Sensors

Abstract

Haptic virtual fixtures and haptic display can provide critical operator feedback and aid in completing teleoperated tasks. This is particularly true for cases in which visual information is not sufficient for the specific task, e.g. attempting to observe and manipulate a transparent object. This work improves upon one particular aspect of user feedback: conveying information to the user through the operator’s sense of touch. Not only is feedback transparency of interest, but so too is the use of intelligent use of force feedback in the form of haptic virtual fixtures. Virtual fixtures help to modify the user’s performance by encouraging or limiting their motions. In practice, providing touch or haptic feedback can be achieved with a priori or real-time information. The former presents problems in that virtual fixtures designed this way are useful only in predefined, well-known and unchanging scenarios. One promising implementation of the latter utilizes real-time streaming point clouds provided by commercially available RGB-Depth (RGB-D) cameras. Problems persist, however. For example, missing or gaps in point cloud data can be caused by material properties, lighting conditions, distance and other factors. Surface geometries are a useful but not exhaustive set of information for teleoperation tasks. One solution is to augment the sparse/missing point cloud with additional geometry information provided by auxiliary sensors, or incorporate additional sensory information which RGB-D cameras are not suited to capture. Furthermore, feedback qualities in telerobotic interaction can impact user performance. Novel visualization and haptic feedback modes are of interest, and use cases of these scenarios are demonstrated on real telerobotic platforms in this thesis.