Real-Time Haptic Interaction with Remote Environments using Non-contact Sensors

Abstract

Haptic interaction with a haptic device has traditionally been limited to interaction with purely virtual objects or to static range scans of physical objects. Haptic interaction with physical objects at a distance has not been possible without the use of a remote contact manipulator. This dissertation describes methods and algorithms for real-time haptic interaction with remote environments using non-contact sensors; remote touch. A method of rendering 3-DOF (degree-of-freedom) force feedback of streaming point cloud data (from a combined range/video sensor) is presented. This is further combined with a novel method for increasing the robustness in interaction with moving physical that is also evaluated. These methods are then extended to demonstrate that forbidden-region virtual fixtures for telemanipulation can be rendered by extending the concept of remote touch to rendering of haptic virtual fixtures. Both of these methods are then extended to 6-DOF haptic interaction with force as well as torque feedback. The presented methods of remote touch rendering and virtual fixture rendering were validated by implementation on commercially available hardware. The methods for remote touch successfully renders force feedback at 1000 Hz based on the user's interaction with the streaming point cloud data. The methods for rendering of virtual fixtures are first demonstrated to protect an isolated beating heart using a forbidden-region virtual fixture and then to keep the user away from, as well as guide the user to, underwater valves.