Robotic Target Following Under Dynamic and Collaborative Vision Feedback

Abstract

Following rapidly and precisely a moving target has become the core functionality in robotic systems for transportation, manufacturing, and surgery. Among existing target following methods, vision-based tracking continues to thrive as one of the most popular and is the closest method to human perception. However, the slow sampling speed of vision sensors, the time delays of visual outputs, and the irregular processing time of image data fundamentally hinder real-time applications. Such limitations from the visual sensing dynamics need special attention when following fast-moving targets. Ignoring the visual sensing dynamics has caused unstable and even unsafe robot behavior. This dissertation provides fast robotic target following considering the sensing dynamics. Starting from the fundamental problem in disturbance compensation where the sampling rate of the feedback sensor is insufficient to capture the high-frequency dynamics of the disturbance signal, we propose an information recovering algorithm to reconstruct fast-sampled signals from slowly-sampled ones. The reconstruction is made possible by re-parameterizing the dense autoregressive models of signals to a sparse structure. To reduce the recovering error at lower signal-to-noise ratios, we improve from the new autoregressive model to a framework using infinite impulse response (IIR) filter structures. Furthermore, the idea of signal recovering is extended to a collaborative sensing system, where two sensors — each sampling at a different rates — collaborate to provide fast signal recovering significantly beyondthe Nyquist frequency of individual sensors. The information recovering algorithm has been engineered into an extended visual servoing control system for a high-speed target following. The extended visual servoing compensates for the following error caused by target motions by estimating the target velocities online, while the recovering algorithm compensates for the slow-sampling and time delays. Under a more challenging scenario where there are multiple feedback signal flows with irregular sampling intervals and variable delays, we propose a memory-enabled auto-restart Kalman filter (M-ARKF) to compensate for the full sensing dynamics, and additionally handle any jumping velocities of the tracked object. Simulation and experimentation on a dual-arm robotic manipulator and a robotic air-hockey player validate the proposed algorithms.