Sub-millimeter Acoustic Tracking for Medical and VR Applications

Abstract

Recent years witness the proliferation of consumer devices such as smartphones, VR headsetsand smart speakers. While the speakers and microphones on those devices are a greatfit to localize and track motion over-the-air using sonar techniques, it is challenging toreliably track motion with millimeter or even sub-millimeter resolution, which hinders theirapplications.My dissertation introduces algorithms and systems to achieve millimeter and sub-millimeteracoustic tracking given the constraints of existing commodity devices. Towards this end, Idesign, implement and evaluate three key innovations. I present the first smartspeaker sys-tems that can contactlessly monitor respiration in real time on new-born infants as well asindividual heart beats in both healthy participants and cardiac patients with irregular heartrhythms. I also design real-time signal processing algorithms that can track acoustic deviceswith millimeter-level resolution. The technique can be applied in both VR/AR applicationsas well as low-power tag localization. Finally, with the various capability to obtain the pre-cise location of sound sources, we combine the efficiency of traditional signal processing andthe capacity of deep neural network to build the first wearable directional hearing systemwith low latency and on-device computation.