Designing Closed-loop Sensing and Feedback Systems for Embodied Human Interaction

Abstract

Wearable and interactive systems create new ways to support embodied human interaction in real-world activities such as learning, physical training, and rehabilitation. In these activities, people rely on their bodies, tools, and surroundings to understand what is happening and decide what to do next. However, many existing systems still focus mainly on accurate sensing or computation, without fully supporting this ongoing process of sensing, acting, and adjusting. This thesis asks how interactive systems can support effective embodied human interaction in real-world use. To answer this question, this thesis studies the design of closed-loop sensing and feedback systems through three human-centered projects. The first project, \textit{BrailleRhythm}, supports early Braille learning by augmenting printed Braille cards with a flexible piezoresistive sensor, achieving over 92\% Braille recognition accuracy. The system senses finger interactions on Braille symbols and provides real-time audio feedback, helping learners connect tactile symbols with their corresponding sounds. The second project is a closed-loop CPR self-training glove that helps users practice CPR outside of traditional training settings. The glove combines distributed tactile sensing and vibrotactile feedback to evaluate compression rate, force, and hand pose in real time, while reducing the need for external audio-visual interfaces. The third project, \textit{MultiSensKnit}, is a multimodal knitted textile sleeve for rehabilitation monitoring. It integrates electromyography (EMG), electrical impedance sensing (EIS), and capacitive sensing into a wearable sleeve to capture muscle activity, tissue changes, and joint movement, and uses these signals to support real-time visualization and behavior understanding. Across these projects, this thesis identifies design insights along three dimensions: form factor, sensing, and feedback. Form factor should preserve natural interaction through unobtrusive design, while balancing durability, portability, wearability, and signal quality. Sensing should capture task-relevant signals, while minimizing hardware complexity and modality interference. Feedback should match the task context, while remaining low-latency, low-burden, and clear across multimodal combinations. Together, these projects show that effective embodied interaction is not achieved by simply adding sensors or feedback. It requires designing the physical form, sensing method, and feedback strategy as one closed loop within the activity itself.