Innovative Rehabilitation Approach for Upper Limb Neurologic Conditions Using Mixed-Reality Simulation and EEG/EMG Biofeedback

Abstract

Recent advances in Augmented Reality (AR), Mixed Reality (MR), Electroencephalogram (EEG), and Electromyogram (EMG) offer significant opportunities in medicine and neuroscience. This research aims to use these technologies to aid stroke patients with upper limb extremity weakness. This work extends the Edge Computing Ecosystem for Neuroscience Patients' Rehabilitation, part of the 'Stroke Rehabilitation Project' by University of Washington Bothell Engineering, University of Washington Seattle Neuroscience, and Rehabilitation Medicine at Harborview Medical Center (UWHM). Recently, UW Bothell's CSSE has also contributed, focusing on solutions for stroke patient rehabilitation. Traditional motor rehabilitation is costly, resource intensive, and often monotonous, reducing patient engagement. Using augmented and mixed reality technologies, interactive environments can be created on mobile devices, providing engaging and motivating experiences for patients. AR simulates real-world scenarios, offering a safe and fun way to practice tasks and aid in rehabilitation. We used EEG and EMG sensors to conduct experiments and to collect data in a controlled environment targeting a reduced set of representative relevant motor tasks. The data were processed using various signal processing and statistical techniques, which in combination with the MR / AR simulation can be used to build a novel feedback and guidance system. This system is a building block of our ``NeuroRehab'' ecosystem, which will use various ML models and algorithms for sequential prediction, with the aim of guiding patients through an optimal rehabilitation path within the 3D simulation environment. Results showed that the combination of Frequency Filtering, ICA, and ERP with CNN model yield, so far, the best accuracy for classifying the motor tasks in EEG and EMG data. These findings contribute to the field of stroke rehabilitation of upper limb extremity weakness. It also contributes to a larger project that aims for better understanding and rehabilitation of other neurological ailments by offering insights to different hand gestures using EMG, and EEG data, and creating a framework for data processing, and feedback systems.