Using wearable sensors and machine learning to enrich lower limb assistive devices

Abstract

In the past, our understanding of how humans move has been greatly restricted because of the tools available to us. Firstly, we had limited gait datasets, which mostly captured simple activities like walking in a straight line at a steady pace or walking on a treadmill. Interesting activities like dancing or obstacle avoidance were largely confined to laboratories, which did not necessarily reflect the complexities and variations that may arise from interactions in the real world. Secondly, gait modeling tools were also limited. So, modeling techniques had to make strong assumptions about human gait like we use cyclic repetitive movement of limbs or that we select from a finite repertoire of activities like flatground walking, stair ascent, descent etc. This has greatly reflected in how we design control for our assistive devices. Thus, most existing control strategies select from a finite list of activities with corresponding predefined gait profiles, which are then repeated in a periodic manner. Even for common activities like flatground walking slight variations that might arises due to interactions with the environment like going around an obstacle are largely ignored. The variations across people are also not accounted for when controlling assistive devices due to extensive tuning required. This dissertation aims at proposing solutions to these challenges by using recent advances in wearable sensors and machine learning. In recent times, there have been great improvements in wearable motion capture systems as well as wearable cameras, and these are expected to grow further in the next few years. This allows human movement tracking in real world environments, as well as recording how humans react to the objects in their environment, and navigate around them. These large datasets then allow the use of data-driven modeling techniques to better understand human movement, and improve assistive devices. We combine the technological improvements in wearable sensing, with improvements in data-driven modeling to address the 4 key current limitations. 1) We tackle the problem of limited gait models in coordinated movement, where we propose an expressive predictive model of gait using recurrent neural networks, which can predict gait for a variety of activities, and does not rely on assumptions about periodicity of gait, or a clearly defined finite set of activities. 2) We analyze how gait predictability is affected by environments, and investigate the use of environment information in improving the predictive models of gait. 3) We investigate how to quantify and compare the complexity of different activities, and provide practical recommendations about the usage of different measures of complexity. 4) We develop a style transfer based solution to personalize assistive devices, with the aim of reducing the effort required for manual tuning, and also propose how to quantify personal style of a user. The dataset generated as a part of this dissertation is freely available and can be accessed through the following github repository- https://github.com/abs711/The-way-of-the-future