Volitional Control of Lower-limb Prosthesis with Vision-assisted Environmental Awareness

Early and reliable prediction of user’s intention to change locomotion mode or speed is critical for a smooth and natural lower limb prosthesis. Meanwhile, incorporation of explicit environmental feedback can facilitate context aware intelligent prosthesis which allows seamless operation in a variety of gait demands. This dissertation introduces environmental awareness through computer vision and enables early and accurate prediction of intention to start, stop or change speeds while walking. Electromyography (EMG), Electroencephalography (EEG), Inertial Measurement Unit (IMU), and Ground Reaction Force (GRF) sensors were used to predict intention to start, stop or increase walking speed. Furthermore, it was investigated whether external emotional music stimuli could enhance the predictive capability of intention prediction methodologies. Application of advanced machine learning and signal processing techniques on pre-movement EEG resulted in an intention prediction system with low latency, high sensitivity and low false positive detection. Affective analysis of EEG suggested that happy music stimuli significantly (

Environment-based Assistance Modulation for a Hip Exosuit via Computer Vision

Just like in humans vision plays a fundamental role in guiding adaptive locomotion, when designing the control strategy for a walking assistive technology, Computer Vision may bring substantial improvements when performing an environment-based assistance modulation. In this work, we developed a hip exosuit controller able to distinguish among three different walking terrains through the use of an RGB camera and to adapt the assistance accordingly. The system was tested with seven healthy participants walking throughout an overground path comprising of staircases and level ground. Subjects performed the task with the exosuit disabled (Exo Off), constant assistance profile (Vision Off ), and with assistance modulation (Vision On). Our results showed that the controller was able to promptly classify in real-time the path in front of the user with an overall accuracy per class above the 85%, and to perform assistance modulation accordingly. Evaluation related to the effects on the user showed that Vision On was able to outperform the other two conditions: we obtained significantly higher metabolic savings than Exo Off, with a peak of about -20% when climbing up the staircase and about -16% in the overall path, and than Vision Off when ascending or descending stairs. Such advancements in the field may yield to a step forward for the exploitation of lightweight walking assistive technologies in real-life scenarios

Effect of lower limb exoskeleton on the modulation of neural activity and gait classification

: Neurorehabilitation with robotic devices requires a paradigm shift to enhance human-robot interaction. The coupling of robot assisted gait training (RAGT) with a brain-machine interface (BMI) represents an important step in this direction but requires better elucidation of the effect of RAGT on the user's neural modulation. Here, we investigated how different exoskeleton walking modes modify brain and muscular activity during exoskeleton assisted gait. We recorded electroencephalographic (EEG) and electromyographic (EMG) activity from ten able-bodied volunteers walking with an exoskeleton with three modes of user assistance (i.e., transparent, adaptive and full assistance) and during free overground gait. Results identified that exoskeleton walking (irrespective of the exoskeleton mode) induces a stronger modulation of central mid-line mu (8-13 Hz) and low-beta (14-20 Hz) rhythms compared to free overground walking. These modifications are accompanied by a significant re-organization of the EMG patterns in exoskeleton walking. On the other hand, we observed no significant differences in neural activity during exoskeleton walking with the different assistance levels. We subsequently implemented four gait classifiers based on deep neural networks trained on the EEG data during the different walking conditions. Our hypothesis was that exoskeleton modes could impact the creation of a BMI-driven RAGT. We demonstrated that all classifiers achieved an average accuracy of 84.13 ± 3.49% in classifying swing and stance phases on their respective datasets. In addition, we demonstrated that the classifier trained on the transparent mode exoskeleton data can classify gait phases during adaptive and full modes with an accuracy of 78.3 ± 4.8%, while the classifier trained on free overground walking data fails to classify the gait during exoskeleton walking (accuracy of 59.4 ± 11.8%). These findings provide important insights into the effect of robotic training on neural activity and contribute to the advancement of BMI technology for improving robotic gait rehabilitation therapy

Identification of gait phases with neural networks for smooth transparent control of a lower limb exoskeleton

Lower limbs exoskeletons provide assistance during standing, squatting, and walking. Gait dynamics, in particular, implies a change in the configuration of the device in terms of contact points, actuation, and system dynamics in general. In order to provide a comfortable experience and maximize performance, the exoskeleton should be controlled smoothly and in a transparent way, which means respectively, minimizing the interaction forces with the user and jerky behavior due to transitions between different configurations. A previous study showed that a smooth control of the exoskeleton can be achieved using a gait phase segmentation based on joint kinematics. Such a segmentation system can be implemented as linear regression and should be personalized for the user after a calibration procedure. In this work, a nonlinear segmentation function based on neural networks is implemented and compared with linear regression. An on-line implementation is then proposed and tested with a subject

A sEMG-driven Musculoskeletal Model to Control Exoskeleton Robot Used in Lower Extremity Rehabilitation

A control system framework of lower extremity rehabilitation exoskeleton robot is presented. It is based on the Neuro-Musculo-Skeletal biological model. Its core composition module, the motion intent parser part, mainly comprises of three distinct parts. The first part is signal acquisition of surface electromyography (sEMG) that is the summation of motor unit action potential (MUAP) starting from central nervous system (CNS).sEMG can be used to decode action intent of operator to make the patient actively participate in specific training .As another composition part, a muscle dynamics model that is comprised of activation and contraction dynamic model is developed. It is mainly used to calculate muscle force. The last part is the skeletal dynamic model that is simplified as a linked segment mechanics. Combined with muscle dynamic model, the joint torque exerted by internal muscles can be exported, which can be used to do a exoskeleton controller design. The developed control framework can make exoskeleton offer assistance to operators during rehabilitation by guiding motions on correct training rehabilitation trajectories, or give force support to be able to perform certain motions. Though the presentation is orientated towards the lower extremity exoskeleton, it is generic and can be applied to almost any part of the human body

Robotic and Sensor Technologies for Mobility in Older People

Maintaining independent mobility is fundamental to independent living and to the quality of life of older people. Robotic and sensor technologies may offer a lot of potential and can make a significant difference in the lives of older people and to their primary caregivers. The aim of this study was to provide a presentation of the methods that are used up till now for analysis and evaluation of human mobility utilizing sensor technologies and to give the state of the art in robotic platforms for supporting older people with mobility limitations. The literature was reviewed and systematic reviews of cohort studies and other authoritative reports were identified. The selection criteria included (1) patients with age â\u89¥60 years; (2) patients with unstable gait, with or without recurrent falls; (3) patients with slow movements, short strides, and little trunk movement; (4) sensor technologies that are currently used for mobility evaluation; and (5) robotic technologies that can serve as a supporting companion for older people with mobility limitations. One hundred eighty-one studies published up until February 2017 were identified, of which 36 were included. Two categories of research were identified from the review regarding the robot and sensor technologies: (1) sensor technologies for mobility analysis and (2) robots for supporting older people with mobility limitations. Potential for robotic and sensor technologies can be taken advantage of for evaluation and support at home for elder persons with mobility limitations in an automated way without the need of the physical presence of any medical personnel, reducing the stress of caregivers

Human Activity Recognition and Control of Wearable Robots

abstract: Wearable robotics has gained huge popularity in recent years due to its wide applications in rehabilitation, military, and industrial fields. The weakness of the skeletal muscles in the aging population and neurological injuries such as stroke and spinal cord injuries seriously limit the abilities of these individuals to perform daily activities. Therefore, there is an increasing attention in the development of wearable robots to assist the elderly and patients with disabilities for motion assistance and rehabilitation. In military and industrial sectors, wearable robots can increase the productivity of workers and soldiers. It is important for the wearable robots to maintain smooth interaction with the user while evolving in complex environments with minimum effort from the user. Therefore, the recognition of the user's activities such as walking or jogging in real time becomes essential to provide appropriate assistance based on the activity. This dissertation proposes two real-time human activity recognition algorithms intelligent fuzzy inference (IFI) algorithm and Amplitude omega ($A \omega$) algorithm to identify the human activities, i.e., stationary and locomotion activities. The IFI algorithm uses knee angle and ground contact forces (GCFs) measurements from four inertial measurement units (IMUs) and a pair of smart shoes. Whereas, the $A \omega$ algorithm is based on thigh angle measurements from a single IMU. This dissertation also attempts to address the problem of online tuning of virtual impedance for an assistive robot based on real-time gait and activity measurement data to personalize the assistance for different users. An automatic impedance tuning (AIT) approach is presented for a knee assistive device (KAD) in which the IFI algorithm is used for real-time activity measurements. This dissertation also proposes an adaptive oscillator method known as amplitude omega adaptive oscillator ($A\omega AO$) method for HeSA (hip exoskeleton for superior augmentation) to provide bilateral hip assistance during human locomotion activities. The $A \omega$ algorithm is integrated into the adaptive oscillator method to make the approach robust for different locomotion activities. Experiments are performed on healthy subjects to validate the efficacy of the human activities recognition algorithms and control strategies proposed in this dissertation. Both the activity recognition algorithms exhibited higher classification accuracy with less update time. The results of AIT demonstrated that the KAD assistive torque was smoother and EMG signal of Vastus Medialis is reduced, compared to constant impedance and finite state machine approaches. The $A\omega AO$ method showed real-time learning of the locomotion activities signals for three healthy subjects while wearing HeSA. To understand the influence of the assistive devices on the inherent dynamic gait stability of the human, stability analysis is performed. For this, the stability metrics derived from dynamical systems theory are used to evaluate unilateral knee assistance applied to the healthy participants.Dissertation/ThesisDoctoral Dissertation Aerospace Engineering 201