Towards Intelligent Lower Limb Prostheses with Activity Recognition

User’s volitional control of lower limb prostheses is still challenging task despite technological advancements. There is still a need for amputees to impose their will upon the prosthesis to drive in an accurate and interactive fashion. This study represents a brief review on control strategies using different sensor modalities for the purpose of phases/events detection and activity recognition. The preliminary work that is associated with middle-level control shows a simple and reliable method for event detection in real-time using a single inertial measurement unit. The outcome shows promising results

Energy Regeneration and Environment Sensing for Robotic Leg Prostheses and Exoskeletons

Robotic leg prostheses and exoskeletons can provide powered locomotor assistance to older adults and/or persons with physical disabilities. However, limitations in automated control and energy-efficient actuation have impeded their transition from research laboratories to real-world environments. With regards to control, the current automated locomotion mode recognition systems being developed rely on mechanical, inertial, and/or neuromuscular sensors, which inherently have limited prediction horizons (i.e., analogous to walking blindfolded). Inspired by the human vision-locomotor control system, here a multi-generation environment sensing and classification system powered by computer vision and deep learning was developed to predict the oncoming walking environments prior to physical interaction, therein allowing for more accurate and robust high-level control decisions. To support this initiative, the “ExoNet” database was developed – the largest and most diverse open-source dataset of wearable camera images of indoor and outdoor real-world walking environments, which were annotated using a novel hierarchical labelling architecture. Over a dozen state-of-the-art deep convolutional neural networks were trained and tested on ExoNet for large-scale image classification and automatic feature engineering. The benchmarked CNN architectures and their environment classification predictions were then quantitatively evaluated and compared using an operational metric called “NetScore”, which balances the classification accuracy with the architectural and computational complexities (i.e., important for onboard real-time inference with mobile computing devices). Of the benchmarked CNN architectures, the EfficientNetB0 network achieved the highest test accuracy; VGG16 the fastest inference time; and MobileNetV2 the best NetScore. These comparative results can inform the optimal architecture design or selection depending on the desired performance of an environment classification system. With regards to energetics, backdriveable actuators with energy regeneration can improve the energy efficiency and extend the battery-powered operating durations by converting some of the otherwise dissipated energy during negative mechanical work into electrical energy. However, the evaluation and control of these regenerative actuators has focused on steady-state level-ground walking. To encompass real-world community mobility more broadly, here an energy regeneration system, featuring mathematical and computational models of human and wearable robotic systems, was developed to simulate energy regeneration and storage during other locomotor activities of daily living, specifically stand-to-sit movements. Parameter identification and inverse dynamic simulations of subject-specific optimized biomechanical models were used to calculate the negative joint mechanical work and power while sitting down (i.e., the mechanical energy theoretically available for electrical energy regeneration). These joint mechanical energetics were then used to simulate a robotic exoskeleton being backdriven and regenerating energy. An empirical characterization of an exoskeleton was carried out using a joint dynamometer system and an electromechanical motor model to calculate the actuator efficiency and to simulate energy regeneration and storage with the exoskeleton parameters. The performance calculations showed that regenerating electrical energy during stand-to-sit movements provide small improvements in energy efficiency and battery-powered operating durations. In summary, this research involved the development and evaluation of environment classification and energy regeneration systems to improve the automated control and energy-efficient actuation of next-generation robotic leg prostheses and exoskeletons for real-world locomotor assistance

Myoelectric forearm prostheses: State of the art from a user-centered perspective

User acceptance of myoelectric forearm prostheses is currently low. Awkward control, lack of feedback, and difficult training are cited as primary reasons. Recently, researchers have focused on exploiting the new possibilities offered by advancements in prosthetic technology. Alternatively, researchers could focus on prosthesis acceptance by developing functional requirements based on activities users are likely to perform. In this article, we describe the process of determining such requirements and then the application of these requirements to evaluating the state of the art in myoelectric forearm prosthesis research. As part of a needs assessment, a workshop was organized involving clinicians (representing end users), academics, and engineers. The resulting needs included an increased number of functions, lower reaction and execution times, and intuitiveness of both control and feedback systems. Reviewing the state of the art of research in the main prosthetic subsystems (electromyographic [EMG] sensing, control, and feedback) showed that modern research prototypes only partly fulfill the requirements. We found that focus should be on validating EMG-sensing results with patients, improving simultaneous control of wrist movements and grasps, deriving optimal parameters for force and position feedback, and taking into account the psychophysical aspects of feedback, such as intensity perception and spatial acuity

Preliminary Investigation of Residual Limb Plantarflexion and Dorsiflexion Muscle Activity During Treadmill Walking for Trans-tibial Amputees

Background: Novel powered prosthetic ankles currently incorporate finite state control, using kinematic and kinetic sensors to differentiate stance and swing phases/sub-phases and control joint impedance and position or torque. For more intuitive control, myoelectric control of the ankle using the remnant residual limb dorsiflexors and plantarflexors, perhaps in concert with kinetic and kinematic sensors, may be possible. Objective: The specific research objective was to assess the feasibility of using myoelectric control of future active or powered prosthetic ankle joints for trans-tibial amputees. Study Design: The project involved human subject trials to determine whether current techniques of myoelectric control of upper extremity prostheses might be readily adapted for lower extremity prosthetic control. Methods: Gait analysis was conducted for three unilateral trans-tibial amputee subjects during ambulation on an instrumented split belt treadmill. Data included ankle plantarflexor and dorsiflexor activity for the residual limb, as well as lower limb kinematics and ground reaction forces and moments of both the sound and prosthetic limbs. Results: These data indicate that: 1) trans-tibial amputees retain some independent ankle plantarflexor and dorsiflexor muscle activity of their residual limb; 2) it is possible to position surface electromyographic electrodes within a trans-tibial socket that maintain contact during ambulation; 3) both the plantarflexors and dorsiflexors of the residual limb are active during gait; 4) plantarflexor and dorsiflexor activity is consistent during multiple gait cycles; and 5) with minimal training, trans-tibial amputees may be able to activate their plantarflexors during push-off. Conclusions: These observations demonstrate the potential for future myoelectric control of active prosthetic ankles. Clinical relevance This study demonstrated the feasibility of applying upper extremity prosthetic myoelectric signal acquisition, processing and control techniques to future myoelectric control of active prosthetic ankles for trans-tibial amputees

Classification of EMG signals to control a prosthetic hand using time-frequesncy representations and Support Vector Machines

Myoelectric signals (MES) are viable control signals for externally-powered prosthetic devices. They may improve both the functionality and the cosmetic appearance of these devices. Conventional controllers, based on the signal\u27s amplitude features in the control strategy, lack a large number of controllable states because signals from independent muscles are required for each degree of freedom (DoF) of the device. Myoelectric pattern recognition systems can overcome this problem by discriminating different residual muscle movements instead of contraction levels of individual muscles. However, the lack of long-term robustness in these systems and the design of counter-intuitive control/command interfaces have resulted in low clinical acceptance levels. As a result, the development of robust, easy to use myoelectric pattern recognition-based control systems is the main challenge in the field of prosthetic control. This dissertation addresses the need to improve the controller\u27s robustness by designing a pattern recognition-based control system that classifies the user\u27s intention to actuate the prosthesis. This system is part of a cost-effective prosthetic hand prototype developed to achieve an acceptable level of functional dexterity using a simple to use interface. A Support Vector Machine (SVM) classifier implemented as a directed acyclic graph (DAG) was created. It used wavelet features from multiple surface EMG channels strategically placed over five forearm muscles. The classifiers were evaluated across seven subjects. They were able to discriminate five wrist motions with an accuracy of 91.5%. Variations of electrode locations were artificially introduced at each recording session as part of the procedure, to obtain data that accounted for the changes in the user\u27s muscle patterns over time. The generalization ability of the SVM was able to capture most of the variability in the data and to maintain an average classification accuracy of 90%. Two principal component analysis (PCA) frameworks were also evaluated to study the relationship between EMG recording sites and the need for feature space reduction. The dimension of the new feature set was reduced with the goal of improving the classification accuracy and reducing the computation time. The analysis indicated that the projection of the wavelet features into a reduced feature space did not significantly improve the accuracy and the computation time. However, decreasing the number of wavelet decomposition levels did lower the computational load without compromising the average signal classification accuracy. Based on the results of this work, a myoelectric pattern recognition-based control system that uses an SVM classifier applied to time-frequency features may be used to discriminate muscle contraction patterns for prosthetic applications

EMG-driven control in lower limb prostheses: a topic-based systematic review

Background The inability of users to directly and intuitively control their state-of-the-art commercial prosthesis contributes to a low device acceptance rate. Since Electromyography (EMG)-based control has the potential to address those inabilities, research has flourished on investigating its incorporation in microprocessor-controlled lower limb prostheses (MLLPs). However, despite the proposed benefits of doing so, there is no clear explanation regarding the absence of a commercial product, in contrast to their upper limb counterparts. Objective and methodologies This manuscript aims to provide a comparative overview of EMG-driven control methods for MLLPs, to identify their prospects and limitations, and to formulate suggestions on future research and development. This is done by systematically reviewing academical studies on EMG MLLPs. In particular, this review is structured by considering four major topics: (1) type of neuro-control, which discusses methods that allow the nervous system to control prosthetic devices through the muscles; (2) type of EMG-driven controllers, which defines the different classes of EMG controllers proposed in the literature; (3) type of neural input and processing, which describes how EMG-driven controllers are implemented; (4) type of performance assessment, which reports the performance of the current state of the art controllers. Results and conclusions The obtained results show that the lack of quantitative and standardized measures hinders the possibility to analytically compare the performances of different EMG-driven controllers. In relation to this issue, the real efficacy of EMG-driven controllers for MLLPs have yet to be validated. Nevertheless, in anticipation of the development of a standardized approach for validating EMG MLLPs, the literature suggests that combining multiple neuro-controller types has the potential to develop a more seamless and reliable EMG-driven control. This solution has the promise to retain the high performance of the currently employed non-EMG-driven controllers for rhythmic activities such as walking, whilst improving the performance of volitional activities such as task switching or non-repetitive movements. Although EMG-driven controllers suffer from many drawbacks, such as high sensitivity to noise, recent progress in invasive neural interfaces for prosthetic control (bionics) will allow to build a more reliable connection between the user and the MLLPs. Therefore, advancements in powered MLLPs with integrated EMG-driven control have the potential to strongly reduce the effects of psychosomatic conditions and musculoskeletal degenerative pathologies that are currently affecting lower limb amputees

IMU-based Deep Neural Networks for Locomotor Intention Prediction

This paper focuses on the design and comparison of different deep neural networks for the real-time prediction of locomotor intentions by using data from inertial measurement units. The deep neural network architectures are convolutional neural networks, recurrent neural networks, and convolutional recurrent neural networks. The input to the architectures are features in the time domain, which have been derived either from one inertial measurement unit placed on the upper right leg of ten healthy subjects, or two inertial measurement units placed on both the upper and lower right leg of ten healthy subjects. The study shows that a WaveNet, i.e., a full convolutional neural network, achieves a peak F1-score of 87.17% in the case of one IMU, and a peak of 97.88% in the case of two IMUs, with a 5-fold cross-validation

CONTROL OF A POWERED ANKLE-FOOT PROSTHESIS: FROM PERCEPTION TO IMPEDANCE MODULATION

Active ankle prostheses controllers are demonstrating gaining smart features to improve the safety and comfort offor users. The perception of user intention to modulate the ankle dynamics is a well-known example of such feature. But not much work focused on the perception of the environment, nor how the environment should be included in the mechanical design and control of the prosthesisprostheses. The proposed work aims to improve the feasibility of integrate the environment perception integration intoto the prostheses controllersler, and to define the desired ankle dynamics, as mechanical impedance, duringof the human walk on different environmental settings. As a preliminary work on environment perception, a vision system was developed that can estimate the ground profileslope and height. The desired prosthesis impedance dynamics is was defined as the dynamics mechanical impedance of a healthy ankle; , therefore,which required the a system identification methodof the human ankle was developed. Simulations showed the inertia parameters of a rigid bodymockup foot can be estimated. Further experiments will show the accuracy of environment perception and of the impedance estimation

FAST USER ACTIVITY PHASE RECOGNITION FOR THE SAFETY OF TRANSFEMORAL PROSTHESIS CONTROL

In the process of creating powered transfemoral prostheses, one of the most important tasks is the provision of the user safety while walking. Experience shows that security depends not only on the mechanical strength of such devices, but also on the quality of their control systems, which, among other things, must ensure that latency and error rates of recognition are acceptable for each of the possible changes in gait. Incorrect or late recognition of the activity mode at best can lead to suboptimal assistance from the auxiliary device, and at worst - to loss of stability of the user with a subsequent fall. Loss of stability can also occur due to exceeding the critical time or critical errors of the activity phase recognition and the associated incorrect commands generated by the control system. In this paper, a method for quickly recognizing the phase of the user's activity based on the properties of Hu’s moment invariants is substantiated. Its use in the intelligent control systems will minimize the critical errors that contribute to the loss of the user's equilibrium with the powered transfemoral prosthesis