Arduino-based myoelectric control: Towards longitudinal study of prosthesis use

Understanding how upper-limb prostheses are used in daily life helps to improve the design and robustness of prosthesis control algorithms and prosthetic components. However, only a very small fraction of published research includes prosthesis use in community settings. The cost, limited battery life, and poor generalisation may be the main reasons limiting the implementation of home-based applications. In this work, we introduce the design of a cost-effective Arduino-based myoelectric control system with wearable electromyogram (EMG) sensors. The design considerations focused on home studies, so the robustness, user-friendly control adjustments, and user supports were the main concerns. Three control algorithms, namely, direct control, abstract control, and linear discriminant analysis (LDA) classification, were implemented in the system. In this paper, we will share our design principles and report the robustness of the system in continuous operation in the laboratory. In addition, we will show a first real-time implementation of the abstract decoder for prosthesis control with an able-bodied participant

Shoulder muscle activation pattern recognition based on sEMG and machine learning algorithms

BACKGROUND AND OBJECTIVE: Surface electromyography (sEMG) has been used for robotic rehabilitation engineering for volitional control of hand prostheses or elbow exoskeleton, however, using sEMG for volitional control of an upper limb exoskeleton has not been perfectly developed. The long-term goal of our study is to process shoulder muscle bio-electrical signals for rehabilitative robotic assistive device motion control. The purposes of this study included: 1) to test the feasibility of machine learning algorithms in shoulder motion pattern recognition using sEMG signals from shoulder and upper limb muscles, 2) to investigate the influence of motion speed, individual variability, EMG recording device, and the amount of EMG datasets on the shoulder motion pattern recognition accuracy. METHODS: A novel convolutional neural network (CNN) structure was constructed to process EMG signals from 12 muscles for the pattern recognition of upper arm motions including resting, drinking, backward-forward motion, and abduction motion. The accuracy of the CNN models for pattern recognition under different motion speeds, among individuals, and by EMG recording devices was statistically analyzed using ANOVA, GLM Univariate analysis, and Chi-square tests. The influence of EMG dataset number used for CNN model training on recognition accuracy was studied by gradually increasing dataset number until the highest accuracy was obtained. RESULTS: Results showed that the accuracy of the normal speed CNN model in motion pattern recognition was 97.57% for normal speed motions and 97.07% for fast speed motions. The accuracy of the cross-subjects CNN model in motion pattern recognition was 79.64%. The accuracy of the cross-device CNN model in motion pattern recognition was 88.93% for normal speed motion and 80.87% for mixed speed. There was a statistical difference in pattern recognition accuracy between different CNN models. CONCLUSION: The EMG signals of shoulder and upper arm muscles from the upper limb motions can be processed using CNN algorithms to recognize the identical motions of the upper limb including drinking, forward/backward, abduction, and resting. A simple CNN model trained by EMG datasets of a designated motion speed accurately detected the motion patterns of the same motion speed, yielding the highest accuracy compared with other mixed CNN models for various speeds of motion pattern recognition. Increase of the number of EMG datasets for CNN model training improved the pattern recognition accuracy

Donning/Doffing and Arm Positioning Influence in Upper Limb Adaptive Prostheses Control

[EN] New upper limb prostheses controllers are continuously being proposed in the literature. However, most of the prostheses commonly used in the real world are based on very old basic controllers. One reason to explain this reluctance to change is the lack of robustness. Traditional controllers have been validated by many users and years, so the introduction of a new controller paradigm requires a lot of strong evidence of a robust behavior. In this work, we approach the robustness against donning/doffing and arm position for recently proposed linear filter adaptive controllers based on myoelectric signals. The adaptive approach allows to introduce some feedback in a natural way in real time in the human-machine collaboration, so it is not so sensitive to input signals changes due to donning/doffing and arm movements. The average completion rate and path efficiency obtained for eight able-bodied subjects donning/doffing five times in four days is 95.83% and 84.19%, respectively, and for four participants using different arm positions is 93.84% and 88.77%, with no statistically significant difference in the results obtained for the different conditions. All these characteristics make the adaptive linear regression a potential candidate for future real world prostheses controllers.This work is partially supported by Ministerio de Educacion, Cultura y Deporte (Spain) under grant FPU15/02870. The authors would like to thank Lucas Parra for the Myo device and Janne M. Hahne for discussions about the subject of the paper.Igual, C.; Camacho-García, A.; Bernabeu Soler, EJ.; Igual García, J. (2020). Donning/Doffing and Arm Positioning Influence in Upper Limb Adaptive Prostheses Control. Applied Sciences. 10(8):1-19. https://doi.org/10.3390/app10082892S119108Esquenazi, A. (2004). Amputation rehabilitation and prosthetic restoration. From surgery to community reintegration. Disability and Rehabilitation, 26(14-15), 831-836. doi:10.1080/09638280410001708850Ziegler-Graham, K., MacKenzie, E. J., Ephraim, P. L., Travison, T. G., & Brookmeyer, R. (2008). Estimating the Prevalence of Limb Loss in the United States: 2005 to 2050. Archives of Physical Medicine and Rehabilitation, 89(3), 422-429. doi:10.1016/j.apmr.2007.11.005Igual, C., Pardo, L. A., Hahne, J., & Igual, J. M. (2019). Myoelectric Control for Upper Limb Prostheses. Electronics, 8(11), 1244. doi:10.3390/electronics8111244Biddiss, E., & Chau, T. (2007). Upper-Limb Prosthetics. American Journal of Physical Medicine & Rehabilitation, 86(12), 977-987. doi:10.1097/phm.0b013e3181587f6cBiddiss, E. A., & Chau, T. T. (2007). Upper limb prosthesis use and abandonment. Prosthetics & Orthotics International, 31(3), 236-257. doi:10.1080/03093640600994581Davidson, J. (2002). A survey of the satisfaction of upper limb amputees with their prostheses, their lifestyles, and their abilities. Journal of Hand Therapy, 15(1), 62-70. doi:10.1053/hanthe.2002.v15.01562Datta, D., Selvarajah, K., & Davey, N. (2004). Functional outcome of patients with proximal upper limb deficiency–acquired and congenital. Clinical Rehabilitation, 18(2), 172-177. doi:10.1191/0269215504cr716oaVujaklija, I., Farina, D., & Aszmann, O. (2016). New developments in prosthetic arm systems. Orthopedic Research and Reviews, Volume 8, 31-39. doi:10.2147/orr.s71468Scheme, E. J., Englehart, K. B., & Hudgins, B. S. (2011). Selective Classification for Improved Robustness of Myoelectric Control Under Nonideal Conditions. IEEE Transactions on Biomedical Engineering, 58(6), 1698-1705. doi:10.1109/tbme.2011.2113182Englehart, K., & Hudgins, B. (2003). A robust, real-time control scheme for multifunction myoelectric control. IEEE Transactions on Biomedical Engineering, 50(7), 848-854. doi:10.1109/tbme.2003.813539Parker, P., Englehart, K., & Hudgins, B. (2006). Myoelectric signal processing for control of powered limb prostheses. Journal of Electromyography and Kinesiology, 16(6), 541-548. doi:10.1016/j.jelekin.2006.08.006Fougner, A., Stavdahl, Ø., Kyberd, P. J., Losier, Y. G., & Parker, P. A. (2012). Control of Upper Limb Prostheses: Terminology and Proportional Myoelectric Control—A Review. IEEE Transactions on Neural Systems and Rehabilitation Engineering, 20(5), 663-677. doi:10.1109/tnsre.2012.2196711Resnik, L., Huang, H. (Helen), Winslow, A., Crouch, D. L., Zhang, F., & Wolk, N. (2018). Evaluation of EMG pattern recognition for upper limb prosthesis control: a case study in comparison with direct myoelectric control. Journal of NeuroEngineering and Rehabilitation, 15(1). doi:10.1186/s12984-018-0361-3Sartori, M., Durandau, G., Došen, S., & Farina, D. (2018). Robust simultaneous myoelectric control of multiple degrees of freedom in wrist-hand prostheses by real-time neuromusculoskeletal modeling. Journal of Neural Engineering, 15(6), 066026. doi:10.1088/1741-2552/aae26bVelliste, M., Perel, S., Spalding, M. C., Whitford, A. S., & Schwartz, A. B. (2008). Cortical control of a prosthetic arm for self-feeding. Nature, 453(7198), 1098-1101. doi:10.1038/nature06996Amsuess, S., Vujaklija, I., Goebel, P., Roche, A. D., Graimann, B., Aszmann, O. C., & Farina, D. (2016). Context-Dependent Upper Limb Prosthesis Control for Natural and Robust Use. IEEE Transactions on Neural Systems and Rehabilitation Engineering, 24(7), 744-753. doi:10.1109/tnsre.2015.2454240Kuiken, T. A., Miller, L. A., Turner, K., & Hargrove, L. J. (2016). A Comparison of Pattern Recognition Control and Direct Control of a Multiple Degree-of-Freedom Transradial Prosthesis. IEEE Journal of Translational Engineering in Health and Medicine, 4, 1-8. doi:10.1109/jtehm.2016.2616123Phinyomark, A., N. Khushaba, R., & Scheme, E. (2018). Feature Extraction and Selection for Myoelectric Control Based on Wearable EMG Sensors. Sensors, 18(5), 1615. doi:10.3390/s18051615Asghari Oskoei, M., & Hu, H. (2007). Myoelectric control systems—A survey. Biomedical Signal Processing and Control, 2(4), 275-294. doi:10.1016/j.bspc.2007.07.009Spanias, J. A., Perreault, E. J., & Hargrove, L. J. (2016). Detection of and Compensation for EMG Disturbances for Powered Lower Limb Prosthesis Control. IEEE Transactions on Neural Systems and Rehabilitation Engineering, 24(2), 226-234. doi:10.1109/tnsre.2015.2413393Castellini, C., & van der Smagt, P. (2008). Surface EMG in advanced hand prosthetics. Biological Cybernetics, 100(1), 35-47. doi:10.1007/s00422-008-0278-1Ameri, A., Akhaee, M. A., Scheme, E., & Englehart, K. (2019). Regression convolutional neural network for improved simultaneous EMG control. Journal of Neural Engineering, 16(3), 036015. doi:10.1088/1741-2552/ab0e2eHahne, J. M., Biebmann, F., Jiang, N., Rehbaum, H., Farina, D., Meinecke, F. C., … Parra, L. C. (2014). Linear and Nonlinear Regression Techniques for Simultaneous and Proportional Myoelectric Control. IEEE Transactions on Neural Systems and Rehabilitation Engineering, 22(2), 269-279. doi:10.1109/tnsre.2014.2305520Ameri, A., Scheme, E. J., Kamavuako, E. N., Englehart, K. B., & Parker, P. A. (2014). Real-Time, Simultaneous Myoelectric Control Using Force and Position-Based Training Paradigms. IEEE Transactions on Biomedical Engineering, 61(2), 279-287. doi:10.1109/tbme.2013.2281595Hochberg, L. R., Serruya, M. D., Friehs, G. M., Mukand, J. A., Saleh, M., Caplan, A. H., … Donoghue, J. P. (2006). Neuronal ensemble control of prosthetic devices by a human with tetraplegia. Nature, 442(7099), 164-171. doi:10.1038/nature04970Interface Prostheses With Classifier-Feedback-Based User Training. (2017). IEEE Transactions on Biomedical Engineering, 64(11), 2575-2583. doi:10.1109/tbme.2016.2641584Thomas, N., Ung, G., McGarvey, C., & Brown, J. D. (2019). Comparison of vibrotactile and joint-torque feedback in a myoelectric upper-limb prosthesis. Journal of NeuroEngineering and Rehabilitation, 16(1). doi:10.1186/s12984-019-0545-5Guémann, M., Bouvier, S., Halgand, C., Borrini, L., Paclet, F., Lapeyre, E., … de Rugy, A. (2018). Sensory and motor parameter estimation for elbow myoelectric control with vibrotactile feedback. Annals of Physical and Rehabilitation Medicine, 61, e467. doi:10.1016/j.rehab.2018.05.1090Markovic, M., Schweisfurth, M. A., Engels, L. F., Farina, D., & Dosen, S. (2018). Myocontrol is closed-loop control: incidental feedback is sufficient for scaling the prosthesis force in routine grasping. Journal of NeuroEngineering and Rehabilitation, 15(1). doi:10.1186/s12984-018-0422-7Pasquina, P. F., Perry, B. N., Miller, M. E., Ling, G. S. F., & Tsao, J. W. (2015). Recent advances in bioelectric prostheses. Neurology: Clinical Practice, 5(2), 164-170. doi:10.1212/cpj.0000000000000132NING, J., & Dario, F. (2014). Myoelectric control of upper limb prosthesis: current status, challenges and recent advances. Frontiers in Neuroengineering, 7. doi:10.3389/conf.fneng.2014.11.00004Lendaro, E., Mastinu, E., Håkansson, B., & Ortiz-Catalan, M. (2017). Real-time Classification of Non-Weight Bearing Lower-Limb Movements Using EMG to Facilitate Phantom Motor Execution: Engineering and Case Study Application on Phantom Limb Pain. Frontiers in Neurology, 8. doi:10.3389/fneur.2017.00470Mastinu, E., Ortiz-Catalan, M., & Hakansson, B. (2015). Analog front-ends comparison in the way of a portable, low-power and low-cost EMG controller based on pattern recognition. 2015 37th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC). doi:10.1109/embc.2015.7318805BECK, T. W., HOUSH, T. J., CRAMER, J. T., MALEK, M. H., MIELKE, M., HENDRIX, R., & WEIR, J. P. (2008). Electrode Shift and Normalization Reduce the Innervation Zone’s Influence on EMG. Medicine & Science in Sports & Exercise, 40(7), 1314-1322. doi:10.1249/mss.0b013e31816c4822Pasquina, P. F., Evangelista, M., Carvalho, A. J., Lockhart, J., Griffin, S., Nanos, G., … Hankin, D. (2015). First-in-man demonstration of a fully implanted myoelectric sensors system to control an advanced electromechanical prosthetic hand. Journal of Neuroscience Methods, 244, 85-93. doi:10.1016/j.jneumeth.2014.07.016Fougner, A., Scheme, E., Chan, A. D. C., Englehart, K., & Stavdahl, Ø. (2011). Resolving the Limb Position Effect in Myoelectric Pattern Recognition. IEEE Transactions on Neural Systems and Rehabilitation Engineering, 19(6), 644-651. doi:10.1109/tnsre.2011.2163529Hwang, H.-J., Hahne, J. M., & Müller, K.-R. (2017). Real-time robustness evaluation of regression based myoelectric control against arm position change and donning/doffing. PLOS ONE, 12(11), e0186318. doi:10.1371/journal.pone.0186318Young, A. J., Hargrove, L. J., & Kuiken, T. A. (2011). The Effects of Electrode Size and Orientation on the Sensitivity of Myoelectric Pattern Recognition Systems to Electrode Shift. IEEE Transactions on Biomedical Engineering, 58(9), 2537-2544. doi:10.1109/tbme.2011.2159216Prahm, C., Schulz, A., Paaben, B., Schoisswohl, J., Kaniusas, E., Dorffner, G., … Aszmann, O. (2019). Counteracting Electrode Shifts in Upper-Limb Prosthesis Control via Transfer Learning. IEEE Transactions on Neural Systems and Rehabilitation Engineering, 27(5), 956-962. doi:10.1109/tnsre.2019.2907200Cipriani, C., Sassu, R., Controzzi, M., & Carrozza, M. C. (2011). Influence of the weight actions of the hand prosthesis on the performance of pattern recognition based myoelectric control: Preliminary study. 2011 Annual International Conference of the IEEE Engineering in Medicine and Biology Society. doi:10.1109/iembs.2011.6090468Amsuss, S., Paredes, L. P., Rudigkeit, N., Graimann, B., Herrmann, M. J., & Farina, D. (2013). Long term stability of surface EMG pattern classification for prosthetic control. 2013 35th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC). doi:10.1109/embc.2013.6610327Scheme, E., & Englehart, K. (2011). Electromyogram pattern recognition for control of powered upper-limb prostheses: State of the art and challenges for clinical use. The Journal of Rehabilitation Research and Development, 48(6), 643. doi:10.1682/jrrd.2010.09.0177Scheme, E., Fougner, A., Stavdahl, Ø., Chan, A. D. C., & Englehart, K. (2010). Examining the adverse effects of limb position on pattern recognition based myoelectric control. 2010 Annual International Conference of the IEEE Engineering in Medicine and Biology. doi:10.1109/iembs.2010.5627638Dohnalek, P., Gajdos, P., & Peterek, T. (2013). Human activity recognition on raw sensor data via sparse approximation. 2013 36th International Conference on Telecommunications and Signal Processing (TSP). doi:10.1109/tsp.2013.6614027Marasco, P. D., Hebert, J. S., Sensinger, J. W., Shell, C. E., Schofield, J. S., Thumser, Z. C., … Orzell, B. M. (2018). Illusory movement perception improves motor control for prosthetic hands. Science Translational Medicine, 10(432). doi:10.1126/scitranslmed.aao6990Mastinu, E., Doguet, P., Botquin, Y., Hakansson, B., & Ortiz-Catalan, M. (2017). Embedded System for Prosthetic Control Using Implanted Neuromuscular Interfaces Accessed Via an Osseointegrated Implant. IEEE Transactions on Biomedical Circuits and Systems, 11(4), 867-877. doi:10.1109/tbcas.2017.2694710Igual, C., Igual, J., Hahne, J. M., & Parra, L. C. (2019). Adaptive Auto-Regressive Proportional Myoelectric Control. IEEE Transactions on Neural Systems and Rehabilitation Engineering, 27(2), 314-322. doi:10.1109/tnsre.2019.2894464Huang, Y., Englehart, K. B., Hudgins, B., & Chan, A. D. C. (2005). A Gaussian Mixture Model Based Classification Scheme for Myoelectric Control of Powered Upper Limb Prostheses. IEEE Transactions on Biomedical Engineering, 52(11), 1801-1811. doi:10.1109/tbme.2005.856295Hahne, J. M., Dahne, S., Hwang, H.-J., Muller, K.-R., & Parra, L. C. (2015). Concurrent Adaptation of Human and Machine Improves Simultaneous and Proportional Myoelectric Control. IEEE Transactions on Neural Systems and Rehabilitation Engineering, 23(4), 618-627. doi:10.1109/tnsre.2015.240113

A temporal-to-spatial neural network for classification of hand movements from electromyography data

Deep convolutional neural networks (CNNs) are appealing for the purpose of classification of hand movements from surface electromyography (sEMG) data because they have the ability to perform automated person-specific feature extraction from raw data. In this paper, we make the novel contribution of proposing and evaluating a design for the early processing layers in the deep CNN for multichannel sEMG. Specifically, we propose a novel temporal-to-spatial (TtS) CNN architecture, where the first layer performs convolution separately on each sEMG channel to extract temporal features. This is motivated by the idea that sEMG signals in each channel are mediated by one or a small subset of muscles, whose temporal activation patterns are associated with the signature features of a gesture. The temporal layer captures these signature features for each channel separately, which are then spatially mixed in successive layers to recognise a specific gesture. A practical advantage is that this approach also makes the CNN simple to design for different sample rates. We use NinaPro database 1 (27 subjects and 52 movements + rest), sampled at 100 Hz, and database 2 (40 subjects and 40 movements + rest), sampled at 2 kHz, to evaluate our proposed CNN design. We benchmark against a feature-based support vector machine (SVM) classifier, two CNNs from the literature, and an additional standard design of CNN. We find that our novel TtS CNN design achieves 66.6% per-class accuracy on database 1, and 67.8% on database 2, and that the TtS CNN outperforms all other compared classifiers using a statistical hypothesis test at the 2% significance level