Algorithms for Neural Prosthetic Applications

abstract: In the last 15 years, there has been a significant increase in the number of motor neural prostheses used for restoring limb function lost due to neurological disorders or accidents. The aim of this technology is to enable patients to control a motor prosthesis using their residual neural pathways (central or peripheral). Recent studies in non-human primates and humans have shown the possibility of controlling a prosthesis for accomplishing varied tasks such as self-feeding, typing, reaching, grasping, and performing fine dexterous movements. A neural decoding system comprises mainly of three components: (i) sensors to record neural signals, (ii) an algorithm to map neural recordings to upper limb kinematics and (iii) a prosthetic arm actuated by control signals generated by the algorithm. Machine learning algorithms that map input neural activity to the output kinematics (like finger trajectory) form the core of the neural decoding system. The choice of the algorithm is thus, mainly imposed by the neural signal of interest and the output parameter being decoded. The various parts of a neural decoding system are neural data, feature extraction, feature selection, and machine learning algorithm. There have been significant advances in the field of neural prosthetic applications. But there are challenges for translating a neural prosthesis from a laboratory setting to a clinical environment. To achieve a fully functional prosthetic device with maximum user compliance and acceptance, these factors need to be addressed and taken into consideration. Three challenges in developing robust neural decoding systems were addressed by exploring neural variability in the peripheral nervous system for dexterous finger movements, feature selection methods based on clinically relevant metrics and a novel method for decoding dexterous finger movements based on ensemble methods.Dissertation/ThesisDoctoral Dissertation Bioengineering 201

Analysis of Small Muscle Movement Effects on EEG Signals

In this thesis, the artefactual effects of the small muscle movements were investigated. Upper frequency bands (30 Hz) of the EEG signal were extracted in order to investigate the artefactual effects of the small muscle movements. When the contamination level is high, the detection of the small muscle artifact can be made with the 92.2% accuracy. If these artifacts are really small such as a single finger movement, the detection accuracy decreases to 64%. But, the detection accuracy increases to 72% after removing the eye blink artifacts. The results of the classification support our hypothesis about the artefactual effects of the small muscle movements

Functional Near Infrared Detection of Real and Imagined Finger Taps Using Support Vector Machine, Linear Discriminant Analysis, and Decision Tree Classification Methods

This study investigates the thesis that given cerebral response samples of an individual\u27s left, right, both, and imagined finger tapping, continuous wave (CW) functional Near Infrared (fNIR), unregistered with fMRI, can differentiate between any two of the four categories. Fifty subjects were outfitted with a single source/detector attached to a single, square pad, affixed to their heads using devices such as elastic bands and caps for light shielding. Slides depicting arrows pointing left, right, both directions, or made of dashed lines were presented to each subject, with a slide of text interspersed between each. Subjects tapped with their left finger, right finger, both left and right finger, or imagined tapping, depending on the type of arrow. Text was presented in between each tapping slide and was read with no tapping. Each slide was presented for twenty seconds and each type of tapping occurred three times in an eight minute, 20 second period. Classification was performed using Support Vector Machine (SVM), Linear Discriminant Analysis (LDA), and decision tree algorithms. Results indicated that left finger tapping can be distinguished from right, both, and imagined right finger-tapping with error rates ranging from 24.92% to 29.51% (SVM), 40.05% to 42.69% (LDA), and 23.34% to 28.85% (decision tree). The decision tree algorithm produced results, on an individual trial basis, with greater than 95% confidence that the results were not due to chance. These results were obtained with no screening out due to individual characteristics such as hair thickness. The generalizations included the use of a large sample of subjects for which the selection criteria only included statutory minimum and maximum ages. This study also produced validation of a method of mitigating hair effect. Raising the sensor was shown to still produce valid results that could not be attributed to chance at a confidence level of 95%. The results are directly applicable to brain-computer interfaces in a number of areas. These relate to validating the ability to classify data collected by a device with a single source/detector, from non-prescreened individuals, with real-time algorithms in a normal environment

Sensor-based artificial intelligence to support people with cognitive and physical disorders

A substantial portion of the world's population deals with disability. Many disabled people do not have equal access to healthcare, education, and employment opportunities, do not receive specific disability-related services, and experience exclusion from everyday life activities. One way to face these issues is through the use of healthcare technologies. Unfortunately, there is a large amount of diverse and heterogeneous disabilities, which require ad-hoc and personalized solutions. Moreover, the design and implementation of effective and efficient technologies is a complex and expensive process involving challenging issues, including usability and acceptability. The work presented in this thesis aims to improve the current state of technologies available to support people with disorders affecting the mind or the motor system by proposing the use of sensors coupled with signal processing methods and artificial intelligence algorithms. The first part of the thesis focused on mental state monitoring. We investigated the application of a low-cost portable electroencephalography sensor and supervised learning methods to evaluate a person's attention. Indeed, the analysis of attention has several purposes, including the diagnosis and rehabilitation of children with attention-deficit/hyperactivity disorder. A novel dataset was collected from volunteers during an image annotation task, and used for the experimental evaluation using different machine learning techniques. Then, in the second part of the thesis, we focused on addressing limitations related to motor disability. We introduced the use of graph neural networks to process high-density electromyography data for upper limbs amputees’ movement/grasping intention recognition for enabling the use of robotic prostheses. High-density electromyography sensors can simultaneously acquire electromyography signals from different parts of the muscle, providing a large amount of spatio-temporal information that needs to be properly exploited to improve recognition accuracy. The investigation of the approach was conducted using a recent real-world dataset consisting of electromyography signals collected from 20 volunteers while performing 65 different gestures. In the final part of the thesis, we developed a prototype of a versatile interactive system that can be useful to people with different types of disabilities. The system can maintain a food diary for frail people with nutrition problems, such as people with neurocognitive diseases or frail elderly people, which may have difficulties due to forgetfulness or physical issues. The novel architecture automatically recognizes the preparation of food at home, in a privacy-preserving and unobtrusive way, exploiting air quality data acquired from a commercial sensor, statistical features extraction, and a deep neural network. A robotic system prototype is used to simplify the interaction with the inhabitant. For this work, a large dataset of annotated sensor data acquired over a period of 8 months from different individuals in different homes was collected. Overall, the results achieved in the thesis are promising, and pave the way for several real-world implementations and future research directions

The Relationship between Anthropometric Variables and Features of Electromyography Signal for Human-Computer Interface

http://doi.org/10.4018/978-1-4666-6090-8 ISBN 13 : 9781466660908 EISBN13: 9781466660915International audienceMuscle-computer interfaces (MCIs) based on surface electromyography (EMG) pattern recognition have been developed based on two consecutive components: feature extraction and classification algorithms. Many features and classifiers are proposed and evaluated, which yield the high classification accuracy and the high number of discriminated motions under a single-session experimental condition. However, there are many limitations to use MCIs in the real-world contexts, such as the robustness over time, noise, or low-level EMG activities. Although the selection of the suitable robust features can solve such problems, EMG pattern recognition has to design and train for a particular individual user to reach high accuracy. Due to different body compositions across users, a feasibility to use anthropometric variables to calibrate EMG recognition system automatically/semi-automatically is proposed. This chapter presents the relationships between robust features extracted from actions associated with surface EMG signals and twelve related anthropometric variables. The strong and significant associations presented in this chapter could benefit a further design of the MCIs based on EMG pattern recognition

Evaluation of surface EMG-based recognition algorithms for decoding hand movements

Myoelectric pattern recognition (MPR) to decode limb movements is an important advancement regarding the control of powered prostheses. However, this technology is not yet in wide clinical use. Improvements in MPR could potentially increase the functionality of powered prostheses. To this purpose, offline accuracy and processing time were measured over 44 features using six classifiers with the aim of determining new configurations of features and classifiers to improve the accuracy and response time of prosthetics control. An efficient feature set (FS: waveform length, correlation coefficient, Hjorth Parameters) was found to improve the motion recognition accuracy. Using the proposed FS significantly increased the performance of linear discriminant analysis, K-nearest neighbor, maximum likelihood estimation (MLE), and support vector machine by 5.5%, 5.7%, 6.3%, and 6.2%, respectively, when compared with the Hudgins\u27 set. Using the FS with MLE provided the largest improvement in offline accuracy over the Hudgins feature set, with minimal effect on the processing time. Among the 44 features tested, logarithmic root mean square and normalized logarithmic energy yielded the highest recognition rates (above 95%). We anticipate that this work will contribute to the development of more accurate surface EMG-based motor decoding systems for the control prosthetic hands

The Analysis of Surface EMG Signals with the Wavelet-Based Correlation Dimension Method

Many attempts have been made to effectively improve a prosthetic system controlled by the classification of surface electromyographic (SEMG) signals. Recently, the development of methodologies to extract the effective features still remains a primary challenge. Previous studies have demonstrated that the SEMG signals have nonlinear characteristics. In this study, by combining the nonlinear time series analysis and the time-frequency domain methods, we proposed the wavelet-based correlation dimension method to extract the effective features of SEMG signals. The SEMG signals were firstly analyzed by the wavelet transform and the correlation dimension was calculated to obtain the features of the SEMG signals. Then, these features were used as the input vectors of a Gustafson-Kessel clustering classifier to discriminate four types of forearm movements. Our results showed that there are four separate clusters corresponding to different forearm movements at the third resolution level and the resulting classification accuracy was 100%, when two channels of SEMG signals were used. This indicates that the proposed approach can provide important insight into the nonlinear characteristics and the time-frequency domain features of SEMG signals and is suitable for classifying different types of forearm movements. By comparing with other existing methods, the proposed method exhibited more robustness and higher classification accuracy

Comparison of CNN-Learned vs. Handcrafted Features for Detection of Parkinson's Disease Dysgraphia in a Multilingual Dataset

Parkinson's disease dysgraphia (PDYS), one of the earliest signs of Parkinson's disease (PD), has been researched as a promising biomarker of PD and as the target of a noninvasive and inexpensive approach to monitoring the progress of the disease. However, although several approaches to supportive PDYS diagnosis have been proposed (mainly based on handcrafted features (HF) extracted from online handwriting or the utilization of deep neural networks), it remains unclear which approach provides the highest discrimination power and how these approaches can be transferred between different datasets and languages. This study aims to compare classification performance based on two types of features: features automatically extracted by a pretrained convolutional neural network (CNN) and HF designed by human experts. Both approaches are evaluated on a multilingual dataset collected from 143 PD patients and 151 healthy controls in the Czech Republic, United States, Colombia, and Hungary. The subjects performed the spiral drawing task (SDT; a language-independent task) and the sentence writing task (SWT; a language-dependent task). Models based on logistic regression and gradient boosting were trained in several scenarios, specifically single language (SL), leave one language out (LOLO), and all languages combined (ALC). We found that the HF slightly outperformed the CNN-extracted features in all considered evaluation scenarios for the SWT. In detail, the following balanced accuracy (BACC) scores were achieved: SL—0.65 (HF), 0.58 (CNN); LOLO—0.65 (HF), 0.57 (CNN); and ALC—0.69 (HF), 0.66 (CNN). However, in the case of the SDT, features extracted by a CNN provided competitive results: SL—0.66 (HF), 0.62 (CNN); LOLO—0.56 (HF), 0.54 (CNN); and ALC—0.60 (HF), 0.60 (CNN). In summary, regarding the SWT, the HF outperformed the CNN-extracted features over 6% (mean BACC of 0.66 for HF, and 0.60 for CNN). In the case of the SDT, both feature sets provided almost identical classification performance (mean BACC of 0.60 for HF, and 0.58 for CNN). Copyright © 2022 Galaz, Drotar, Mekyska, Gazda, Mucha, Zvoncak, Smekal, Faundez-Zanuy, Castrillon, Orozco-Arroyave, Rapcsak, Kincses, Brabenec and Rektorova

Development and optimization of a low-cost myoelectric upper limb prosthesis

Tese de Mestrado Integrado, Engenharia Biomédica e Biofísica (Engenharia Clínica e Instrumentação Médica), 2022, Universidade de Lisboa, Faculdade de CiênciasIn recent years, the increase in the number of accidents, chronic diseases, such as diabetes, and the impoverishment of certain developing countries have contributed to a significant increase in prostheses users. The loss of a particular limb entails numerous changes in the daily life of each user, which are amplified when the user loses their hand. Therefore, replacing the hand is an urgent necessity. Developing upper limb prostheses will allow the re-establishment of the physical and motor functions of the upper limb as well as reduction of the rates of depression. Therefore, the prosthetic industry has been reinventing itself and evolving. It is already possible to control a prosthesis through the user's myoelectric signals, control known as pattern recognition control. In addition, additive manufacturing technologies such as 3D printing have gained strength in prosthetics. The use of this type of technology allows the product to reach the user much faster and reduces the weight of the devices, making them lighter. Despite these advances, the rejection rate of this type of device is still high since most prostheses available on the market are slow, expensive and heavy. Because of that, academia and institutions have been investigating ways to overcome these limitations. Nevertheless, the dependence on the number of acquisition channels is still limiting since most users do not have a large available forearm surface area to acquire the user’s myoelectric signals. This work intends to solve some of these problems and answer the questions imposed by the industry and researchers. The main objective is to test if developing a subject independent, fast and simple microcontroller is possible. Subsequently, we recorded data from forty volunteers through the BIOPAC acquisition system. After that, the signals were filtered through two different processes. The first was digital filtering and the application of wavelet threshold noise reduction. Later, the signal was divided into smaller windows (100 and 250 milliseconds) and thirteen features were extracted in the temporal domain. During all these steps, the MatLab® software was used. After extraction, three feature selection methods were used to optimize the classification process, where machine learning algorithms are implemented. The classification was divided into different parts. First, the classifier had to distinguish whether the volunteer was making some movement or was at rest. In the case of detected movement, the classifier would have to, on a second level, try to understand if they were moving only one finger or performing a movement that involved the flexion of more than one finger (grip). If the volunteer was performing a grip on the third level, the classifier would have to identify whether the volunteer was performing a spherical or triad grip. Finally, to understand the influence of the database on the classification, two methods were used: cross-validation and split validation. After analysing the results, the e-NABLE Unlimbited arm was printed on The Original Prusa i3 MK3, where polylactic acid (PLA) was used. This dissertation showed that the results obtained in the 250-millisecond window were better than the obtained ones in a 100-millisecond window. In general, the best classifier was the K-Nearest Neighbours (KNN) with k=2, except for the first level that was LDA. The best results were obtained for the first classification level, with an accuracy greater than 90%. Although the results obtained for the second and third levels were close to 80%, it was concluded that it was impossible to develop a microcontroller dependent only on one acquisition channel. These results agree with the anatomical characteristics since they are originated from the same muscle group. The cross-validation results were lower than those obtained in the training-test methodology, which allowed us to conclude that the inter variability that exists between the subjects significantly affects the classification performance. Furthermore, both the dominant and non-dominant arms were used in this work, which also increased the discrepancy between signals. Indeed, the results showed that it is impossible to develop a microcontroller adaptable to all users. Therefore, in the future, the best path will be to opt for the customization of the prototype. In order to test the implementation of a microcontroller in the printed model, it was necessary to design a support structure in Solidworks that would support the motors used to flex the fingers and Arduino to control the motors. Consequently, the e-NABLE model was re adapted, making it possible to develop a clinical training prototype. Even though it is a training prototype, it is lighter than those on the market and cheaper. The objectives of this work have been fulfilled and many answers have been given. However, there is always space for improvement. Although, this dissertation has some limitations, it certainly contributed to clarify many of the doubts that still exist in the scientific community. Hopefully, it will help to further develop the prosthetic industry.Nos últimos anos, o aumento do número de acidentes por doenças crónicas, como, por exemplo, a diabetes, e o empobrecimento de determinados países em desenvolvimento têm contribuído para um aumento significativo no número de utilizadores de próteses. A perda de um determinado membro acarreta inúmeras mudanças no dia-a-dia de cada utilizador. Estas são amplificadas quando a perda é referente à mão ou parte do antebraço. A mão é uma ferramenta essencial no dia-a-dia de cada ser humano, uma vez que é através dela que são realizadas as atividades básicas, como, por exemplo, tomar banho, lavar os dentes, comer, preparar refeições, etc. A substituição desta ferramenta é, portanto, uma necessidade, não só porque permitirá restabelecer as funções físicas e motoras do membro superior, como, também, reduzirá o nível de dependência destes utilizadores de outrem e, consequentemente, das taxas de depressão. Para colmatar as necessidades dos utilizadores, a indústria prostética tem-se reinventado e evoluído, desenvolvendo próteses para o membro superior cada vez mais sofisticadas. Com efeito, já é possível controlar uma prótese através da leitura e análise dos sinais mioelétricos do próprio utilizador, o que é denominado por muitos investigadores de controlo por reconhecimento de padrões. Este tipo de controlo é personalizável e permite adaptar a prótese a cada utilizador. Para além do uso de sinais elétricos provenientes do musculo do utilizador, a impressão 3D, uma técnica de manufatura aditiva, têm ganho força no campo da prostética. Por conseguinte, nos últimos anos os investigadores têm impresso inúmeros modelos com diferentes materiais que vão desde o uso de termoplásticos, ao uso de materiais flexíveis. A utilização deste tipo de tecnologia permite, para além de uma rápida entrega do produto ao utilizador, uma diminuição no tempo de construção de uma prótese tornando-a mais leve e barata. Além do mais, a impressão 3D permite criar protótipos mais sustentáveis, uma vez que existe uma redução na quantidade de material desperdiçado. Embora já existam inúmeras soluções, a taxa de rejeição deste tipo de dispositivos é ainda bastante elevada, uma vez que a maioria das próteses disponíveis no mercado, nomeadamente as mioelétricas, são lentas, caras e pesadas. Ainda que existam alguns estudos que se debrucem neste tipo de tecnologias, bem como na sua evolução científica, o número de elétrodos utilizados é ainda significativo. Desta forma, e, tendo em conta que a maioria dos utilizadores não possuí uma área de superfície do antebraço suficiente para ser feita a aquisição dos sinais mioelétricos, o trabalho feito pela academia não se revelou tão contributivo para a indústria prostética como este prometia inicialmente. Este trabalho pretende resolver alguns desses problemas e responder às questões mais impostas pela indústria e investigadores, para que, no futuro, o número de utilizadores possa aumentar, assim como o seu índice de satisfação relativamente ao produto. Para tal, recolheram-se os sinais mioelétricos de quarenta voluntários, através do sistema de aquisição BIOPAC. Após a recolha, filtraram-se os sinais de seis voluntários através de dois processos diferentes. No primeiro, utilizaram-se filtros digitais e no segundo aplicou-se a transformada de onda para a redução do ruído. De seguida, o sinal foi segmentado em janelas mais pequenas de 100 e 250 milissegundos e extraíram-se treze features no domínio temporal. Para que o processo de classificação fosse otimizado, foram aplicados três métodos de seleção de features. A classificação foi dividida em três níveis diferentes nos quais dois algoritmos de aprendizagem automática foram implementados, individualmente. No primeiro nível, o objetivo foi a distinção entre os momentos em que o voluntário fazia movimento ou que estava em repouso. Caso o output do classificador fosse a classe movimento, este teria de, num segundo nível, tentar perceber se o voluntário estaria a mexer apenas um dedo ou a realizar um movimento que envolvesse a flexão de mais de que um dedo (preensão). No caso de uma preensão, passava-se ao terceiro nível onde o classificador teria de identificar se o voluntário estaria a realizar a preensão esférica ou em tríade. Para todos os níveis de classificação, obtiveram-se resultados para o método de validação cruzada e o método de teste e treino, sendo que neste, 70% dos dados foram utilizados como conjunto de treino e 30% como teste. Efetuada a análise dos resultados, escolheu-se um dos modelos da comunidade e-NABLE. O modelo foi impresso na impressora The Original Prusa i3 MK3S e o material escolhido foi o ácido poliláctico (PLA). Para que fosse possível testar a implementação de um microcontrolador num modelo que originalmente depende da flexão do cotovelo realizada pelo utilizador, foi necessário desenhar uma estrutura de suporte que suportasse, não só os motores utilizados para flexionar os dedos, como, também, o Arduíno. O suporte desenhado foi impresso com o mesmo material e com a mesma impressora. Os resultados obtidos mostraram que a janela de 250 milissegundo foi a melhor e que, regra geral, o melhor classificador é o K-Nearest Neighbors (KNN) com k=2, com exceção do primeiro nível, em que o melhor classificador foi o Linear Discriminant Analysis (LDA). Os melhores resultados obtiveram-se no primeiro nível de classificação onde a accuracy foi superior a 90%. Embora os resultados obtidos para o segundo e terceiro nível tenham sido próximos de 80%, concluiu-se que não era possível desenvolver um microcontrolador dependente apenas de um canal de aquisição. Tal era expectável, uma vez que os movimentos estudados são originados pelo mesmo grupo muscular e a intervariabilidade dos sujeitos um fator significativo. Os resultados da validação cruzada foram menos precisos do que os obtidos para a metodologia de treino-teste, o que permitiu concluir que a intervariabilidade existente entre os voluntários afeta significativamente o processo de classificação. Para além disso, os voluntários utilizaram o braço dominante e o braço não dominante, o que acabou por aumentar a discrepância entre os sinais recolhidos. Com efeito, os resultados mostraram que não é possível desenvolver um microcontrolador que seja adaptável a todos os utilizadores e, portanto, no futuro, o melhor caminho será optar pela personalização do protótipo. Tendo o conhecimento prévio desta evidência, o protótipo desenvolvido neste trabalho apenas servirá como protótipo de treino para o utilizador. Ainda assim, este é bem mais leve que os existentes no mercado e muito mais barato. Nele é ainda possível testar e controlar alguns dos componentes que no futuro irão fazer parte da prótese completa, prevenindo acidentes. Não obstante o cumprimento dos objetivos deste trabalho e das muitas respostas que por ele foram dadas, existe sempre espaço para melhorias. Dado à limitação de tempo, não foi possível testar o microcontrolador em tempo-real nem efetuar testes mecânicos de flexibilidade e resistência dos materiais da prótese. Deste modo, seria interessante no futuro fazer testes de performance em tempo real e submeter a prótese a condições extremas, para que a tensão elástica e a tensão dos pins sejam testadas. Para além disso, testar os mecanismos de segurança da prótese quando o utilizador tem de fazer muita força é fundamental. O teste destes parâmetros evitará a ocorrência de falhas que poderão magoar o utilizador, bem como estragar os objetos com os quais a prótese poderá interagir. Por fim, é necessário melhorar o aspeto cosmético das próteses. Para que isso aconteça, poderão ser utilizados polímeros com uma coloração próxima do tom da pele do utilizador. Uma outra forma de melhorar este aspeto, seria fazer o scanning do braço saudável do utilizador e usar materiais flexíveis para as articulações e dedos que, juntamente com uma palma de termoplásticos resistentes e um microcontrolador, permitissem um movimento bastante natural próximo do biológico. Em suma, apesar de algumas limitações, este trabalho contribuiu para o esclarecimento de muitas das dúvidas que ainda existiam na comunidade científica e ajudará a desenvolver a indústria prostética