11 research outputs found

    A non-invasive human-machine interfacing framework for investigating dexterous control of hand muscles

    Get PDF
    The recent fast development of virtual reality and robotic assistive devices enables to augment the capabilities of able-body individuals as well as to overcome the motor missing functions of neurologically impaired or amputee individuals. To control these devices, movement intentions can be captured from biological structures involved in the process of motor planning and execution, such as the central nervous system (CNS), the peripheral nervous system (in particular the spinal motor neurons) and the musculoskeletal system. Thus, human-machine interfaces (HMI) enable to transfer neural information from the neuro-muscular system to machines. To prevent any risks due to surgical operations or tissue damage in implementing these HMIs, a non-invasive approach is proposed in this thesis. In the last five decades, surface electromyography (sEMG) has been extensively explored as a non-invasive source of neural information. EMG signals are constituted by the mixed electrical activity of several recruited motor units, the fundamental components of muscle contraction. High-density sEMG (HD-sEMG) with the use of blind source separation methods enabled to identify the discharge patterns of many of these active motor units. From these decomposed discharge patterns, the net common synaptic input (CSI) to the corresponding spinal motor neurons was quantified with cross-correlation in the time and frequency domain or with principal component analysis (PCA) on one or few muscles. It has been hypothesised that this CSI would result from the contribution of spinal descending commands sent by supra-spinal structures and afferences integrated by spinal interneurons. Another motor strategy implying the integration of descending commands at the spinal level is the one regarding the coordination of many muscles to control a large number of articular joints. This neurophysiological mechanism was investigated by measuring a single EMG amplitude per muscle, thus without the use of HD-sEMG and decomposition. In this case, the aim was to understand how the central nervous system (CNS) could control a large set of muscles actuating a vast set of combinations of degrees of freedom in a modular way. Thus, time-invariant patterns of muscle coordination, i.e. muscle synergies , were found in animals and humans from EMG amplitude of many muscles, modulated by time-varying commands to be combined to fulfil complex movements. In this thesis, for the first time, we present a non-invasive framework for human-machine interfaces based on both spinal motor neuron recruitment strategy and muscle synergistic control for unifying the understanding of these two motor control strategies and producing control signals correlated to biomechanical quantities. This implies recording both from many muscles and using HD-sEMG for each muscle. We investigated 14 muscles of the hand, 6 extrinsic and 8 intrinsic. The first two studies, (in Chapters 2 and 3, respectively) present the framework for CSI quantification by PCA and the extraction of the synergistic organisation of spinal motor neurons innervating the 14 investigated muscles. For the latter analysis, in Chapter 3, we proposed the existence of what we named as motor neuron synergies extracted with non-negative matrix factorisation (NMF) from the identified motor neurons. In these first two studies, we considered 7 subjects and 7 grip types involving differently all the four fingers in opposition with the thumb. In the first study, we found that the variance explained by the CSI among all motor neuron spike trains was (53.0 ± 10.9) % and its cross-correlation with force was 0.67 ± 0.10, remarkably high with respect to previous findings. In the second study, 4 motor neuron synergies were identified and associated with the actuation of one finger in opposition with the thumb, finding even higher correlation values with force (over 0.8) for each synergy associated with a finger during the actuation of the relative finger. In Chapter 4, we then extended the set of analysed movements in a vast repertoire of gestures and repeated the analysis of Chapter 3 by finding a different synergistic organisation during the execution of tens of tasks. We divided the contribution among extrinsic and intrinsic muscles and we found that intrinsic better enable single-finger spatial discrimination, while no difference was found in regression of joint angles by dividing the two groups of muscles. Finally, in Chapter 5 we proposed the techniques of the previous chapters for cases of impairment due both to amputation and stroke. We analysed one case of pre and post rehabilitation sessions of a trans-humeral amputee, the case of a post-stroke trans-radial amputee and three cases of acute stroke, i.e. less than one month from the stroke event. We present future perspectives (Chapter 6) aimed to design and implement a platform for both rehabilitation monitoring and myoelectric control. Thus, this thesis provides a bridge between two extensively studied motor control mechanisms, i.e. motor neuron recruitment and muscle synergies, and proposes this framework as suitable for rehabilitation monitoring and control of assistive devices.Open Acces

    Deep Learning Methods for Hand Gesture Recognition via High-Density Surface Electromyogram (HD-sEMG) Signals

    Get PDF
    Hand Gesture Recognition (HGR) using surface Electromyogram (sEMG) signals can be considered as one of the most important technologies in making efficient Human Machine Interface (HMI) systems. In particular, sEMG-based hand gesture has been a topic of growing interest for development of assistive systems to improve the quality of life in individuals suffering from amputated limbs. Generally speaking, myoelectric prosthetic devices work by classifying existing patterns of the collected sEMG signals and synthesizing intended gestures. While conventional myoelectric control systems, e.g., on/off control or direct-proportional, have potential advantages, challenges such as limited Degree of Freedom (DoF) due to crosstalk have resulted in the emergence of data-driven solutions. More specifically, to improve efficiency, intuitiveness, and the control performance of hand prosthetic systems, several Artificial Intelligence (AI) algorithms ranging from conventional Machine Learning (ML) models to highly complicated Deep Neural Network (DNN) architectures have been designed for sEMG-based hand gesture recognition in myoelectric prosthetic devices. In this thesis, we, first, perform a literature review on hand gesture recognition methods and elaborate on the recently proposed Deep Learning/Machine Learning (DL/ML) models in the literature. Then, our utilized High-Density sEMG (HD-sEMG) dataset is introduced and the rationales behind our main focus on this particular type of sEMG dataset are explained. We, then, develop a Vision Transformer (ViT)-based model for gesture recognition with HD-sEMG signals and evaluate its performance under different conditions such as variable window sizes, number of electrode channels, and model's complexity. We compare its performance with that of two conventional ML and one DL algorithm that are typically adopted in this domain. Furthermore, we introduce another capability of our proposed framework for instantaneous training, which is its ability to classify hand gestures based on a single frame of HD-sEMG dataset. Following that, we introduce the idea of integrating the macroscopic and microscopic neural drive information obtained from HD-sEMG data into a hybrid ViT-based framework for gesture recognition, which outperforms a standalone ViT architecture in terms of classification accuracy. Here, microscopic neural drive information (also called Motor Unit Spike Trains) refers to the neural commands sent by the brain and spinal cord to individual muscle fibers and are extracted from HD-sEMG signals using Blind Source Separation (BSP) algorithms. Finally, we design an alternative and novel hand gesture recognition model based on the less-explored topic of Spiking Neural Networks (SNN), which performs spatio-temporal gesture recognition in an event-based fashion. As opposed to the classical DNN architectures, SNNs are of the capacity to imitate human brain's cognitive function by using biologically inspired models of neurons and synapses. Therefore, they are more biologically explainable and computationally efficient

    Deep spiking neural networks with applications to human gesture recognition

    Get PDF
    The spiking neural networks (SNNs), as the 3rd generation of Artificial Neural Networks (ANNs), are a class of event-driven neuromorphic algorithms that potentially have a wide range of application domains and are applicable to a variety of extremely low power neuromorphic hardware. The work presented in this thesis addresses the challenges of human gesture recognition using novel SNN algorithms. It discusses the design of these algorithms for both visual and auditory domain human gesture recognition as well as event-based pre-processing toolkits for audio signals. From the visual gesture recognition aspect, a novel SNN-based event-driven hand gesture recognition system is proposed. This system is shown to be effective in an experiment on hand gesture recognition with its spiking recurrent convolutional neural network (SCRNN) design, which combines both designed convolution operation and recurrent connectivity to maintain spatial and temporal relations with address-event-representation (AER) data. The proposed SCRNN architecture can achieve arbitrary temporal resolution, which means it can exploit temporal correlations between event collections. This design utilises a backpropagation-based training algorithm and does not suffer from gradient vanishing/explosion problems. From the audio perspective, a novel end-to-end spiking speech emotion recognition system (SER) is proposed. This system employs the MFCC as its main speech feature extractor as well as a self-designed latency coding algorithm to effciently convert the raw signal to AER input that can be used for SNN. A two-layer spiking recurrent architecture is proposed to address temporal correlations between spike trains. The robustness of this system is supported by several open public datasets, which demonstrate state of the arts recognition accuracy and a significant reduction in network size, computational costs, and training speed. In addition to directly contributing to neuromorphic SER, this thesis proposes a novel speech-coding algorithm based on the working mechanism of humans auditory organ system. The algorithm mimics the functionality of the cochlea and successfully provides an alternative method of event-data acquisition for audio-based data. The algorithm is then further simplified and extended into an application of speech enhancement which is jointly used in the proposed SER system. This speech-enhancement method uses the lateral inhibition mechanism as a frequency coincidence detector to remove uncorrelated noise in the time-frequency spectrum. The method is shown to be effective by experiments for up to six types of noise.The spiking neural networks (SNNs), as the 3rd generation of Artificial Neural Networks (ANNs), are a class of event-driven neuromorphic algorithms that potentially have a wide range of application domains and are applicable to a variety of extremely low power neuromorphic hardware. The work presented in this thesis addresses the challenges of human gesture recognition using novel SNN algorithms. It discusses the design of these algorithms for both visual and auditory domain human gesture recognition as well as event-based pre-processing toolkits for audio signals. From the visual gesture recognition aspect, a novel SNN-based event-driven hand gesture recognition system is proposed. This system is shown to be effective in an experiment on hand gesture recognition with its spiking recurrent convolutional neural network (SCRNN) design, which combines both designed convolution operation and recurrent connectivity to maintain spatial and temporal relations with address-event-representation (AER) data. The proposed SCRNN architecture can achieve arbitrary temporal resolution, which means it can exploit temporal correlations between event collections. This design utilises a backpropagation-based training algorithm and does not suffer from gradient vanishing/explosion problems. From the audio perspective, a novel end-to-end spiking speech emotion recognition system (SER) is proposed. This system employs the MFCC as its main speech feature extractor as well as a self-designed latency coding algorithm to effciently convert the raw signal to AER input that can be used for SNN. A two-layer spiking recurrent architecture is proposed to address temporal correlations between spike trains. The robustness of this system is supported by several open public datasets, which demonstrate state of the arts recognition accuracy and a significant reduction in network size, computational costs, and training speed. In addition to directly contributing to neuromorphic SER, this thesis proposes a novel speech-coding algorithm based on the working mechanism of humans auditory organ system. The algorithm mimics the functionality of the cochlea and successfully provides an alternative method of event-data acquisition for audio-based data. The algorithm is then further simplified and extended into an application of speech enhancement which is jointly used in the proposed SER system. This speech-enhancement method uses the lateral inhibition mechanism as a frequency coincidence detector to remove uncorrelated noise in the time-frequency spectrum. The method is shown to be effective by experiments for up to six types of noise

    Biomechatronics: Harmonizing Mechatronic Systems with Human Beings

    Get PDF
    This eBook provides a comprehensive treatise on modern biomechatronic systems centred around human applications. A particular emphasis is given to exoskeleton designs for assistance and training with advanced interfaces in human-machine interaction. Some of these designs are validated with experimental results which the reader will find very informative as building-blocks for designing such systems. This eBook will be ideally suited to those researching in biomechatronic area with bio-feedback applications or those who are involved in high-end research on manmachine interfaces. This may also serve as a textbook for biomechatronic design at post-graduate level

    Intelligent Biosignal Processing in Wearable and Implantable Sensors

    Get PDF
    This reprint provides a collection of papers illustrating the state-of-the-art of smart processing of data coming from wearable, implantable or portable sensors. Each paper presents the design, databases used, methodological background, obtained results, and their interpretation for biomedical applications. Revealing examples are brain–machine interfaces for medical rehabilitation, the evaluation of sympathetic nerve activity, a novel automated diagnostic tool based on ECG data to diagnose COVID-19, machine learning-based hypertension risk assessment by means of photoplethysmography and electrocardiography signals, Parkinsonian gait assessment using machine learning tools, thorough analysis of compressive sensing of ECG signals, development of a nanotechnology application for decoding vagus-nerve activity, detection of liver dysfunction using a wearable electronic nose system, prosthetic hand control using surface electromyography, epileptic seizure detection using a CNN, and premature ventricular contraction detection using deep metric learning. Thus, this reprint presents significant clinical applications as well as valuable new research issues, providing current illustrations of this new field of research by addressing the promises, challenges, and hurdles associated with the synergy of biosignal processing and AI through 16 different pertinent studies. Covering a wide range of research and application areas, this book is an excellent resource for researchers, physicians, academics, and PhD or master students working on (bio)signal and image processing, AI, biomaterials, biomechanics, and biotechnology with applications in medicine

    Augmentation of Brain Function: Facts, Fiction and Controversy. Volume III: From Clinical Applications to Ethical Issues and Futuristic Ideas

    Get PDF
    The final volume in this tripartite series on Brain Augmentation is entitled “From Clinical Applications to Ethical Issues and Futuristic Ideas”. Many of the articles within this volume deal with translational efforts taking the results of experiments on laboratory animals and applying them to humans. In many cases, these interventions are intended to help people with disabilities in such a way so as to either restore or extend brain function. Traditionally, therapies in brain augmentation have included electrical and pharmacological techniques. In contrast, some of the techniques discussed in this volume add specificity by targeting select neural populations. This approach opens the door to where and how to promote the best interventions. Along the way, results have empowered the medical profession by expanding their understanding of brain function. Articles in this volume relate novel clinical solutions for a host of neurological and psychiatric conditions such as stroke, Parkinson’s disease, Huntington’s disease, epilepsy, dementia, Alzheimer’s disease, autism spectrum disorders (ASD), traumatic brain injury, and disorders of consciousness. In disease, symptoms and signs denote a departure from normal function. Brain augmentation has now been used to target both the core symptoms that provide specificity in the diagnosis of a disease, as well as other constitutional symptoms that may greatly handicap the individual. The volume provides a report on the use of repetitive transcranial magnetic stimulation (rTMS) in ASD with reported improvements of core deficits (i.e., executive functions). TMS in this regard departs from the present-day trend towards symptomatic treatment that leaves unaltered the root cause of the condition. In diseases, such as schizophrenia, brain augmentation approaches hold promise to avoid lengthy pharmacological interventions that are usually riddled with side effects or those with limiting returns as in the case of Parkinson’s disease. Brain stimulation can also be used to treat auditory verbal hallucination, visuospatial (hemispatial) neglect, and pain in patients suffering from multiple sclerosis. The brain acts as a telecommunication transceiver wherein different bandwidth of frequencies (brainwave oscillations) transmit information. Their baseline levels correlate with certain behavioral states. The proper integration of brain oscillations provides for the phenomenon of binding and central coherence. Brain augmentation may foster the normalization of brain oscillations in nervous system disorders. These techniques hold the promise of being applied remotely (under the supervision of medical personnel), thus overcoming the obstacle of travel in order to obtain healthcare. At present, traditional thinking would argue the possibility of synergism among different modalities of brain augmentation as a way of increasing their overall effectiveness and improving therapeutic selectivity. Thinking outside of the box would also provide for the implementation of brain-to-brain interfaces where techniques, proper to artificial intelligence, could allow us to surpass the limits of natural selection or enable communications between several individual brains sharing memories, or even a global brain capable of self-organization. Not all brains are created equal. Brain stimulation studies suggest large individual variability in response that may affect overall recovery/treatment, or modify desired effects of a given intervention. The subject’s age, gender, hormonal levels may affect an individual’s cortical excitability. In addition, this volume discusses the role of social interactions in the operations of augmenting technologies. Finally, augmenting methods could be applied to modulate consciousness, even though its neural mechanisms are poorly understood. Finally, this volume should be taken as a debate on social, moral and ethical issues on neurotechnologies. Brain enhancement may transform the individual into someone or something else. These techniques bypass the usual routes of accommodation to environmental exigencies that exalted our personal fortitude: learning, exercising, and diet. This will allow humans to preselect desired characteristics and realize consequent rewards without having to overcome adversity through more laborious means. The concern is that humans may be playing God, and the possibility of an expanding gap in social equity where brain enhancements may be selectively available to the wealthier individuals. These issues are discussed by a number of articles in this volume. Also discussed are the relationship between the diminishment and enhancement following the application of brain-augmenting technologies, the problem of “mind control” with BMI technologies, free will the duty to use cognitive enhancers in high-responsibility professions, determining the population of people in need of brain enhancement, informed public policy, cognitive biases, and the hype caused by the development of brain- augmenting approaches

    XLIII Jornadas de Automática: libro de actas: 7, 8 y 9 de septiembre de 2022, Logroño (La Rioja)

    Get PDF
    [Resumen] Las Jornadas de Automática (JA) son el evento más importante del Comité Español de Automática (CEA), entidad científico-técnica con más de cincuenta años de vida y destinada a la difusión e implantación de la Automática en la sociedad. Este año se celebra la cuadragésima tercera edición de las JA, que constituyen el punto de encuentro de la comunidad de Automática de nuestro país. La presente edición permitirá dar visibilidad a los nuevos retos y resultados del ámbito, y su uso en un gran número de aplicaciones, entre otras, las energías renovables, la bioingeniería o la robótica asistencial. Además de la componente científica, que se ve reflejada en este libro de actas, las JA son un punto de encuentro de las diferentes generaciones de profesores, investigadores y profesionales, incluyendo la componente social que es de vital importancia. Esta edición 2022 de las JA se celebra en Logroño, capital de La Rioja, región mundialmente conocida por la calidad de sus vinos de Denominación de Origen y que ha asumido el desafío de poder ganar competitividad a través de la transformación verde y digital. Pero también por ser la cuna del castellano e impulsar el Valle de la Lengua con la ayuda de las nuevas tecnologías, entre ellas la Automática Inteligente. Los organizadores de estas JA, pertenecientes al Área de Ingeniería de Sistemas y Automática del Departamento de Ingeniería Eléctrica de la Universidad de La Rioja (UR), constituyen un pilar fundamental en el apoyo a la región para el estudio, implementación y difusión de estos retos. Esta edición, la primera en formato íntegramente presencial después de la pandemia de la covid-19, cuenta con más de 200 asistentes y se celebra a caballo entre el Edificio Politécnico de la Escuela Técnica Superior de Ingeniería Industrial y el Monasterio de Yuso situado en San Millán de la Cogolla, dos marcos excepcionales para la realización de las JA. Como parte del programa científico, dos sesiones plenarias harán hincapié, respectivamente, sobre soluciones de control para afrontar los nuevos retos energéticos, y sobre la calidad de los datos para una inteligencia artificial (IA) imparcial y confiable. También, dos mesas redondas debatirán aplicaciones de la IA y la implantación de la tecnología digital en la actividad profesional. Adicionalmente, destacaremos dos clases magistrales alineadas con tecnología de última generación que serán impartidas por profesionales de la empresa. Las JA también van a albergar dos competiciones: CEABOT, con robots humanoides, y el Concurso de Ingeniería de Control, enfocado a UAVs. A todas estas actividades hay que añadir las reuniones de los grupos temáticos de CEA, las exhibiciones de pósteres con las comunicaciones presentadas a las JA y los expositores de las empresas. Por último, durante el evento se va a proceder a la entrega del “Premio Nacional de Automática” (edición 2022) y del “Premio CEA al Talento Femenino en Automática”, patrocinado por el Gobierno de La Rioja (en su primera edición), además de diversos galardones enmarcados dentro de las actividades de los grupos temáticos de CEA. Las actas de las XLIII Jornadas de Automática están formadas por un total de 143 comunicaciones, organizadas en torno a los nueve Grupos Temáticos y a las dos Líneas Estratégicas de CEA. Los trabajos seleccionados han sido sometidos a un proceso de revisión por pares
    corecore