Affective Man-Machine Interface: Unveiling human emotions through biosignals

As is known for centuries, humans exhibit an electrical profile. This profile is altered through various psychological and physiological processes, which can be measured through biosignals; e.g., electromyography (EMG) and electrodermal activity (EDA). These biosignals can reveal our emotions and, as such, can serve as an advanced man-machine interface (MMI) for empathic consumer products. However, such a MMI requires the correct classification of biosignals to emotion classes. This chapter starts with an introduction on biosignals for emotion detection. Next, a state-of-the-art review is presented on automatic emotion classification. Moreover, guidelines are presented for affective MMI. Subsequently, a research is presented that explores the use of EDA and three facial EMG signals to determine neutral, positive, negative, and mixed emotions, using recordings of 21 people. A range of techniques is tested, which resulted in a generic framework for automated emotion classification with up to 61.31% correct classification of the four emotion classes, without the need of personal profiles. Among various other directives for future research, the results emphasize the need for parallel processing of multiple biosignals

Towards Domain Generalization for ECG and EEG Classification: Algorithms and Benchmarks

Despite their immense success in numerous fields, machine and deep learning systems have not yet been able to firmly establish themselves in mission-critical applications in healthcare. One of the main reasons lies in the fact that when models are presented with previously unseen, Out-of-Distribution samples, their performance deteriorates significantly. This is known as the Domain Generalization (DG) problem. Our objective in this work is to propose a benchmark for evaluating DG algorithms, in addition to introducing a novel architecture for tackling DG in biosignal classification. In this paper, we describe the Domain Generalization problem for biosignals, focusing on electrocardiograms (ECG) and electroencephalograms (EEG) and propose and implement an open-source biosignal DG evaluation benchmark. Furthermore, we adapt state-of-the-art DG algorithms from computer vision to the problem of 1D biosignal classification and evaluate their effectiveness. Finally, we also introduce a novel neural network architecture that leverages multi-layer representations for improved model generalizability. By implementing the above DG setup we are able to experimentally demonstrate the presence of the DG problem in ECG and EEG datasets. In addition, our proposed model demonstrates improved effectiveness compared to the baseline algorithms, exceeding the state-of-the-art in both datasets. Recognizing the significance of the distribution shift present in biosignal datasets, the presented benchmark aims at urging further research into the field of biomedical DG by simplifying the evaluation process of proposed algorithms. To our knowledge, this is the first attempt at developing an open-source framework for evaluating ECG and EEG DG algorithms.Comment: Accepted in IEEE Transactions on Emerging Topics in Computational Intelligenc

Detection of abnormalities in ECG using Deep Learning

A significant part of healthcare is focused on the information that the physiological signals offer about the health state of an individual. The Electrocardiogram (ECG) cyclic behaviour gives insight on a subject’s emotional, behavioral and cardiovascular state. These signals often present abnormal events that affects their analysis. Two examples are the noise, that occurs during the acquisition, and symptomatic patterns, that are produced by pathologies. This thesis proposes a Deep Neural Networks framework that learns the normal behaviour of an ECG while detecting abnormal events, tested in two different settings: detection of different types of noise, and; symptomatic events caused by different pathologies. Two algorithms were developed for noise detection, using an autoencoder and Convolutional Neural Networks (CNN), reaching accuracies of 98,18% for the binary class model and 70,74% for the multi-class model, which is able to discern between base wandering, muscle artifact and electrode motion noise. As for the arrhythmia detection algorithm was developed using an autoencoder and Recurrent Neural Networks with Gated Recurrent Units (GRU) architecture. With an accuracy of 56,85% and an average sensitivity of 61.13%, compared to an average sensitivity of 75.22% for a 12 class model developed by Hannun et al. The model detects 7 classes: normal sinus rhythm, paced rhythm, ventricular bigeminy, sinus bradycardia, atrial fibrillation, atrial flutter and pre-excitation. It was concluded that the process of learning the machine learned features of the normal ECG signal, currently sacrifices the accuracy for higher generalization. It performs better at discriminating the presence of abnormal events in ECG than classifying different types of events. In the future, these algorithms could represent a huge contribution in signal acquisition for wearables and the study of pathologies visible in not only in ECG, but also EMG and respiratory signals, especially applied to active learning

Learning Biosignals with Deep Learning

The healthcare system, which is ubiquitously recognized as one of the most influential system in society, is facing new challenges since the start of the decade.The myriad of physiological data generated by individuals, namely in the healthcare system, is generating a burden on physicians, losing effectiveness on the collection of patient data. Information systems and, in particular, novel deep learning (DL) algorithms have been prompting a way to take this problem. This thesis has the aim to have an impact in biosignal research and industry by presenting DL solutions that could empower this field. For this purpose an extensive study of how to incorporate and implement Convolutional Neural Networks (CNN), Recursive Neural Networks (RNN) and Fully Connected Networks in biosignal studies is discussed. Different architecture configurations were explored for signal processing and decision making and were implemented in three different scenarios: (1) Biosignal learning and synthesis; (2) Electrocardiogram (ECG) biometric systems, and; (3) Electrocardiogram (ECG) anomaly detection systems. In (1) a RNN-based architecture was able to replicate autonomously three types of biosignals with a high degree of confidence. As for (2) three CNN-based architectures, and a RNN-based architecture (same used in (1)) were used for both biometric identification, reaching values above 90% for electrode-base datasets (Fantasia, ECG-ID and MIT-BIH) and 75% for off-person dataset (CYBHi), and biometric authentication, achieving Equal Error Rates (EER) of near 0% for Fantasia and MIT-BIH and bellow 4% for CYBHi. As for (3) the abstraction of healthy clean the ECG signal and detection of its deviation was made and tested in two different scenarios: presence of noise using autoencoder and fully-connected network (reaching 99% accuracy for binary classification and 71% for multi-class), and; arrhythmia events by including a RNN to the previous architecture (57% accuracy and 61% sensitivity). In sum, these systems are shown to be capable of producing novel results. The incorporation of several AI systems into one could provide to be the next generation of preventive medicine, as the machines have access to different physiological and anatomical states, it could produce more informed solutions for the issues that one may face in the future increasing the performance of autonomous preventing systems that could be used in every-day life in remote places where the access to medicine is limited. These systems will also help the study of the signal behaviour and how they are made in real life context as explainable AI could trigger this perception and link the inner states of a network with the biological traits.O sistema de saúde, que é ubiquamente reconhecido como um dos sistemas mais influentes da sociedade, enfrenta novos desafios desde o ínicio da década. A miríade de dados fisiológicos gerados por indíviduos, nomeadamente no sistema de saúde, está a gerar um fardo para os médicos, perdendo a eficiência no conjunto dos dados do paciente. Os sistemas de informação e, mais espcificamente, da inovação de algoritmos de aprendizagem profunda (DL) têm sido usados na procura de uma solução para este problema. Esta tese tem o objetivo de ter um impacto na pesquisa e na indústria de biosinais, apresentando soluções de DL que poderiam melhorar esta área de investigação. Para esse fim, é discutido um extenso estudo de como incorporar e implementar redes neurais convolucionais (CNN), redes neurais recursivas (RNN) e redes totalmente conectadas para o estudo de biosinais. Diferentes arquiteturas foram exploradas para processamento e tomada de decisão de sinais e foram implementadas em três cenários diferentes: (1) Aprendizagem e síntese de biosinais; (2) sistemas biométricos com o uso de eletrocardiograma (ECG), e; (3) Sistema de detecção de anomalias no ECG. Em (1) uma arquitetura baseada na RNN foi capaz de replicar autonomamente três tipos de sinais biológicos com um alto grau de confiança. Quanto a (2) três arquiteturas baseadas em CNN e uma arquitetura baseada em RNN (a mesma usada em (1)) foram usadas para ambas as identificações, atingindo valores acima de 90 % para conjuntos de dados à base de eletrodos (Fantasia, ECG-ID e MIT -BIH) e 75 % para o conjunto de dados fora da pessoa (CYBHi) e autenticação, atingindo taxas de erro iguais (EER) de quase 0 % para Fantasia e MIT-BIH e abaixo de 4 % para CYBHi. Quanto a (3) a abstração de sinais limpos e assimptomáticos de ECG e a detecção do seu desvio foram feitas e testadas em dois cenários diferentes: na presença de ruído usando um autocodificador e uma rede totalmente conectada (atingindo 99 % de precisão na classificação binária e 71 % na multi-classe), e; eventos de arritmia incluindo um RNN na arquitetura anterior (57 % de precisão e 61 % de sensibilidade). Em suma, esses sistemas são mais uma vez demonstrados como capazes de produzir resultados inovadores. A incorporação de vários sistemas de inteligência artificial em um unico sistema pederá desencadear a próxima geração de medicina preventiva. Os algoritmos ao terem acesso a diferentes estados fisiológicos e anatómicos, podem produzir soluções mais informadas para os problemas que se possam enfrentar no futuro, aumentando o desempenho de sistemas autónomos de prevenção que poderiam ser usados na vida quotidiana, nomeadamente em locais remotos onde o acesso à medicinas é limitado. Estes sistemas também ajudarão o estudo do comportamento do sinal e como eles são feitos no contexto da vida real, pois a IA explicável pode desencadear essa percepção e vincular os estados internos de uma rede às características biológicas

Unsupervised Heart-rate Estimation in Wearables With Liquid States and A Probabilistic Readout

Heart-rate estimation is a fundamental feature of modern wearable devices. In this paper we propose a machine intelligent approach for heart-rate estimation from electrocardiogram (ECG) data collected using wearable devices. The novelty of our approach lies in (1) encoding spatio-temporal properties of ECG signals directly into spike train and using this to excite recurrently connected spiking neurons in a Liquid State Machine computation model; (2) a novel learning algorithm; and (3) an intelligently designed unsupervised readout based on Fuzzy c-Means clustering of spike responses from a subset of neurons (Liquid states), selected using particle swarm optimization. Our approach differs from existing works by learning directly from ECG signals (allowing personalization), without requiring costly data annotations. Additionally, our approach can be easily implemented on state-of-the-art spiking-based neuromorphic systems, offering high accuracy, yet significantly low energy footprint, leading to an extended battery life of wearable devices. We validated our approach with CARLsim, a GPU accelerated spiking neural network simulator modeling Izhikevich spiking neurons with Spike Timing Dependent Plasticity (STDP) and homeostatic scaling. A range of subjects are considered from in-house clinical trials and public ECG databases. Results show high accuracy and low energy footprint in heart-rate estimation across subjects with and without cardiac irregularities, signifying the strong potential of this approach to be integrated in future wearable devices.Comment: 51 pages, 12 figures, 6 tables, 95 references. Under submission at Elsevier Neural Network

Algorithms for information extraction and signal annotation on long-term biosignals using clustering techniques

Dissertação para obtenção do Grau de Mestre em Engenharia BiomédicaOne of the biggest challenges when analysing data is to extract information from it, especially if we dealing with very large sized data, which brings a new set of barriers to be overcome. The extracted information can be used to aid physicians in their diagnosis since biosignals often carry vital information on the subjects. In this research work, we present a signal-independent algorithm with two main goals: perform events detection in biosignals and, with those events, extract information using a set of distance measures which will be used as input to a parallel version of the k-means clustering algorithm. The first goal is achieved by using two different approaches. Events can be found based on peaks detection through an adaptive threshold defined as the signal’s root mean square (RMS) or by morphological analysis through the computation of the signal’s meanwave. The final goal is achieved by dividing the distance measures into n parts and by performing k-means individually. In order to improve speed performance, parallel computing techniques were applied. For this study, a set of different types of signals was acquired and annotated by our algorithm. By visual inspection, the L1 and L2 Minkowski distances returned an output that allowed clustering signals’ cycles with an efficiency of 97:5% and 97:3%, respectively. Using the meanwave distance, our algorithm achieved an accuracy of 97:4%. For the downloaded ECGs from the Physionet databases, the developed algorithm detected 638 out of 644 manually annotated events provided by physicians. The fact that this algorithm can be applied to long-term raw biosignals and without requiring any prior information about them makes it an important contribution in biosignals’ information extraction and annotation

Human activity recognition for an intelligent knee orthosis

Dissertação para obtenção do Grau de Mestre em Engenharia BiomédicaActivity recognition with body-worn sensors is a large and growing field of research. In this thesis we evaluate the possibility to recognize human activities based on data from biosignal sensors solely placed on or under an existing passive knee orthosis, which will produce the needed information to integrate sensors into the orthosis in the future. The development of active orthotic knee devices will allow population to ambulate in a more natural, efficient and less painful manner than they might with a traditional orthosis. Thus, the term ’active orthosis’ refers to a device intended to increase the ambulatory ability of a person suffering from a knee pathology by applying forces to correct the position only when necessary and thereby make usable over longer periods of time. The contribution of this work is the evaluation of the ability to recognize activities with these restrictions on sensor placement as well as providing a proof-of-concept for the development of an activity recognition system for an intelligent orthosis. We use accelerometers and a goniometer placed on the orthosis and Electromyography (EMG) sensors placed on the skin under the orthosis to measure motion and muscle activity respectively. We segment signals in motion primitives semi-automatically and apply Hidden-Markov-Models (HMM) to classify the isolated motion primitives. We discriminate between seven activities like for example walking stairs up and ascend a hill. In a user study with six participants, we evaluate the systems performance for each of the different biosignal modalities alone as well as the combinations of them. For the best performing combination, we reach an average person-dependent accuracy of 98% and a person-independent accuracy of 79%