Multimodal emotion recognition based on the fusion of vision, EEG, ECG, and EMG signals

This paper presents a novel approach for emotion recognition (ER) based on Electroencephalogram (EEG), Electromyogram (EMG), Electrocardiogram (ECG), and computer vision. The proposed system includes two different models for physiological signals and facial expressions deployed in a real-time embedded system. A custom dataset for EEG, ECG, EMG, and facial expression was collected from 10 participants using an Affective Video Response System. Time, frequency, and wavelet domain-specific features were extracted and optimized, based on their Visualizations from Exploratory Data Analysis (EDA) and Principal Component Analysis (PCA). Local Binary Patterns (LBP), Local Ternary Patterns (LTP), Histogram of Oriented Gradients (HOG), and Gabor descriptors were used for differentiating facial emotions. Classification models, namely decision tree, random forest, and optimized variants thereof, were trained using these features. The optimized Random Forest model achieved an accuracy of 84%, while the optimized Decision Tree achieved 76% for the physiological signal-based model. The facial emotion recognition (FER) model attained an accuracy of 84.6%, 74.3%, 67%, and 64.5% using K-Nearest Neighbors (KNN), Random Forest, Decision Tree, and XGBoost, respectively. Performance metrics, including Area Under Curve (AUC), F1 score, and Receiver Operating Characteristic Curve (ROC), were computed to evaluate the models. The outcome of both results, i.e., the fusion of bio-signals and facial emotion analysis, is given to a voting classifier to get the final emotion. A comprehensive report is generated using the Generative Pretrained Transformer (GPT) language model based on the resultant emotion, achieving an accuracy of 87.5%. The model was implemented and deployed on a Jetson Nano. The results show its relevance to ER. It has applications in enhancing prosthetic systems and other medical fields such as psychological therapy, rehabilitation, assisting individuals with neurological disorders, mental health monitoring, and biometric security

Classification of breast tissue in mammograms using efficient coding

<p>Abstract</p> <p>Background</p> <p>Female breast cancer is the major cause of death by cancer in western countries. Efforts in Computer Vision have been made in order to improve the diagnostic accuracy by radiologists. Some methods of lesion diagnosis in mammogram images were developed based in the technique of principal component analysis which has been used in efficient coding of signals and 2D Gabor wavelets used for computer vision applications and modeling biological vision.</p> <p>Methods</p> <p>In this work, we present a methodology that uses efficient coding along with linear discriminant analysis to distinguish between mass and non-mass from 5090 region of interest from mammograms.</p> <p>Results</p> <p>The results show that the best rates of success reached with Gabor wavelets and principal component analysis were 85.28% and 87.28%, respectively. In comparison, the model of efficient coding presented here reached up to 90.07%.</p> <p>Conclusions</p> <p>Altogether, the results presented demonstrate that independent component analysis performed successfully the efficient coding in order to discriminate mass from non-mass tissues. In addition, we have observed that LDA with ICA bases showed high predictive performance for some datasets and thus provide significant support for a more detailed clinical investigation.</p

Application of artificial intelligence techniques for automated detection of myocardial infarction: A review

Myocardial infarction (MI) results in heart muscle injury due to receiving insufficient blood flow. MI is the most common cause of mortality in middle-aged and elderly individuals around the world. To diagnose MI, clinicians need to interpret electrocardiography (ECG) signals, which requires expertise and is subject to observer bias. Artificial intelligence-based methods can be utilized to screen for or diagnose MI automatically using ECG signals. In this work, we conducted a comprehensive assessment of artificial intelligence-based approaches for MI detection based on ECG as well as other biophysical signals, including machine learning (ML) and deep learning (DL) models. The performance of traditional ML methods relies on handcrafted features and manual selection of ECG signals, whereas DL models can automate these tasks. The review observed that deep convolutional neural networks (DCNNs) yielded excellent classification performance for MI diagnosis, which explains why they have become prevalent in recent years. To our knowledge, this is the first comprehensive survey of artificial intelligence techniques employed for MI diagnosis using ECG and other biophysical signals.Comment: 16 pages, 8 figure

CES-513 Stages for Developing Control Systems using EMG and EEG Signals: A survey

Bio-signals such as EMG (Electromyography), EEG (Electroencephalography), EOG (Electrooculogram), ECG (Electrocardiogram) have been deployed recently to develop control systems for improving the quality of life of disabled and elderly people. This technical report aims to review the current deployment of these state of the art control systems and explain some challenge issues. In particular, the stages for developing EMG and EEG based control systems are categorized, namely data acquisition, data segmentation, feature extraction, classification, and controller. Some related Bio-control applications are outlined. Finally a brief conclusion is summarized.

Towards an accurate sleep apnea detection based on ECG signal: The quintessential of a wise feature selection

A wise feature selection from minute-to-minute Electrocardiogram (ECG) signal is a challenging task for many reasons, but mostly because of the promise of the accurate detection of clinical disorders, such as the sleep apnea. In this study, the ECG signal was modeled in order to obtain the Heart Rate Variability (HRV) and the ECG-Derived Respiration (EDR). Selected features techniques were used for benchmark with different classifiers such as Artificial Neural Networks (ANN) and Support Vector Machine(SVM), among others. The results evidence that the best accuracy was 82.12%, with a sensitivity and specificity of 88.41% and 72.29%, respectively. In addition, experiments revealed that a wise feature selection may improve the system accuracy. Therefore, the proposed model revealed to be reliable and simpler alternative to classical solutions for the sleep apnea detection, for example the ones based on the Polysomnography.info:eu-repo/semantics/publishedVersio

A real time classification algorithm for EEG-based BCI driven by self-induced emotions

Background and objective: The aim of this paper is to provide an efficient, parametric, general, and completely automatic real time classification method of electroencephalography (EEG) signals obtained from self-induced emotions. The particular characteristics of the considered low-amplitude signals (a self-induced emotion produces a signal whose amplitude is about 15% of a really experienced emotion) require exploring and adapting strategies like the Wavelet Transform, the Principal Component Analysis (PCA) and the Support Vector Machine (SVM) for signal processing, analysis and classification. Moreover, the method is thought to be used in a multi-emotions based Brain Computer Interface (BCI) and, for this reason, an ad hoc shrewdness is assumed. Method: The peculiarity of the brain activation requires ad-hoc signal processing by wavelet decomposition, and the definition of a set of features for signal characterization in order to discriminate different self-induced emotions. The proposed method is a two stages algorithm, completely parameterized, aiming at a multi-class classification and may be considered in the framework of machine learning. The first stage, the calibration, is off-line and is devoted at the signal processing, the determination of the features and at the training of a classifier. The second stage, the real-time one, is the test on new data. The PCA theory is applied to avoid redundancy in the set of features whereas the classification of the selected features, and therefore of the signals, is obtained by the SVM. Results: Some experimental tests have been conducted on EEG signals proposing a binary BCI, based on the self-induced disgust produced by remembering an unpleasant odor. Since in literature it has been shown that this emotion mainly involves the right hemisphere and in particular the T8 channel, the classification procedure is tested by using just T8, though the average accuracy is calculated and reported also for the whole set of the measured channels. Conclusions: The obtained classification results are encouraging with percentage of success that is, in the average for the whole set of the examined subjects, above 90%. An ongoing work is the application of the proposed procedure to map a large set of emotions with EEG and to establish the EEG headset with the minimal number of channels to allow the recognition of a significant range of emotions both in the field of affective computing and in the development of auxiliary communication tools for subjects affected by severe disabilities

Detecció automàtica i robusta de Bursts en EEG de nounats amb HIE. Enfocament tensorial

[ANGLÈS] Hypoxic-Ischemic Encephalopathy (HIE) is an important cause of brain injury in the newborn, and can result in long-term devastating consequences. Burst-suppression pattern is one of several indicators of severe pathology in the EEG signal that may occur after brain damage caused by e.g. asphyxia around the time of birth. The goal of this thesis is to design a robust method to detect burst patterns automatically regardless of the physiologic and extra-physiologic artifacts that may occur at any time. At first, a pre-detector has been designed to obtain potential burst candidates from different patients. Then, a post-classification has been implemented, applying high dimensional feature extraction methods, to get the real burst patterns from these patients with a high sensitivity.[CASTELLÀ] La Hipoxia-Isquemia Encefálica (HIE) es una causa importante de lesión cerebral en los recién nacidos, pudiendo acarrear devastadoras consecuencias a largo plazo. El patrón Burst-Suppression es uno de los indicadores dados en patologías severas en señales EEG los cuales ocurren después de una lesión cerebral causada, por ejemplo, por una asfixia poco después del nacimiento. El objetivo de esta tésis es diseñar un método robusto que detecte automáticamente patrones Burst, prescindiendo de los artefactos fisiológicos y extra-fisiológicos que puedan aparecer en cualquier momento. Primeramente, se ha diseñado un pre-detector para obtener los candidatos potenciales a Burst provenientes de diferentes pacientes. Seguidamente, se ha implementado una post-clasificación, aplicando métodos de extracción de características para altas dimensiones, para obtener patrones reales de Burst con una alta sensitividad.[CATALÀ] La Hipòxia-Isquèmia Encefàlica (HIE) és una causa important de lesió cerebral en nounats, que poden comportar devastadores conseqüències a llarg termini. El patró Burst-Suppression és un dels indicadors donats en patologies severes en els senyals EEG els quals ocorren després d'una lesió cerebral causada, per exemple, per una asfixia poc després del naixement. L'objectiu d'aquesta tesis és dissenyar un mètode robust que detecti automàticament patrons Burst, prescindint dels artefactes fisiològics i extra-fisiològics que poden aparèixer en qualsevol moment. Primerament, s'ha dissenyat un pre-detector per obtenir els candidats potencials a Burst provinents de diferents pacients. Seguidament, s'ha implementat una post-classificació, aplicant mètodes d'extracció de característiques per a altes dimensions, per tal d'obtenir patrons reals de Burst amb una alta sensitivitat

The effect of electronic word of mouth communication on purchase intention moderate by trust: a case online consumer of Bahawalpur Pakistan

The aim of this study is concerned with improving the previous research finding complete filling the research gaps and introducing the e-WOM on purchase intention and brand trust as a moderator between the e-WOM, and purchase intention an online user in Bahawalpur city Pakistan, therefore this study was a focus at linking the research gap of previous literature of past study based on individual awareness from the real-life experience. we collected data from the online user of the Bahawalpur Pakistan. In this study convenience sampling has been used to collect data and instruments of this study adopted from the previous study. The quantitative research methodology used to collect data, survey method was used to assemble data for this study, 300 questionnaire were distributed in Bahawalpur City due to the ease, reliability, and simplicity, effective recovery rate of 67% as a result 202 valid response was obtained for the effect of e-WOM on purchase intention and moderator analysis has been performed. Hypotheses of this research are analyzed by using Structural Equation Modeling (SEM) based on Partial Least Square (PLS). The result of this research is e-WOM significantly positive effect on purchase intention and moderator role of trust significantly affects the relationship between e-WOM, and purchase intention. The addition of brand trust in the model has contributed to the explanatory power, some studied was conduct on brand trust as a moderator and this study has contributed to the literature in this favor. significantly this study focused on current marketing research. Unlike past studies focused on western context, this study has extended the regional literature on e-WOM, and purchase intention to be intergrading in Bahawalpur Pakistan context. Lastly, future studies are recommended to examine the effect of trust in other countries allow for the comparison of the findings