Deep learning for healthcare applications based on physiological signals: A review

Background and objective: We have cast the net into the ocean of knowledge to retrieve the latest scientific research on deep learning methods for physiological signals. We found 53 research papers on this topic, published from 01.01.2008 to 31.12.2017. Methods: An initial bibliometric analysis shows that the reviewed papers focused on Electromyogram(EMG), Electroencephalogram(EEG), Electrocardiogram(ECG), and Electrooculogram(EOG). These four categories were used to structure the subsequent content review. Results: During the content review, we understood that deep learning performs better for big and varied datasets than classic analysis and machine classification methods. Deep learning algorithms try to develop the model by using all the available input. Conclusions: This review paper depicts the application of various deep learning algorithms used till recently, but in future it will be used for more healthcare areas to improve the quality of diagnosi

EMERALD—Exercise Monitoring Emotional Assistant

The increase in the elderly population in today’s society entails the need for new policies to maintain an adequate level of care without excessively increasing social spending. One of the possible options is to promote home care for the elderly. In this sense, this paper introduces a personal assistant designed to help elderly people in their activities of daily living. This system, called EMERALD, is comprised of a sensing platform and different mechanisms for emotion detection and decision-making that combined produces a cognitive assistant that engages users in Active Aging. The contribution of the paper is twofold—on the one hand, the integration of low-cost sensors that among other characteristics allows for detecting the emotional state of the user at an affordable cost; on the other hand, an automatic activity suggestion module that engages the users, mainly oriented to the elderly, in a healthy lifestyle. Moreover, by continuously correcting the system using the on-line monitoring carried out through the sensors integrated in the system, the system is personalized, and, in broad terms, emotionally intelligent. A functional prototype is being currently tested in a daycare centre in the northern area of Portugal where preliminary tests show positive results.This research was partially funded by the Fundacao para a Ciencia e Tecnologia (FCT) within the projects UID/CEC/00319/2019 and Post-Doc Grant SFRH/BPD/102696/2014 (Angelo Costa). This work is also partially funded by the MINECO/FEDER TIN2015-65515-C4-1-R and RISEWISE (RISEWomen with disabilities In Social Engagement) EU project under Agreement No. 690874.info:eu-repo/semantics/publishedVersio

Multimodal emotion recognition based on the fusion of vision, EEG, ECG, and EMG signals

This paper presents a novel approach for emotion recognition (ER) based on Electroencephalogram (EEG), Electromyogram (EMG), Electrocardiogram (ECG), and computer vision. The proposed system includes two different models for physiological signals and facial expressions deployed in a real-time embedded system. A custom dataset for EEG, ECG, EMG, and facial expression was collected from 10 participants using an Affective Video Response System. Time, frequency, and wavelet domain-specific features were extracted and optimized, based on their Visualizations from Exploratory Data Analysis (EDA) and Principal Component Analysis (PCA). Local Binary Patterns (LBP), Local Ternary Patterns (LTP), Histogram of Oriented Gradients (HOG), and Gabor descriptors were used for differentiating facial emotions. Classification models, namely decision tree, random forest, and optimized variants thereof, were trained using these features. The optimized Random Forest model achieved an accuracy of 84%, while the optimized Decision Tree achieved 76% for the physiological signal-based model. The facial emotion recognition (FER) model attained an accuracy of 84.6%, 74.3%, 67%, and 64.5% using K-Nearest Neighbors (KNN), Random Forest, Decision Tree, and XGBoost, respectively. Performance metrics, including Area Under Curve (AUC), F1 score, and Receiver Operating Characteristic Curve (ROC), were computed to evaluate the models. The outcome of both results, i.e., the fusion of bio-signals and facial emotion analysis, is given to a voting classifier to get the final emotion. A comprehensive report is generated using the Generative Pretrained Transformer (GPT) language model based on the resultant emotion, achieving an accuracy of 87.5%. The model was implemented and deployed on a Jetson Nano. The results show its relevance to ER. It has applications in enhancing prosthetic systems and other medical fields such as psychological therapy, rehabilitation, assisting individuals with neurological disorders, mental health monitoring, and biometric security

Physiological-based Driver Monitoring Systems: A Scoping Review

A physiological-based driver monitoring system (DMS) has attracted research interest and has great potential for providing more accurate and reliable monitoring of the driver’s state during a driving experience. Many driving monitoring systems are driver behavior-based or vehicle-based. When these non-physiological based DMS are coupled with physiological-based data analysis from electroencephalography (EEG), electrooculography (EOG), electrocardiography (ECG), and electromyography (EMG), the physical and emotional state of the driver may also be assessed. Drivers’ wellness can also be monitored, and hence, traffic collisions can be avoided. This paper highlights work that has been published in the past five years related to physiological-based DMS. Specifically, we focused on the physiological indicators applied in DMS design and development. Work utilizing key physiological indicators related to driver identification, driver alertness, driver drowsiness, driver fatigue, and drunk driver is identified and described based on the PRISMA Extension for Scoping Reviews (PRISMA-Sc) Framework. The relationship between selected papers is visualized using keyword co-occurrence. Findings were presented using a narrative review approach based on classifications of DMS. Finally, the challenges of physiological-based DMS are highlighted in the conclusion. Doi: 10.28991/CEJ-2022-08-12-020 Full Text: PD

Euclidean Distance Distortion Based Robust and Blind Mesh Watermarking

The three-dimensional (3D) polygonal meshes are recently widely used in several domains, which necessitate the realistic visualization of the objects. Moreover, there is an urgent need to protect the 3D data properties for preventing unauthorized reproduction. The 3D digital watermarking technology is one of the best solutions to protect data from piracy during transmission through the internet. The current work proposed a novel robust watermarking scheme of polygonal meshes for copyright protection purposes. The proposed algorithm is based on the characteristics of the mesh geometry to embed a sequence of data bits into the object by slightly adjusting the vertex positions. Furthermore, the proposed method used a blind detection scheme. The watermarked model is perceptually indistinguishable from the original one and the embedded watermark is invariant to affine transformation. Through simulations, the quality of the watermarked object as well as the inserted watermark robustness against various types of attacks were tested and evaluated to prove the validity and the efficiency of our algorithm

Heart rate monitoring using human speech spectral features

This paper attempts to establish a correlation between the human speech, emotions and human heart rate. The study highlights a possible contactless human heart rate measurement technique useful for monitoring of patient condition from realtime speech recordings. The distance between the average peak-to-peak distances in speech Mel-frequency cepstral coefficients are used as the speech features. The features when tested on 20 classifiers from the data collected from 30 subjects indicate a non-separable classification problem, however, the classification accuracies indicate the existence of strong correlation between the human speech, emotion and heart-rates

Seamless Multimodal Biometrics for Continuous Personalised Wellbeing Monitoring

Artificially intelligent perception is increasingly present in the lives of every one of us. Vehicles are no exception, (...) In the near future, pattern recognition will have an even stronger role in vehicles, as self-driving cars will require automated ways to understand what is happening around (and within) them and act accordingly. (...) This doctoral work focused on advancing in-vehicle sensing through the research of novel computer vision and pattern recognition methodologies for both biometrics and wellbeing monitoring. The main focus has been on electrocardiogram (ECG) biometrics, a trait well-known for its potential for seamless driver monitoring. Major efforts were devoted to achieving improved performance in identification and identity verification in off-the-person scenarios, well-known for increased noise and variability. Here, end-to-end deep learning ECG biometric solutions were proposed and important topics were addressed such as cross-database and long-term performance, waveform relevance through explainability, and interlead conversion. Face biometrics, a natural complement to the ECG in seamless unconstrained scenarios, was also studied in this work. The open challenges of masked face recognition and interpretability in biometrics were tackled in an effort to evolve towards algorithms that are more transparent, trustworthy, and robust to significant occlusions. Within the topic of wellbeing monitoring, improved solutions to multimodal emotion recognition in groups of people and activity/violence recognition in in-vehicle scenarios were proposed. At last, we also proposed a novel way to learn template security within end-to-end models, dismissing additional separate encryption processes, and a self-supervised learning approach tailored to sequential data, in order to ensure data security and optimal performance. (...)Comment: Doctoral thesis presented and approved on the 21st of December 2022 to the University of Port

VIRHUS: uma plataforma computacional para a simulação de sinais fisiológicos de humanos virtuais

The ability to access bio-signals of participants for research activity is limited specially for informal settings like academic projects. These limitations can be in part overcome it by using software to simulated good enough physiological data. In this work we propose and develop a computational platform to simulate (biological signals of ) virtual humans as a service. The system adopts the concept of digital twin to structure the simulation processes. In this case, the system is not sensing real participants, rather uses pre-recorded signals as inputs to auto-encoders that generate realistic synthetic signal for a virtual human, i.e., a digital twin. The pre-recorded signals used were the electrocardiogram, electrodermal activity and electromyography signals which were labeled with the ongoing emotion. The system, VIRHUS, offers an interactive web environment to create the required virtual humans and manage the simulation processes. A scalable backend takes care of the asynchronous generation of signals, that can be streamed to endpoints (and consumed by external applications) or exported as files, for convenience. As a proof of concept, the “virtual human” data can be parameterized to include emotional traits in the bio-signals (happy, sad,...), generating meaningful variations in data for applications developers.A capacidade de aceder aos biossinais de participantes para actividades de investigação é limitada, especialmente em contextos informais, tais como projectos académicos. Estas limitações podem ser parcialmente ultrapassadas através da utilização de software para simular dados fisiológicos suficientemente fidedignos. Neste trabalho, propomos e desenvolvemos uma plataforma computacional para simular (sinais biológicos de ) seres humanos virtuais como um serviço. O sistema adopta o conceito de réplica digital ("digital twin") para estruturar os processos de simulação. Neste caso, o sistema não está a monitorar participantes reais, mas utiliza sinais pré-gravados como entradas para autocodificadores que geram um sinal sintético realista para um ser humano virtual, ou seja, uma réplica digital. Os sinais pré-gravados utilizados foram o electrocardiograma, a actividade electrodérmica e os sinais electromiográficos que foram marcados com a emoção em progresso. O sistema, VIRHUS, fornece um ambiente web interactivo para criar os seres humanos virtuais necessários e gerir os processos de simulação. Um backend escalável cuida da geração assíncrona de sinais, que podem ser transmitidos para pontos de acesso programático (e consumidos por aplicações externas) ou exportados como ficheiros por conveniência. Como prova de conceito, os dados "humanos virtuais" podem ser parametrizados para incluir traços emocionais nos biossinais (feliz, triste,...), gerando variações significativas nos dados para os programadores de aplicações.Mestrado em Engenharia Informátic

An Empirical Study Comparing Unobtrusive Physiological Sensors for Stress Detection in Computer Work.

Several unobtrusive sensors have been tested in studies to capture physiological reactions to stress in workplace settings. Lab studies tend to focus on assessing sensors during a specific computer task, while in situ studies tend to offer a generalized view of sensors' efficacy for workplace stress monitoring, without discriminating different tasks. Given the variation in workplace computer activities, this study investigates the efficacy of unobtrusive sensors for stress measurement across a variety of tasks. We present a comparison of five physiological measurements obtained in a lab experiment, where participants completed six different computer tasks, while we measured their stress levels using a chest-band (ECG, respiration), a wristband (PPG and EDA), and an emerging thermal imaging method (perinasal perspiration). We found that thermal imaging can detect increased stress for most participants across all tasks, while wrist and chest sensors were less generalizable across tasks and participants. We summarize the costs and benefits of each sensor stream, and show how some computer use scenarios present usability and reliability challenges for stress monitoring with certain physiological sensors. We provide recommendations for researchers and system builders for measuring stress with physiological sensors during workplace computer use