Multi-hierarchical Convolutional Network for Efficient Remote Photoplethysmograph Signal and Heart Rate Estimation from Face Video Clips

Heart beat rhythm and heart rate (HR) are important physiological parameters of the human body. This study presents an efficient multi-hierarchical spatio-temporal convolutional network that can quickly estimate remote physiological (rPPG) signal and HR from face video clips. First, the facial color distribution characteristics are extracted using a low-level face feature Generation (LFFG) module. Then, the three-dimensional (3D) spatio-temporal stack convolution module (STSC) and multi-hierarchical feature fusion module (MHFF) are used to strengthen the spatio-temporal correlation of multi-channel features. In the MHFF, sparse optical flow is used to capture the tiny motion information of faces between frames and generate a self-adaptive region of interest (ROI) skin mask. Finally, the signal prediction module (SP) is used to extract the estimated rPPG signal. The experimental results on the three datasets show that the proposed network outperforms the state-of-the-art methods.Comment: 33 pages,9 figure

Extracting Fetal ECG from a Single Maternal Abdominal Record

This work presents the variations of photoplethysmogram (ECG) morphology with age. ECG measurement is done noninvasively at the index finger on both right and left hands for a sample of erectile dysfunction (ED) subjects. Some parameters are derived from the analysis of ECG contour showed in association with age. The age is found to be an important factor that affects the contour of ECG signals which accelerates the disappearance of ECG’s dicrotic notch and ECG’s inflection point as well. Arterial compliance is found to be degraded with age due to the fall of arterial elasticity. This study approaches the establishment of usefulness of ECG’s contour analysis as an investigator to the changes in the elastic properties of the vascular system, and as a detector of early sub-clinical atherosclerosis

Sources of inaccuracy in photoplethysmography for continuous cardiovascular monitoring

Photoplethysmography (PPG) is a low-cost, noninvasive optical technique that uses change in light transmission with changes in blood volume within tissue to provide information for cardiovascular health and fitness. As remote health and wearable medical devices become more prevalent, PPG devices are being developed as part of wearable systems to monitor parameters such as heart rate (HR) that do not require complex analysis of the PPG waveform. However, complex analyses of the PPG waveform yield valuable clinical information, such as: blood pressure, respiratory information, sympathetic nervous system activity, and heart rate variability. Systems aiming to derive such complex parameters do not always account for realistic sources of noise, as testing is performed within controlled parameter spaces. A wearable monitoring tool to be used beyond fitness and heart rate must account for noise sources originating from individual patient variations (e.g., skin tone, obesity, age, and gender), physiology (e.g., respiration, venous pulsation, body site of measurement, and body temperature), and external perturbations of the device itself (e.g., motion artifact, ambient light, and applied pressure to the skin). Here, we present a comprehensive review of the literature that aims to summarize these noise sources for future PPG device development for use in health monitoring

A Novel Deep Learning Technique for Morphology Preserved Fetal ECG Extraction from Mother ECG using 1D-CycleGAN

Monitoring the electrical pulse of fetal heart through a non-invasive fetal electrocardiogram (fECG) can easily detect abnormalities in the developing heart to significantly reduce the infant mortality rate and post-natal complications. Due to the overlapping of maternal and fetal R-peaks, the low amplitude of the fECG, systematic and ambient noises, typical signal extraction methods, such as adaptive filters, independent component analysis, empirical mode decomposition, etc., are unable to produce satisfactory fECG. While some techniques can produce accurate QRS waves, they often ignore other important aspects of the ECG. Our approach, which is based on 1D CycleGAN, can reconstruct the fECG signal from the mECG signal while maintaining the morphology due to extensive preprocessing and appropriate framework. The performance of our solution was evaluated by combining two available datasets from Physionet, "Abdominal and Direct Fetal ECG Database" and "Fetal electrocardiograms, direct and abdominal with reference heartbeat annotations", where it achieved an average PCC and Spectral-Correlation score of 88.4% and 89.4%, respectively. It detects the fQRS of the signal with accuracy, precision, recall and F1 score of 92.6%, 97.6%, 94.8% and 96.4%, respectively. It can also accurately produce the estimation of fetal heart rate and R-R interval with an error of 0.25% and 0.27%, respectively. The main contribution of our work is that, unlike similar studies, it can retain the morphology of the ECG signal with high fidelity. The accuracy of our solution for fetal heart rate and R-R interval length is comparable to existing state-of-the-art techniques. This makes it a highly effective tool for early diagnosis of fetal heart diseases and regular health checkups of the fetus.Comment: 24 pages, 11 figure

The effects of optical sensor-tissue separation in endocavitary photoplethysmography

Objective: Intestinal anastomotic failure that occurs mainly due to ischaemia is a serious risk in colorectal cancer patients undergoing surgery. Surgeons continue to rely on subjective methods such as visual inspection to assess intestinal viability during surgery and there are no clinical tools to directly monitor viability postoperatively. A dual-wavelength reflectance optical sensor has been developed for continuous and dynamic monitoring of intestinal viability via the intestinal lumen. Maintaining direct contact between the sensor and the inner intestinal wall can be difficult in an intraluminal design, therefore impacting on signal acquisition and quality. This paper investigates the effect of direct contact versus variable distances between the sensor and the tissue surface of the buccal mucosa as a surrogate. Approach: The in vivo study involved 20 healthy volunteers to measure the effect of optical sensor-tissue distances on the ability to acquire photoplethysmography signals and their quality. Signals were acquired from the buccal mucosa at five optical sensor-tissue distances. Main results: Distances between 0 mm (contact) to 5 mm were the most optimal, producing signals of high quality and signal-to-noise ratio, resulting in reliable estimations of the blood oxygen saturation. Distances exceeding 5 mm compromised the acquired signals, and were of poor quality, thereby unreliably estimating the blood oxygen saturation. Significance: The developed optical sensor proved to be reliable for acquiring photoplethysmography signals for cases where distances between the optical sensor-tissue may arise during the assessment of intraluminal intestinal viability

Heart rates estimation using rPPG methods in challenging imaging conditions

Abstract. The cardiovascular system plays a crucial role in maintaining the body’s equilibrium by regulating blood flow and oxygen supply to different organs and tissues. While contact-based techniques like electrocardiography and photoplethysmography are commonly used in healthcare and clinical monitoring, they are not practical for everyday use due to their skin contact requirements. Therefore, non-contact alternatives like remote photoplethysmography (rPPG) have gained significant attention in recent years. However, extracting accurate heart rate information from rPPG signals under challenging imaging conditions, such as image degradation and occlusion, remains a significant challenge. Therefore, this thesis aims to investigate the effectiveness of rPPG methods in extracting heart rate information from rPPG signals in these imaging conditions. It evaluates the effectiveness of both traditional rPPG approaches and rPPG pre-trained deep learning models in the presence of real-world image transformations, such as occlusion of the faces by sunglasses or facemasks, as well as image degradation caused by noise artifacts and motion blur. The study also explores various image restoration techniques to enhance the performance of the selected rPPG methods and experiments with various fine-tuning methods of the best-performing pre-trained model. The research was conducted on three databases, namely UBFC-rPPG, UCLA-rPPG, and UBFC-Phys, and includes comprehensive experiments. The results of this study offer valuable insights into the efficacy of rPPG in practical scenarios and its potential as a non-contact alternative to traditional cardiovascular monitoring techniques

Study of stress detection and proposal of stress-related features using commercial-off-the-shelf wrist wearables

This paper discusses the possibility of detecting personal stress making use of popular wearable devices available in the market. Different instruments found in the literature to measure stress-related features are reviewed, distinguishing between subjective tests and mechanisms supported by the analysis of physiological signals from clinical devices. Taking them as a reference, a solution to estimate stress based on the use of commercial-off-the-shelf wrist wearables and machine learning techniques is described. A mobile app was developed to induce stress in a uniform and systematic way. The app implements well-known stress inducers, such as the Paced Auditory Serial Addition Test, the Stroop Color-Word Interference Test, and a hyperventilation activity. Wearables are used to collect physiological data used to train classifiers that provide estimations on personal stress levels. The solution has been validated in an experiment involving 19 subjects, offering an average accuracy and F-measures close to 0.99 in an individual model and an accuracy and F-measure close to 0.85 in a global 2-level classifier model. Stress can be a worrying problem in different scenarios, such as in educational settings. Thus, the last part of the paper describes the proposal of a set of stress related indicators aimed to support the management of stress over time in such settings.Agencia Estatal de Investigación | Ref. TIN2016-80515-RUniversidade de Vig

Deep learning-based signal processing approaches for improved tracking of human health and behaviour with wearable sensors

This thesis explores two lines of research in the context of sequential data and machine learning in the remote environment, i.e., outside the lab setting - using data acquired from wearable devices. Firstly, we explore Generative Adversarial Networks (GANs) as a reliable tool for time series generation, imputation and forecasting. Secondly, we investigate the applicability of novel deep learning frameworks to sequential data processing and their advantages over traditional methods. More specifically, we use our models to unlock additional insights and biomarkers in human-centric datasets. Our first research avenue concerns the generation of sequential physiological data. Access to physiological data, particularly medical data, has become heavily regulated in recent years, which has presented bottlenecks in developing computational models to assist in diagnosing and treating patients. Therefore, we explore GAN models to generate medical time series data that adhere to privacy-preserving regulations. We present our novel methods of generating and imputing synthetic, multichannel sequential medical data while complying with privacy regulations. Addressing these concerns allows for sharing and disseminating medical data and, in turn, developing clinical research in the relevant fields. Secondly, we explore novel deep learning technologies applied to human-centric sequential data to unlock further insights while addressing the idea of environmentally sustainable AI. We develop novel deep learning processing methods to estimate human activity and heart rate through convolutional networks. We also introduce our ‘time series-to-time series GAN’, which maps photoplethysmograph data to blood pressure measurements. Importantly, we denoise artefact-laden biosignal data to a competitive standard using a custom objective function and novel application of GANs. These deep learning methods help to produce nuanced biomarkers and state-of-the-art insights from human physiological data. The work laid out in this thesis provides a foundation for state-of-the-art deep learning methods for sequential data processing while keeping a keen eye on sustain- able AI

Advanced Signal Processing in Wearable Sensors for Health Monitoring

Smart, wearables devices on a miniature scale are becoming increasingly widely available, typically in the form of smart watches and other connected devices. Consequently, devices to assist in measurements such as electroencephalography (EEG), electrocardiogram (ECG), electromyography (EMG), blood pressure (BP), photoplethysmography (PPG), heart rhythm, respiration rate, apnoea, and motion detection are becoming more available, and play a significant role in healthcare monitoring. The industry is placing great emphasis on making these devices and technologies available on smart devices such as phones and watches. Such measurements are clinically and scientifically useful for real-time monitoring, long-term care, and diagnosis and therapeutic techniques. However, a pertaining issue is that recorded data are usually noisy, contain many artefacts, and are affected by external factors such as movements and physical conditions. In order to obtain accurate and meaningful indicators, the signal has to be processed and conditioned such that the measurements are accurate and free from noise and disturbances. In this context, many researchers have utilized recent technological advances in wearable sensors and signal processing to develop smart and accurate wearable devices for clinical applications. The processing and analysis of physiological signals is a key issue for these smart wearable devices. Consequently, ongoing work in this field of study includes research on filtration, quality checking, signal transformation and decomposition, feature extraction and, most recently, machine learning-based methods