A Comparative Evaluation of Heart Rate Estimation Methods using Face Videos

This paper presents a comparative evaluation of methods for remote heart rate estimation using face videos, i.e., given a video sequence of the face as input, methods to process it to obtain a robust estimation of the subjects heart rate at each moment. Four alternatives from the literature are tested, three based in hand crafted approaches and one based on deep learning. The methods are compared using RGB videos from the COHFACE database. Experiments show that the learning-based method achieves much better accuracy than the hand crafted ones. The low error rate achieved by the learning based model makes possible its application in real scenarios, e.g. in medical or sports environments.Comment: Accepted in "IEEE International Workshop on Medical Computing (MediComp) 2020

Cardiovascular assessment by imaging photoplethysmography – a review

AbstractOver the last few years, the contactless acquisition of cardiovascular parameters using cameras has gained immense attention. The technique provides an optical means to acquire cardiovascular information in a very convenient way. This review provides an overview on the technique’s background and current realizations. Besides giving detailed information on the most widespread application of the technique, namely the contactless acquisition of heart rate, we outline further concepts and we critically discuss the current state.</jats:p

Remote Assessment of the Cardiovascular Function Using Camera-Based Photoplethysmography

Camera-based photoplethysmography (cbPPG) is a novel measurement technique that allows the continuous monitoring of vital signs by using common video cameras. In the last decade, the technology has attracted a lot of attention as it is easy to set up, operates remotely, and offers new diagnostic opportunities. Despite the growing interest, cbPPG is not completely established yet and is still primarily the object of research. There are a variety of reasons for this lack of development including that reliable and autonomous hardware setups are missing, that robust processing algorithms are needed, that application fields are still limited, and that it is not completely understood which physiological factors impact the captured signal. In this thesis, these issues will be addressed. A new and innovative measuring system for cbPPG was developed. In the course of three large studies conducted in clinical and non-clinical environments, the system’s great flexibility, autonomy, user-friendliness, and integrability could be successfully proven. Furthermore, it was investigated what value optical polarization filtration adds to cbPPG. The results show that a perpendicular filter setting can significantly enhance the signal quality. In addition, the performed analyses were used to draw conclusions about the origin of cbPPG signals: Blood volume changes are most likely the defining element for the signal's modulation. Besides the hardware-related topics, the software topic was addressed. A new method for the selection of regions of interest (ROIs) in cbPPG videos was developed. Choosing valid ROIs is one of the most important steps in the processing chain of cbPPG software. The new method has the advantage of being fully automated, more independent, and universally applicable. Moreover, it suppresses ballistocardiographic artifacts by utilizing a level-set-based approach. The suitability of the ROI selection method was demonstrated on a large and challenging data set. In the last part of the work, a potentially new application field for cbPPG was explored. It was investigated how cbPPG can be used to assess autonomic reactions of the nervous system at the cutaneous vasculature. The results show that changes in the vasomotor tone, i.e. vasodilation and vasoconstriction, reflect in the pulsation strength of cbPPG signals. These characteristics also shed more light on the origin problem. Similar to the polarization analyses, they support the classic blood volume theory. In conclusion, this thesis tackles relevant issues regarding the application of cbPPG. The proposed solutions pave the way for cbPPG to become an established and widely accepted technology

Remote Bio-Sensing: Open Source Benchmark Framework for Fair Evaluation of rPPG

Remote Photoplethysmography (rPPG) is a technology that utilizes the light absorption properties of hemoglobin, captured via camera, to analyze and measure blood volume pulse (BVP). By analyzing the measured BVP, various physiological signals such as heart rate, stress levels, and blood pressure can be derived, enabling applications such as the early prediction of cardiovascular diseases. rPPG is a rapidly evolving field as it allows the measurement of vital signals using camera-equipped devices without the need for additional devices such as blood pressure monitors or pulse oximeters, and without the assistance of medical experts. Despite extensive efforts and advances in this field, serious challenges remain, including issues related to skin color, camera characteristics, ambient lighting, and other sources of noise, which degrade performance accuracy. We argue that fair and evaluable benchmarking is urgently required to overcome these challenges and make any meaningful progress from both academic and commercial perspectives. In most existing work, models are trained, tested, and validated only on limited datasets. Worse still, some studies lack available code or reproducibility, making it difficult to fairly evaluate and compare performance. Therefore, the purpose of this study is to provide a benchmarking framework to evaluate various rPPG techniques across a wide range of datasets for fair evaluation and comparison, including both conventional non-deep neural network (non-DNN) and deep neural network (DNN) methods. GitHub URL: https://github.com/remotebiosensing/rppg.Comment: 19 pages, 10 figure

Spatio-temporal analysis of blood perfusion by imaging photoplethysmography

Imaging photoplethysmography (iPPG) has attracted much attention over the last years. The vast majority of works focuses on methods to reliably extract the heart rate from videos. Only a few works addressed iPPGs ability to exploit spatio-temporal perfusion pattern to derive further diagnostic statements. This work directs at the spatio-temporal analysis of blood perfusion from videos. We present a novel algorithm that bases on the two-dimensional representation of the blood pulsation (perfusion map). The basic idea behind the proposed algorithm consists of a pairwise estimation of time delays between photoplethysmographic signals of spatially separated regions. The probabilistic approach yields a parameter denoted as perfusion speed. We compare the perfusion speed versus two parameters, which assess the strength of blood pulsation (perfusion strength and signal to noise ratio). Preliminary results using video data with different physiological stimuli (cold pressure test, cold face test) show that all measures are in fluenced by those stimuli (some of them with statistical certainty). The perfusion speed turned out to be more sensitive than the other measures in some cases. However, our results also show that the intraindividual stability and interindividual comparability of all used measures remain critical points. This work proves the general feasibility of employing the perfusion speed as novel iPPG quantity. Future studies will address open points like the handling of ballistocardiographic effects and will try to deepen the understanding of the predominant physiological mechanisms and their relation to the algorithmic performance

Contactless heart rate measurement in newborn infants using a multimodal 3D camera system

Newborns and preterm infants require accurate and continuous monitoring of their vital parameters. Contact-based methods of monitoring have several disadvantages, thus, contactless systems have increasingly attracted the neonatal communities' attention. Camera-based photoplethysmography is an emerging method of contactless heart rate monitoring. We conducted a pilot study in 42 healthy newborn and near-term preterm infants for assessing the feasibility and accuracy of a multimodal 3D camera system on heart rates (HR) in beats per min (bpm) compared to conventional pulse oximetry. Simultaneously, we compared the accuracy of 2D and 3D vision on HR measurements. The mean difference in HR between pulse oximetry and 2D-technique added up to + 3.0 bpm [CI−3.7 – 9.7; p = 0.359, limits of agreement (LOA) ± 36.6]. In contrast, 3D-technique represented a mean difference in HR of + 8.6 bpm (CI 2.0–14.9; p = 0.010, LOA ± 44.7) compared to pulse oximetry HR. Both, intra- and interindividual variance of patient characteristics could be eliminated as a source for the results and the measuring accuracy achieved. Additionally, we proved the feasibility of this emerging method. Camera-based photoplethysmography seems to be a promising approach for HR measurement of newborns with adequate precision; however, further research is warranted

Video pulse rate variability analysis in stationary and motion conditions

Background: In the last few years, some studies have measured heart rate (HR) or heart rate variability (HRV) parameters using a video camera. This technique focuses on the measurement of the small changes in skin colour caused by blood perfusion. To date, most of these works have obtained HRV parameters in stationary conditions, and there are practically no studies that obtain these parameters in motion scenarios and by conducting an in-depth statistical analysis. Methods: In this study, a video pulse rate variability (PRV) analysis is conducted by measuring the pulse-to-pulse (PP) intervals in stationary and motion conditions. Firstly, given the importance of the sampling rate in a PRV analysis and the low frame rate of commercial cameras, we carried out an analysis of two models to evaluate their performance in the measurements. We propose a selective tracking method using the Viola–Jones and KLT algorithms, with the aim of carrying out a robust video PRV analysis in stationary and motion conditions. Data and results of the proposed method are contrasted with those reported in the state of the art. Results: The webcam achieved better results in the performance analysis of video cameras. In stationary conditions, high correlation values were obtained in PRV parameters with results above 0.9. The PP time series achieved an RMSE (mean ± standard deviation) of 19.45 ± 5.52 ms (1.70 ± 0.75 bpm). In the motion analysis, most of the PRV parameters also achieved good correlation results, but with lower values as regards stationary conditions. The PP time series presented an RMSE of 21.56 ± 6.41 ms (1.79 ± 0.63 bpm). Conclusions: The statistical analysis showed good agreement between the reference system and the proposed method. In stationary conditions, the results of PRV parameters were improved by our method in comparison with data reported in related works. An overall comparative analysis of PRV parameters in motion conditions was more limited due to the lack of studies or studies containing insufficient data analysis. Based on the results, the proposed method could provide a low-cost, contactless and reliable alternative for measuring HR or PRV parameters in non-clinical environments.Peer ReviewedPostprint (published version

Video-based sympathetic arousal assessment via peripheral blood flow estimation

Electrodermal activity (EDA) is considered a standard marker of sympathetic activity. However, traditional EDA measurement requires electrodes in steady contact with the skin. Can sympathetic arousal be measured using only an optical sensor, such as an RGB camera? This paper presents a novel approach to infer sympathetic arousal by measuring the peripheral blood flow on the face or hand optically. We contribute a self-recorded dataset of 21 participants, comprising synchronized videos of participants' faces and palms and gold-standard EDA and photoplethysmography (PPG) signals. Our results show that we can measure peripheral sympathetic responses that closely correlate with the ground truth EDA. We obtain median correlations of 0.57 to 0.63 between our inferred signals and the ground truth EDA using only videos of the participants' palms or foreheads or PPG signals from the foreheads or fingers. We also show that sympathetic arousal is best inferred from the forehead, finger, or palm.Comment: Accepted and to be published at Biomedical Optics Expres