Optimal color channel combination across skin tones for remote heart rate measurement in camera-based photoplethysmography

Objective: The heart rate is an essential vital sign that can be measured remotely with camera-based photoplethysmography (cbPPG). Systems for cbPPG typically use cameras that deliver red, green, and blue (RGB) channels. The combination of these channels has been proven to increase signal-to-noise ratio (SNR) and heart rate measurement accuracy (ACC). However, many combinations remain untested, the comparison of proposed combinations on large datasets is insufficiently investigated, and the interplay with skin tone is rarely addressed. Methods: Eight regions of interest and eight color spaces with a total of 25 color channels were compared in terms of ACC and SNR based on the Binghamton-Pittsburgh-RPI Multimodal Spontaneous Emotion Database (BP4D+). Additionally, two systematic grid searches were performed to evaluate ACC in the space of linear combinations of the RGB channels. Results: Glabella and forehead regions of interest provided highest ACC (up to 74.1 %) and SNR (> -3 dB) with the hue channel H from HSV color space and the chrominance channel Q from NTSC color space. The grid searches revealed a global optimum of linear RGB combinations (ACC: 79.2 %). This optimum occurred for all skin tones, although ACC dropped for darker skin tones. Conclusion: Through systematic grid searches we were able to identify the skin tone independent optimal linear RGB color combination for measuring heart rate with cbPPG. Our results proved on a large dataset that the identified optimum outperformed conventionally used color channels. Significance: The presented findings provide useful evidence for future considerations of algorithmic approaches for cbPPG

Frame registration for motion compensation in imaging photoplethysmography

© 2018 by the authors. Licensee MDPI, Basel, Switzerland. Imaging photoplethysmography (iPPG) is an emerging technology used to assess microcirculation and cardiovascular signs by collecting backscattered light from illuminated tissue using optical imaging sensors. An engineering approach is used to evaluate whether a silicone cast of a human palm might be effectively utilized to predict the results of image registration schemes for motion compensation prior to their application on live human tissue. This allows us to establish a performance baseline for each of the algorithms and to isolate performance and noise fluctuations due to the induced motion from the temporally changing physiological signs. A multi-stage evaluation model is developed to qualitatively assess the influence of the region of interest (ROI), system resolution and distance, reference frame selection, and signal normalization on extracted iPPG waveforms from live tissue. We conclude that the application of image registration is able to deliver up to 75% signal-to-noise (SNR) improvement (4.75 to 8.34) over an uncompensated iPPG signal by employing an intensity-based algorithm with a moving reference frame

Sleep detection with photoplethysmography for wearable-based health monitoring

Remote health monitoring has gained increasing attention in the recent years. Detecting sleep patterns provides users with insights on their personal health issues, and can help in the diagnosis of various sleep disorders. Conventional methods are focused on the acceleration data, or are not suitable for continuous monitoring, like the polysomnography. Wearable devices enable a way to continuously measure photoplethysmography signal. Photoplethysmography signal contains information on multiple physiological systems, and can be used to detect sleep patterns. Sleep detection using wearable-based photoplethysmography signal offers a convenient and easy way to monitor health. In this thesis, a photoplethysmography-based sleep detection method for wearable-based health monitoring is described. This technique aims to separate wakefulness and asleep states with adequate accuracy. To examine the importance of good quality data in sleep detection, the quality of the signal is assessed. The proposed method uses statistical and heart rate based features extracted from the photoplethysmography signal. Using the most relevant features, various supervised learning algorithms are trained, compared and evaluated. These algorithms are logistic regression, decision tree, random forest, support vector machine, k-nearest neighbors, and Naive Bayes. The best performance is obtained by the random forest classifier. The method received an overall accuracy of 81 percent. It was able to detect the sleep periods with 86 percent accuracy and the awake periods with 74 percent accuracy. Motion artifacts occurring during the awake time caused distortion to the signal. Features related to the shape of the signal improved the accuracy of sleep detection, since signal distortion was associated with the awake time. It is concluded that photoplethysmography signal provides a good alternative for wearable-based sleep detection. Future studies with more comprehensive sleep level analysis could be conducted to provide valuable information on the quality of sleep.Viime vuosina etänä tapahtuva terveyden seuranta on saanut yhä enemmän huomiota. Unen tunnistaminen antaa käyttäjille tietoa heidän henkilökohtaisista terveysongelmistaan ja voi auttaa erilaisten unihäiriöiden diagnosoinnissa. Tavanomaiset menetelmät käyttävät kiihtyvyyteen perustuvaa dataa, tai eivät ole soveltuvia jatkuvaan seurantaan, kuten polysomnografia. Puettavan teknologian avulla fotopletysmografiasignaalin jatkuva mittaus on mahdollista. Fotopletysmografiasignaali sisältää tietoa useista fysiologisista järjestelmistä ja sitä voidaan käyttää unen tunnistamiseen. Puettavan teknologian avulla mitatun fotopletysmografiasignaalin käyttö unen tunnistuksessa tarjoaa kätevän ja helpon tavan seurata terveyttä. Tässä diplomityössä kuvataan fotopletysmografiaan perustuva unenhavaitsemismenetelmä, joka soveltuu puettavaa teknologiaa hyödyntävään terveyden seurantaan. Tekniikalla pyritään erottamaan hereillä olo ja uni riittävän tarkasti. Signaalin laatu arvioidaan, jotta voidaan tutkia datan laadun tärkeys unen tunnistuksessa. Kehitetty menetelmä käyttää tilastollisia ja sykkeeseen perustuvia ominaisuuksia, jotka on erotettu fotopletysmografiasignaalista. Tärkeimpiä ominaisuuksia käyttämällä erilaisia valvottuja oppimisalgoritmeja koulutetaan, vertaillaan ja arvioidaan. Käytetyt algoritmit ovat logistinen regressio, päätöspuu, satunnainen metsä, tukivektorikone, k-lähimmät naapurit ja Naive Bayes. Paras tulos saadaan käyttämällä satunnainen metsä -algoritmia. Menetelmällä saavutetaan 81 prosentin kokonaistarkkuus. Uni pystytään tunnistamaan 86 prosentin tarkkuudella ja hereillä olo 74 prosentin tarkkuudella. Hereillä ollessa liikkeestä johtuvat häiriöt aiheuttavat vääristymää signaaliin. Signaalin muotoon liittyvät ominaisuudet paransivat unentunnistuksen tarkkuutta, koska signaalin vääristyminen yhdistettiin hereilläoloaikaan. Tutkimuksen tuloksista voidaan tehdä johtopäätös, että fotopletysmografiasignaali tarjoaa hyvän vaihtoehdon puettavaa teknologiaa hyödyntävään unen tunnistamiseen. Tulevaisuudessa unen eri vaiheita voitaisiin tutkia kattavammin, jolloin saataisiin arvokasta tietoa unen laadusta

Photoplethysmography based remote health monitoring system

One of the world's most leading killer diseases is the cardiovascular disease, which accounts for 16.7 million deaths annually. Out of the total population in the world, about 22 million people run the risk of sudden heart failure. However, saving the lives of cardiac patients can be improved by the emergency monitoring so that the initiation of treatment can be taken up within the crucial hour. The acquired signals by pulse oximetry provide significant information about the heart-rate, arterial blood oxygenation, blood pressure and respiratory-rate. Telemedicine provides a great impact in the emergency monitoring of patients located in remote nonclinical environments. A home cardiac telemedicine emergency system based on photoplethysmography has been developed. The acquired signals are processed, transmitted and stored in a local PC. Finally, the data are sent to the remote terminal located at the hospital through internet. The diagnoses are done by specialists from the reading and action can be immediately taken in emergency cases

Remote Bio-Sensing: Open Source Benchmark Framework for Fair Evaluation of rPPG

Remote Photoplethysmography (rPPG) is a technology that utilizes the light absorption properties of hemoglobin, captured via camera, to analyze and measure blood volume pulse (BVP). By analyzing the measured BVP, various physiological signals such as heart rate, stress levels, and blood pressure can be derived, enabling applications such as the early prediction of cardiovascular diseases. rPPG is a rapidly evolving field as it allows the measurement of vital signals using camera-equipped devices without the need for additional devices such as blood pressure monitors or pulse oximeters, and without the assistance of medical experts. Despite extensive efforts and advances in this field, serious challenges remain, including issues related to skin color, camera characteristics, ambient lighting, and other sources of noise, which degrade performance accuracy. We argue that fair and evaluable benchmarking is urgently required to overcome these challenges and make any meaningful progress from both academic and commercial perspectives. In most existing work, models are trained, tested, and validated only on limited datasets. Worse still, some studies lack available code or reproducibility, making it difficult to fairly evaluate and compare performance. Therefore, the purpose of this study is to provide a benchmarking framework to evaluate various rPPG techniques across a wide range of datasets for fair evaluation and comparison, including both conventional non-deep neural network (non-DNN) and deep neural network (DNN) methods. GitHub URL: https://github.com/remotebiosensing/rppg.Comment: 19 pages, 10 figure

Respiratory Rate Estimation from Face Videos

Vital signs, such as heart rate (HR), heart rate variability (HRV), respiratory rate (RR), are important indicators for a person's health. Vital signs are traditionally measured with contact sensors, and may be inconvenient and cause discomfort during continuous monitoring. Commercial cameras are promising contact-free sensors, and remote photoplethysmography (rPPG) have been studied to remotely monitor heart rate from face videos. For remote RR measurement, most prior art was based on small periodical motions of chest regions caused by breathing cycles, which are vulnerable to subjects' voluntary movements. This paper explores remote RR measurement based on rPPG obtained from face videos. The paper employs motion compensation, two-phase temporal filtering, and signal pruning to capture signals with high quality. The experimental results demonstrate that the proposed framework can obtain accurate RR results and can provide HR, HRV and RR measurement synergistically in one framework