Frame registration for motion compensation in imaging photoplethysmography

© 2018 by the authors. Licensee MDPI, Basel, Switzerland. Imaging photoplethysmography (iPPG) is an emerging technology used to assess microcirculation and cardiovascular signs by collecting backscattered light from illuminated tissue using optical imaging sensors. An engineering approach is used to evaluate whether a silicone cast of a human palm might be effectively utilized to predict the results of image registration schemes for motion compensation prior to their application on live human tissue. This allows us to establish a performance baseline for each of the algorithms and to isolate performance and noise fluctuations due to the induced motion from the temporally changing physiological signs. A multi-stage evaluation model is developed to qualitatively assess the influence of the region of interest (ROI), system resolution and distance, reference frame selection, and signal normalization on extracted iPPG waveforms from live tissue. We conclude that the application of image registration is able to deliver up to 75% signal-to-noise (SNR) improvement (4.75 to 8.34) over an uncompensated iPPG signal by employing an intensity-based algorithm with a moving reference frame

Exploring remote photoplethysmography signals for deepfake detection in facial videos

Abstract. With the advent of deep learning-based facial forgeries, also called "deepfakes", the feld of accurately detecting forged videos has become a quickly growing area of research. For this endeavor, remote photoplethysmography, the process of extracting biological signals such as the blood volume pulse and heart rate from facial videos, offers an interesting avenue for detecting fake videos that appear utterly authentic to the human eye. This thesis presents an end-to-end system for deepfake video classifcation using remote photoplethysmography. The minuscule facial pixel colour changes are used to extract the rPPG signal, from which various features are extracted and used to train an XGBoost classifer. The classifer is then tested using various colour-to-blood volume pulse methods (OMIT, POS, LGI and CHROM) and three feature extraction window lengths of two, four and eight seconds. The classifer was found effective at detecting deepfake videos with an accuracy of 85 %, with minimal performance difference found between the window lengths. The GREEN channel signal was found to be important for this classifcationEtäfotoplethysmografian hyödyntäminen syväväärennösten tunnistamiseen. Tiivistelmä. Syväväärennösten eli syväoppimiseen perustuvien kasvoväärennöksien yleistyessä väärennösten tarkasta tunnistamisesta koneellisesti on tullut nopeasti kasvava tutkimusalue. Etäfotoplethysmografa (rPPG) eli biologisten signaalien kuten veritilavuuspulssin tai sykkeen mittaaminen videokuvasta tarjoaa kiinnostavan keinon tunnistaa väärennöksiä, jotka vaikuttavat täysin aidoilta ihmissilmälle. Tässä diplomityössä esitellään etäfotoplethysmografaan perustuva syväväärennösten tunnistusmetodi. Kasvojen minimaalisia värimuutoksia hyväksikäyttämällä mitataan fotoplethysmografasignaali, josta lasketuilla ominaisuuksilla koulutetaan XGBoost-luokittelija. Luokittelijaa testataan usealla eri värisignaalista veritilavuussignaaliksi muuntavalla metodilla sekä kolmella eri ominaisuuksien ikkunapituudella. Luokittelija pystyy tunnistamaan väärennetyn videon aidosta 85 % tarkkuudella. Eri ikkunapituuksien välillä oli minimaalisia eroja, ja vihreän värin signaalin havaittiin olevan luokittelun suorituskyvyn kannalta merkittävä

Analisis Photoplethysmography Jarak Jauh dalam berbagai Kondisi Pencahayaan

Photoplethysmography (PPG) konvensional untuk mengukur kecepatan jantung memiliki keterbatasan tersendiri, salah satunya yaitu diperlukannya kontak langsung dengan bagian tubuh pasien. rPPG (remote-Photoplethysmography) dapat digunakan untuk melakukan pemantauan jantung dari seorang pasien berbasis citra. Sama halnya dengan sistem lain yang berbasis kamera, algoritma rPPG sangat bergantung pada kondisi pencahayaan. Oleh karena itu diperlukan suatu studi analisis yang mengindahkan aspek tentang pengaruh kondisi dan arah cahaya terhadap subjek yang diamati terhadap hasil estimasi laju denyut jantung dengan algoritma rPPG. Pada penelitian ini diimplementasikan algoritma rPPG dengan Short-Time Fourier Transform (STFT) untuk memperkirakan laju detak jantung dalam berbagai kondisi cahaya. Hasil yang diperoleh merupakan analisa spektral dari perubahan frame video pada area dahi terhadap perubahan waktu dari model input warna Green-Channel dan HSV (Hue, Saturation, Value). Perbandingan dengan pengukuran ground truth pada pencahayaan 260 lux, 19 lux, dan 11 lux, estimasi laju detak jantung yang didapatkan dari input Green Channel menghasilkan persentase error rata-rata 0,038, 0,118, dan 0,229, dimana hasil persentase rata-rata error ini lebih rendah dari masukan HSV, yaitu 0,095, 0,212, dan 0,24

Automated Remote Pulse Oximetry System (ARPOS)

Funding: This research is funded by the School of Computer Science and by St Leonard’s Postgraduate College Doctoral Scholarship, both at the University of St Andrews for Pireh Pirzada’s PhD. Early work was funded by the Digital Health & Care Innovation Centre (DHI).Current methods of measuring heart rate (HR) and oxygen levels (SPO2) require physical contact, are individualised, and for accurate oxygen levels may also require a blood test. No-touch or non-invasive technologies are not currently commercially available for use in healthcare settings. To date, there has been no assessment of a system that measures HR and SPO2 using commercial off-the-shelf camera technology that utilises R, G, B and IR data. Moreover, no formal remote photoplethysmography studies have been done in real life scenarios with participants at home with different demographic characteristics. This novel study addresses all these objectives by developing, optimising, and evaluating a system that measures the HR and SPO2 of 40 participants. HR and SPO2 are determined by measuring the frequencies from different wavelength band regions using FFT and radiometric measurements after pre-processing face regions of interest (forehead, lips, and cheeks) from Colour, IR and Depth data. Detrending, interpolating, hamming, and normalising the signal with FastICA produced the lowest RMSE of 7.8 for HR with the r-correlation value of 0.85 and RMSE 2.3 for SPO2. This novel system could be used in several critical care settings, including in care homes and in hospitals and prompt clinical intervention as required.Publisher PDFPeer reviewe

Signal processing techniques for cardiovascular monitoring applications using conventional and video-based photoplethysmography

Photoplethysmography (PPG)-based monitoring devices will probably play a decisive role in healthcare environment of the future, which will be preventive, predictive, personalized and participatory. Indeed, this optical technology presents several practical advantages over gold standard methods based on electrocardiography, because PPG wearable devices can be comfortably used for long-term continuous monitoring during daily life activities. Contactless video-based PPG technique, also known as imaging photoplethysmography (iPPG), has also attracted much attention recently. In that case, the cardiac pulse is remotely measured from the subtle skin color changes resulting from the blood circulation, using a simple video camera. PPG/iPPG have a lot of potential for a wide range of cardiovascular applications. Hence, there is a substantial need for signal processing techniques to explore these applications and to improve the reliability of the PPG/iPPG-based parameters. \par A part of the thesis is dedicated to the development of robust processing schemes to estimate heart rate from the PPG/iPPG signals. The proposed approaches were built on adaptive frequency tracking algorithms that were previously developed in our group. These tools, based on adaptive band-pass filters, provide instantaneous frequency estimates of the input signal(s) with a very low time delay, making them suitable for real-time applications. In case of conventional PPG, a prior adaptive noise cancellation step involving the use of accelerometer signals was also necessary to reconstruct clean PPG signals during the regions corrupted by motion artifacts. Regarding iPPG, after comparing different regions of interest on the subject face, we hypothesized that the simultaneous use of different iPPG signal derivation methods (i.e. methods to derive the iPPG time series from the pixel values of the consecutive frames) could be advantageous. Methods to assess signal quality online and to incorporate it into instantaneous frequency estimation were also examined and successfully applied to improve system reliability. \par This thesis also explored different innovative applications involving PPG/iPPG signals. The detection of atrial fibrillation was studied. Novel features derived directly from the PPG waveforms, designed to reflect the morphological changes observed during arrhythmic episodes, were proposed and proven to be successful for atrial fibrillation detection. Arrhythmia detection and robust heart rate estimation approaches were combined in another study aimed at reducing the number of false arrhythmia alarms in the intensive care unit by exploiting signals from independent sources, including PPG. Evaluation on a hidden dataset demonstrated that the number of false alarms was drastically reduced while almost no true alarm was suppressed. Finally, other aspects of the iPPG technology were examined, such as the measurement of pulse rate variability indexes from the iPPG signals and the estimation of respiratory rate from the iPPG interbeat intervals

Generative Adversarial Network for Photoplethysmography Reconstruction

Photoplethysmography (PPG) is an optical measurement method for blood pulse wave monitoring. The method has been widely applied in both clinical and wearable devices to collect physiological parameters, such as heart rate (HR) and heart rate variability (HRV). Unfortunately, the PPG signals are very vulnerable to motion artifacts, caused by inevitable movements of human users. To obtain reliable results from PPG-based monitoring, methods to denoise the PPG signals are necessary. Methods proposed in the literature, including signal decomposition, time-series analysis, and deep-learning based methods, reduce the effect of noise in PPG signals. However, their performance is insufficient for low signal-to-noise ratio PPG signals, or limited to noise from certain types of activities. Therefore, the aim of this study is to develop a method to remove the motion artifacts and reconstruct noisy PPG signals without any prior knowledge about the noise. In this thesis, a deep convolutional generative adversarial network (DC-GAN) based method is proposed to reconstruct the PPG signals corrupted by real-world motion artifacts. The proposed method leverages the temporal information from the distorted signal and its preceding data points to obtain the clean PPG signal. A GAN-based model is trained to generate succeeding clean PPG signals by previous data points. A sliding window moving at a fixed step on the noisy signal is used to select and update the input for the trained model by the information within the noisy signal. A PPG dataset collected by smartwatches in a health monitoring study is used to train, validate, and test the method in this study. A noisy dataset generated with real-world motion artifacts of different noise levels and lengths is used to evaluate the proposed and baseline methods. Three state-of-the-art PPG reconstruction methods are compared with our method. Two metrics, including maximum peak-to-peak error and RMSSD error, are extracted from the original and reconstructed signals to estimate the reconstruction error for HR and HRV. Our method outperforms state-of-the-art methods with the lowest values of the two evaluation matrices at all noise levels and lengths. The proposed method achieves 0.689, 1.352 and 1.821 seconds of maximum peak-to-peak errors for 5-second, 10-second, and 15-second noise at the highest noise level, respectively, and achieves 0.021, 0.048 and 0.067 seconds of RMSSD errors for the same noise cases. Consequently, our method performs the best in reconstructing distorted PPG signals and provides reliable estimation for both HR and HRV

The 2023 wearable photoplethysmography roadmap

Photoplethysmography is a key sensing technology which is used in wearable devices such as smartwatches and fitness trackers. Currently, photoplethysmography sensors are used to monitor physiological parameters including heart rate and heart rhythm, and to track activities like sleep and exercise. Yet, wearable photoplethysmography has potential to provide much more information on health and wellbeing, which could inform clinical decision making. This Roadmap outlines directions for research and development to realise the full potential of wearable photoplethysmography. Experts discuss key topics within the areas of sensor design, signal processing, clinical applications, and research directions. Their perspectives provide valuable guidance to researchers developing wearable photoplethysmography technology