76 research outputs found
Remote Bio-Sensing: Open Source Benchmark Framework for Fair Evaluation of rPPG
Remote Photoplethysmography (rPPG) is a technology that utilizes the light
absorption properties of hemoglobin, captured via camera, to analyze and
measure blood volume pulse (BVP). By analyzing the measured BVP, various
physiological signals such as heart rate, stress levels, and blood pressure can
be derived, enabling applications such as the early prediction of
cardiovascular diseases. rPPG is a rapidly evolving field as it allows the
measurement of vital signals using camera-equipped devices without the need for
additional devices such as blood pressure monitors or pulse oximeters, and
without the assistance of medical experts. Despite extensive efforts and
advances in this field, serious challenges remain, including issues related to
skin color, camera characteristics, ambient lighting, and other sources of
noise, which degrade performance accuracy. We argue that fair and evaluable
benchmarking is urgently required to overcome these challenges and make any
meaningful progress from both academic and commercial perspectives. In most
existing work, models are trained, tested, and validated only on limited
datasets. Worse still, some studies lack available code or reproducibility,
making it difficult to fairly evaluate and compare performance. Therefore, the
purpose of this study is to provide a benchmarking framework to evaluate
various rPPG techniques across a wide range of datasets for fair evaluation and
comparison, including both conventional non-deep neural network (non-DNN) and
deep neural network (DNN) methods. GitHub URL:
https://github.com/remotebiosensing/rppg.Comment: 19 pages, 10 figure
Exploring remote photoplethysmography signals for deepfake detection in facial videos
Abstract. With the advent of deep learning-based facial forgeries, also called "deepfakes", the feld of accurately detecting forged videos has become a quickly growing area of research. For this endeavor, remote photoplethysmography, the process of extracting biological signals such as the blood volume pulse and heart rate from facial videos, offers an interesting avenue for detecting fake videos that appear utterly authentic to the human eye.
This thesis presents an end-to-end system for deepfake video classifcation using remote photoplethysmography. The minuscule facial pixel colour changes are used to extract the rPPG signal, from which various features are extracted and used to train an XGBoost classifer. The classifer is then tested using various colour-to-blood volume pulse methods (OMIT, POS, LGI and CHROM) and three feature extraction window lengths of two, four and eight seconds.
The classifer was found effective at detecting deepfake videos with an accuracy of 85 %, with minimal performance difference found between the window lengths. The GREEN channel signal was found to be important for this classifcationEtäfotoplethysmografian hyödyntäminen syväväärennösten tunnistamiseen. Tiivistelmä. Syväväärennösten eli syväoppimiseen perustuvien kasvoväärennöksien yleistyessä väärennösten tarkasta tunnistamisesta koneellisesti on tullut nopeasti kasvava tutkimusalue. Etäfotoplethysmografa (rPPG) eli biologisten signaalien kuten veritilavuuspulssin tai sykkeen mittaaminen videokuvasta tarjoaa kiinnostavan keinon tunnistaa väärennöksiä, jotka vaikuttavat täysin aidoilta ihmissilmälle.
Tässä diplomityössä esitellään etäfotoplethysmografaan perustuva syväväärennösten tunnistusmetodi. Kasvojen minimaalisia värimuutoksia hyväksikäyttämällä mitataan fotoplethysmografasignaali, josta lasketuilla ominaisuuksilla koulutetaan XGBoost-luokittelija. Luokittelijaa testataan usealla eri värisignaalista veritilavuussignaaliksi muuntavalla metodilla sekä kolmella eri ominaisuuksien ikkunapituudella.
Luokittelija pystyy tunnistamaan väärennetyn videon aidosta 85 % tarkkuudella. Eri ikkunapituuksien välillä oli minimaalisia eroja, ja vihreän värin signaalin havaittiin olevan luokittelun suorituskyvyn kannalta merkittävä
Automatic roI detection for camera-based pulse-rate measurement
Remote photoplethysmography (rPPG) enables contactless measurement of pulse-rate by detecting pulse-induced colour changes on human skin using a regular camera. Most of existing rPPG methods exploit the subject face as the Region of Interest (RoI) for pulse-rate measurement by automatic face detection. However, face detection is a suboptimal solution since (1) not all the subregions in a face contain the skin pixels where pulse-signal can be extracted, (2) it fails to locate the RoI in cases when the frontal face is invisible (e.g., side-view faces). In this paper, we present a novel automatic RoI detection method for camerabased pulse-rate measurement, which consists of three main steps: subregion tracking, feature extraction, and clustering of skin regions. To evaluate the robustness of the proposed method, 36 video recordings are made of 6 subjects with different skin-types performing 6 types of head motion. Experimental results show that for the video sequences containing subjects with brighter skin-types and modest body motions, the accuracy of the pulse-rates measured by our method (94 %) is comparable to that obtained by a face detector (92 %), while the average SNR is significantly improved from 5.8 dB to 8.6 dB
- …