78 research outputs found
Face liveness detection by rPPG features and contextual patch-based CNN
Abstract. Face anti-spoofing plays a vital role in security systems including face payment systems and face recognition systems. Previous studies showed that live faces and presentation attacks have significant differences in both remote photoplethysmography (rPPG) and texture information. We propose a generalized method exploiting both rPPG and texture features for face anti-spoofing task. First, we design multi-scale long-term statistical spectral (MS-LTSS) features with variant granularities for the representation of rPPG information. Second, a contextual patch-based convolutional neural network (CP-CNN) is used for extracting global-local and multi-level deep texture features simultaneously. Finally, weight summation strategy is employed for decision level fusion of the two types of features, which allow the proposed system to be generalized for detecting not only print attack and replay attack, but also mask attack. Comprehensive experiments were conducted on five databases, namely 3DMAD, HKBU-Mars V1, MSU-MFSD, CASIA-FASD, and OULU-NPU, to show the superior results of the proposed method compared with state-of-the-art methods.Tiivistelmä. Kasvojen anti-spoofingilla on keskeinen rooli turvajärjestelmissä, mukaan lukien kasvojen maksujärjestelmät ja kasvojentunnistusjärjestelmät. Aiemmat tutkimukset osoittivat, että elävillä kasvoilla ja esityshyökkäyksillä on merkittäviä eroja sekä etävalopölymografiassa (rPPG) että tekstuuri-informaatiossa, ehdotamme yleistettyä menetelmää, jossa hyödynnetään sekä rPPG: tä että tekstuuriominaisuuksia kasvojen anti-spoofing -tehtävässä. Ensinnäkin rPPG-informaation esittämiseksi on suunniteltu monivaiheisia pitkän aikavälin tilastollisia spektrisiä (MS-LTSS) ominaisuuksia, joissa on muunneltavissa olevat granulariteetit. Toiseksi, kontekstuaalista patch-pohjaista konvoluutioverkkoa (CP-CNN) käytetään globaalin paikallisen ja monitasoisen syvään tekstuuriominaisuuksiin samanaikaisesti. Lopuksi, painoarvostusstrategiaa käytetään päätöksentekotason fuusioon, joka auttaa yleistämään menetelmää paitsi hyökkäys- ja toistoiskuille, mutta myös peittää hyökkäyksen. Kattavat kokeet suoritettiin viidellä tietokannalla, nimittäin 3DMAD, HKBU-Mars V1, MSU-MFSD, CASIA-FASD ja OULU-NPU, ehdotetun menetelmän parempien tulosten osoittamiseksi verrattuna uusimpiin menetelmiin
A Survey of PPG's Application in Authentication
Biometric authentication prospered because of its convenient use and
security. Early generations of biometric mechanisms suffer from spoofing
attacks. Recently, unobservable physiological signals (e.g.,
Electroencephalogram, Photoplethysmogram, Electrocardiogram) as biometrics
offer a potential remedy to this problem. In particular, Photoplethysmogram
(PPG) measures the change in blood flow of the human body by an optical method.
Clinically, researchers commonly use PPG signals to obtain patients' blood
oxygen saturation, heart rate, and other information to assist in diagnosing
heart-related diseases. Since PPG signals contain a wealth of individual
cardiac information, researchers have begun to explore their potential in cyber
security applications. The unique advantages (simple acquisition, difficult to
steal, and live detection) of the PPG signal allow it to improve the security
and usability of the authentication in various aspects. However, the research
on PPG-based authentication is still in its infancy. The lack of
systematization hinders new research in this field. We conduct a comprehensive
study of PPG-based authentication and discuss these applications' limitations
before pointing out future research directions.Comment: Accepted by Computer & Security (COSE
Contrast-Phys: Unsupervised Video-based Remote Physiological Measurement via Spatiotemporal Contrast
Video-based remote physiological measurement utilizes face videos to measure
the blood volume change signal, which is also called remote
photoplethysmography (rPPG). Supervised methods for rPPG measurements achieve
state-of-the-art performance. However, supervised rPPG methods require face
videos and ground truth physiological signals for model training. In this
paper, we propose an unsupervised rPPG measurement method that does not require
ground truth signals for training. We use a 3DCNN model to generate multiple
rPPG signals from each video in different spatiotemporal locations and train
the model with a contrastive loss where rPPG signals from the same video are
pulled together while those from different videos are pushed away. We test on
five public datasets, including RGB videos and NIR videos. The results show
that our method outperforms the previous unsupervised baseline and achieves
accuracies very close to the current best supervised rPPG methods on all five
datasets. Furthermore, we also demonstrate that our approach can run at a much
faster speed and is more robust to noises than the previous unsupervised
baseline. Our code is available at
https://github.com/zhaodongsun/contrast-phys.Comment: accepted to ECCV 202
PhysFormer: Facial Video-based Physiological Measurement with Temporal Difference Transformer
Remote photoplethysmography (rPPG), which aims at measuring heart activities
and physiological signals from facial video without any contact, has great
potential in many applications (e.g., remote healthcare and affective
computing). Recent deep learning approaches focus on mining subtle rPPG clues
using convolutional neural networks with limited spatio-temporal receptive
fields, which neglect the long-range spatio-temporal perception and interaction
for rPPG modeling. In this paper, we propose the PhysFormer, an end-to-end
video transformer based architecture, to adaptively aggregate both local and
global spatio-temporal features for rPPG representation enhancement. As key
modules in PhysFormer, the temporal difference transformers first enhance the
quasi-periodic rPPG features with temporal difference guided global attention,
and then refine the local spatio-temporal representation against interference.
Furthermore, we also propose the label distribution learning and a curriculum
learning inspired dynamic constraint in frequency domain, which provide
elaborate supervisions for PhysFormer and alleviate overfitting. Comprehensive
experiments are performed on four benchmark datasets to show our superior
performance on both intra- and cross-dataset testings. One highlight is that,
unlike most transformer networks needed pretraining from large-scale datasets,
the proposed PhysFormer can be easily trained from scratch on rPPG datasets,
which makes it promising as a novel transformer baseline for the rPPG
community. The codes will be released at
https://github.com/ZitongYu/PhysFormer.Comment: Accepted by CVPR202
rPPG-MAE: Self-supervised Pre-training with Masked Autoencoders for Remote Physiological Measurement
Remote photoplethysmography (rPPG) is an important technique for perceiving
human vital signs, which has received extensive attention. For a long time,
researchers have focused on supervised methods that rely on large amounts of
labeled data. These methods are limited by the requirement for large amounts of
data and the difficulty of acquiring ground truth physiological signals. To
address these issues, several self-supervised methods based on contrastive
learning have been proposed. However, they focus on the contrastive learning
between samples, which neglect the inherent self-similar prior in physiological
signals and seem to have a limited ability to cope with noisy. In this paper, a
linear self-supervised reconstruction task was designed for extracting the
inherent self-similar prior in physiological signals. Besides, a specific
noise-insensitive strategy was explored for reducing the interference of motion
and illumination. The proposed framework in this paper, namely rPPG-MAE,
demonstrates excellent performance even on the challenging VIPL-HR dataset. We
also evaluate the proposed method on two public datasets, namely PURE and
UBFC-rPPG. The results show that our method not only outperforms existing
self-supervised methods but also exceeds the state-of-the-art (SOTA) supervised
methods. One important observation is that the quality of the dataset seems
more important than the size in self-supervised pre-training of rPPG. The
source code is released at https://github.com/linuxsino/rPPG-MAE
Explainable and Interpretable Face Presentation Attack Detection Methods
Decision support systems based on machine learning (ML) techniques are excelling in most artificial intelligence (AI) fields, over-performing other AI methods, as well as humans. However, challenges still exist that do not favour the dominance of AI in some applications. This proposal focuses on a critical one: lack of transparency and explainability, reducing trust and accountability of an AI system. The fact that most AI methods still operate as complex black boxes, makes the inner processes which sustain their predictions still unattainable. The awareness around these observations foster the need to regulate many sensitive domains where AI has been applied in order to interpret, explain and audit the reliability of the ML based systems.
Although modern-day biometric recognition (BR) systems are already benefiting from the performance gains achieved with AI (which can account for and learn subtle changes in the person to be authenticated or statistical mismatches between samples), it is still in the dark ages of black box models, without reaping the benefits of the mismatches between samples), it is still in the dark ages of black box models, without reaping the benefits of the XAI field. This work will focus on studying AI explainability in the field of biometrics focusing in particular use cases in BR, such as verification/ identification of individuals and liveness detection (LD) (aka, antispoofing).
The main goals of this work are: i) to become acquainted with the state-of-the-art in explainability and biometric recognition and PAD methods; ii) to develop an experimental work xxxxx
Tasks 1st semester
(1) Study of the state of the art- bibliography review on state of the art for presentation attack detection
(2) Get acquainted with the previous work of the group in the topic
(3) Data preparation and data pre-processing
(3) Define the experimental protocol, including performance metrics
(4) Perform baseline experiments
(5) Write monography
Tasks 2nd semester
(1) Update on the state of the art
(2) Data preparation and data pre-processing
(3) Propose and implement a methodology for interpretability in biometrics
(4) Evaluation of the performance and comparison with baseline and state of the art approaches
(5) Dissertation writing
Referências bibliográficas principais: (*)
[Doshi17] B. Kim and F. Doshi-Velez, "Interpretable machine learning: The fuss, the concrete and the questions," 2017
[Mol19] Christoph Molnar. Interpretable Machine Learning. 2019
[Sei18] C. Seibold, W. Samek, A. Hilsmann, and P. Eisert, "Accurate and robust neural networks for security related applications exampled by face morphing attacks," arXiv preprint arXiv:1806.04265, 2018
[Seq20] Sequeira, Ana F., João T. Pinto, Wilson Silva, Tiago Gonçalves and Cardoso, Jaime S., "Interpretable Biometrics: Should We Rethink How Presentation Attack Detection is Evaluated?", 8th IWBF2020
[Wilson18] W. Silva, K. Fernandes, M. J. Cardoso, and J. S. Cardoso, "Towards complementary explanations using deep neural networks," in Understanding and Interpreting Machine Learning in MICA. Springer, 2018
[Wilson19] W. Silva, K. Fernandes, and J. S. Cardoso, "How to produce complementary explanations using an Ensemble Model," in IJCNN. 2019
[Wilson19A] W. Silva, M. J. Cardoso, and J. S. Cardoso, "Image captioning as a proxy for Explainable Decisions" in Understanding and Interpreting Machine Learning in MICA, 2019 (Submitted
- …