Multi-View Face Recognition From Single RGBD Models of the Faces

This work takes important steps towards solving the following problem of current interest: Assuming that each individual in a population can be modeled by a single frontal RGBD face image, is it possible to carry out face recognition for such a population using multiple 2D images captured from arbitrary viewpoints? Although the general problem as stated above is extremely challenging, it encompasses subproblems that can be addressed today. The subproblems addressed in this work relate to: (1) Generating a large set of viewpoint dependent face images from a single RGBD frontal image for each individual; (2) using hierarchical approaches based on view-partitioned subspaces to represent the training data; and (3) based on these hierarchical approaches, using a weighted voting algorithm to integrate the evidence collected from multiple images of the same face as recorded from different viewpoints. We evaluate our methods on three datasets: a dataset of 10 people that we created and two publicly available datasets which include a total of 48 people. In addition to providing important insights into the nature of this problem, our results show that we are able to successfully recognize faces with accuracies of 95% or higher, outperforming existing state-of-the-art face recognition approaches based on deep convolutional neural networks

Individual stable space : an approach to face recognition under uncontrolled conditions

There usually exist many kinds of variations in face images taken under uncontrolled conditions, such as changes of pose, illumination, expression, etc. Most previous works on face recognition (FR) focus on particular variations and usually assume the absence of others. Instead of such a ldquodivide and conquerrdquo strategy, this paper attempts to directly address face recognition under uncontrolled conditions. The key is the individual stable space (ISS), which only expresses personal characteristics. A neural network named ISNN is proposed to map a raw face image into the ISS. After that, three ISS-based algorithms are designed for FR under uncontrolled conditions. There are no restrictions for the images fed into these algorithms. Moreover, unlike many other FR techniques, they do not require any extra training information, such as the view angle. These advantages make them practical to implement under uncontrolled conditions. The proposed algorithms are tested on three large face databases with vast variations and achieve superior performance compared with other 12 existing FR techniques.<br /

Total Capture: A 3D Deformation Model for Tracking Faces, Hands, and Bodies

We present a unified deformation model for the markerless capture of multiple scales of human movement, including facial expressions, body motion, and hand gestures. An initial model is generated by locally stitching together models of the individual parts of the human body, which we refer to as the "Frankenstein" model. This model enables the full expression of part movements, including face and hands by a single seamless model. Using a large-scale capture of people wearing everyday clothes, we optimize the Frankenstein model to create "Adam". Adam is a calibrated model that shares the same skeleton hierarchy as the initial model but can express hair and clothing geometry, making it directly usable for fitting people as they normally appear in everyday life. Finally, we demonstrate the use of these models for total motion tracking, simultaneously capturing the large-scale body movements and the subtle face and hand motion of a social group of people

Empirical mode decomposition-based facial pose estimation inside video sequences

We describe a new pose-estimation algorithm via integration of the strength in both empirical mode decomposition (EMD) and mutual information. While mutual information is exploited to measure the similarity between facial images to estimate poses, EMD is exploited to decompose input facial images into a number of intrinsic mode function (IMF) components, which redistribute the effect of noise, expression changes, and illumination variations as such that, when the input facial image is described by the selected IMF components, all the negative effects can be minimized. Extensive experiments were carried out in comparisons to existing representative techniques, and the results show that the proposed algorithm achieves better pose-estimation performances with robustness to noise corruption, illumination variation, and facial expressions

Verificação facial em duas etapas para dispositivos móveis

Orientadores: Jacques Wainer, Fernanda Alcântara AndalóDissertação (mestrado) - Universidade Estadual de Campinas, Instituto de ComputaçãoResumo: Dispositivos móveis, como smartphones e tablets, se tornaram mais populares e acessíveis nos últimos anos. Como consequência de sua ubiquidade, esses aparelhos guardam diversos tipos de informações pessoais (fotos, conversas de texto, coordenadas GPS, dados bancários, entre outros) que só devem ser acessadas pelo dono do dispositivo. Apesar de métodos baseados em conhecimento, como senhas numéricas ou padrões, ainda estejam entre as principais formas de assegurar a identidade do usuário, traços biométricos tem sido utilizados para garantir uma autenticação mais segura e prática. Entre eles, reconhecimento facial ganhou atenção nos últimos anos devido aos recentes avanços nos dispositivos de captura de imagens e na crescente disponibilidade de fotos em redes sociais. Aliado a isso, o aumento de recursos computacionais, com múltiplas CPUs e GPUs, permitiu o desenvolvimento de modelos mais complexos e robustos, como redes neurais profundas. Porém, apesar da evolução das capacidades de dispositivos móveis, os métodos de reconhecimento facial atuais ainda não são desenvolvidos considerando as características do ambiente móvel, como processamento limitado, conectividade instável e consumo de bateria. Neste trabalho, nós propomos um método de verificação facial otimizado para o ambiente móvel. Ele consiste em um procedimento em dois níveis que combina engenharia de características (histograma de gradientes orientados e análise de componentes principais por regiões) e uma rede neural convolucional para verificar se o indivíduo presente em uma imagem corresponde ao dono do dispositivo. Nós também propomos a \emph{Hybrid-Fire Convolutional Neural Network}, uma arquitetura ajustada para dispositivos móveis que processa informação de pares de imagens. Finalmente, nós apresentamos uma técnica para adaptar o limiar de aceitação do método proposto para imagens com características diferentes daquelas presentes no treinamento, utilizando a galeria de imagens do dono do dispositivo. A solução proposta se compara em acurácia aos métodos de reconhecimento facial do estado da arte, além de possuir um modelo 16 vezes menor e 4 vezes mais rápido ao processar uma imagem em smartphones modernos. Por último, nós também organizamos uma base de dados composta por 2873 selfies de 56 identidades capturadas em condições diversas, a qual esperamos que ajude pesquisas futuras realizadas neste cenárioAbstract: Mobile devices, such as smartphones and tablets, had their popularity and affordability greatly increased in recent years. As a consequence of their ubiquity, these devices now carry all sorts of personal data (\emph{e.g.} photos, text conversations, GPS coordinates, banking information) that should be accessed only by the device's owner. Even though knowledge-based procedures, such as entering a PIN or drawing a pattern, are still the main methods to secure the owner's identity, recently biometric traits have been employed for a more secure and effortless authentication. Among them, face recognition has gained more attention in past years due to recent improvements in image-capturing devices and the availability of images in social networks. In addition to that, the increase in computational resources, with multiple CPUs and GPUs, enabled the design of more complex and robust models, such as deep neural networks. Although the capabilities of mobile devices have been growing in past years, most recent face recognition techniques are still not designed considering the mobile environment's characteristics, such as limited processing power, unstable connectivity and battery consumption. In this work, we propose a facial verification method optimized to the mobile environment. It consists of a two-tiered procedure that combines hand-crafted features (histogram of oriented gradients and local region principal component analysis) and a convolutional neural network to verify if the person depicted in a picture corresponds to the device owner. We also propose \emph{Hybrid-Fire Convolutional Neural Network}, an architecture tweaked for mobile devices that process encoded information of a pair of face images. Finally, we expose a technique to adapt our method's acceptance thresholds to images with different characteristics than those present during training, by using the device owner's enrolled gallery. The proposed solution performs a par to the state-of-the-art face recognition methods, while having a model 16 times smaller and 4 times faster when processing an image in recent smartphone models. Finally, we have collected a new dataset of selfie pictures comprising 2873 images from 56 identities with varied capture conditions, that hopefully will support future researches in this scenarioMestradoCiência da ComputaçãoMestre em Ciência da Computaçã

FML: Face Model Learning from Videos

Monocular image-based 3D reconstruction of faces is a long-standing problem in computer vision. Since image data is a 2D projection of a 3D face, the resulting depth ambiguity makes the problem ill-posed. Most existing methods rely on data-driven priors that are built from limited 3D face scans. In contrast, we propose multi-frame video-based self-supervised training of a deep network that (i) learns a face identity model both in shape and appearance while (ii) jointly learning to reconstruct 3D faces. Our face model is learned using only corpora of in-the-wild video clips collected from the Internet. This virtually endless source of training data enables learning of a highly general 3D face model. In order to achieve this, we propose a novel multi-frame consistency loss that ensures consistent shape and appearance across multiple frames of a subject's face, thus minimizing depth ambiguity. At test time we can use an arbitrary number of frames, so that we can perform both monocular as well as multi-frame reconstruction.Comment: CVPR 2019 (Oral). Video: https://www.youtube.com/watch?v=SG2BwxCw0lQ, Project Page: https://gvv.mpi-inf.mpg.de/projects/FML19

Face Detection and Verification using Local Binary Patterns

This thesis proposes a robust Automatic Face Verification (AFV) system using Local Binary Patterns (LBP). AFV is mainly composed of two modules: Face Detection (FD) and Face Verification (FV). The purpose of FD is to determine whether there are any face in an image, while FV involves confirming or denying the identity claimed by a person. The contributions of this thesis are the following: 1) a real-time multiview FD system which is robust to illumination and partial occlusion, 2) a FV system based on the adaptation of LBP features, 3) an extensive study of the performance evaluation of FD algorithms and in particular the effect of FD errors on FV performance. The first part of the thesis addresses the problem of frontal FD. We introduce the system of Viola and Jones which is the first real-time frontal face detector. One of its limitations is the sensitivity to local lighting variations and partial occlusion of the face. In order to cope with these limitations, we propose to use LBP features. Special emphasis is given to the scanning process and to the merging of overlapped detections, because both have a significant impact on the performance. We then extend our frontal FD module to multiview FD. In the second part, we present a novel generative approach for FV, based on an LBP description of the face. The main advantages compared to previous approaches are a very fast and simple training procedure and robustness to bad lighting conditions. In the third part, we address the problem of estimating the quality of FD. We first show the influence of FD errors on the FV task and then empirically demonstrate the limitations of current detection measures when applied to this task. In order to properly evaluate the performance of a face detection module, we propose to embed the FV into the performance measuring process. We show empirically that the proposed methodology better matches the final FV performance