Fast Landmark Localization with 3D Component Reconstruction and CNN for Cross-Pose Recognition

Two approaches are proposed for cross-pose face recognition, one is based on the 3D reconstruction of facial components and the other is based on the deep Convolutional Neural Network (CNN). Unlike most 3D approaches that consider holistic faces, the proposed approach considers 3D facial components. It segments a 2D gallery face into components, reconstructs the 3D surface for each component, and recognizes a probe face by component features. The segmentation is based on the landmarks located by a hierarchical algorithm that combines the Faster R-CNN for face detection and the Reduced Tree Structured Model for landmark localization. The core part of the CNN-based approach is a revised VGG network. We study the performances with different settings on the training set, including the synthesized data from 3D reconstruction, the real-life data from an in-the-wild database, and both types of data combined. We investigate the performances of the network when it is employed as a classifier or designed as a feature extractor. The two recognition approaches and the fast landmark localization are evaluated in extensive experiments, and compared to stateof-the-art methods to demonstrate their efficacy.Comment: 14 pages, 12 figures, 4 table

Simultaneous Facial Landmark Detection, Pose and Deformation Estimation under Facial Occlusion

Facial landmark detection, head pose estimation, and facial deformation analysis are typical facial behavior analysis tasks in computer vision. The existing methods usually perform each task independently and sequentially, ignoring their interactions. To tackle this problem, we propose a unified framework for simultaneous facial landmark detection, head pose estimation, and facial deformation analysis, and the proposed model is robust to facial occlusion. Following a cascade procedure augmented with model-based head pose estimation, we iteratively update the facial landmark locations, facial occlusion, head pose and facial de- formation until convergence. The experimental results on benchmark databases demonstrate the effectiveness of the proposed method for simultaneous facial landmark detection, head pose and facial deformation estimation, even if the images are under facial occlusion.Comment: International Conference on Computer Vision and Pattern Recognition, 201

Extremely Low Quality Image Face Recognition

Aastate jooksul on piltide töötlemine ja analüüs arenenud pakkudes nüüd igapäevastele väljakutsetele praktilisi lahendusi. Uute lahenduste ja ettepanekute sünd toob kaasa ka uusi väljakutseid, mis on paratamatult seotud innovaatiliste uuendustega. Olemasolevad näotuvastuse algoritmid on hästi toiminud ja neid on muu hulgas rakendatud sellistes lahendustes nagu sotsiaalmeedia kujutise märgistamine, mobiiltelefoni näo biomeetriline autentimine ja sisserände piirikontrolli näotuvastus. Põhjus miks need algoritmid on suutnud eelnimetatud stsenaariumides hästi toimida tuleneb sellest, et kasutuskõlblike kujutiste kvaliteet on tavaliselt kõrge eraldusvõimega [1].Teistes näidetes kus näotuvastus vajalikuks osutub nagu linna turvakaamerad, lennujaama kaamerad ja muud situatsioonid kus kujutise salvestuskvaliteeti ei saa kontrollida või manipuleerida, muutub jõulisema lahenduse leidmine pea kohustuslikuks, et oleks võimalik nägu tuvastada sõltumata kaadri suurusest, valgusoludest, rassist, vanusest, kehaasendist või muudest varieeruvatest faktoritest, mis võivad oluliselt muuta algoritmide võimet kujutistest aru saada.Käesoleva töö eesmärk on tuvastada ja testida alternatiivseid meetodeid näotuvastusülesannete täitmiseks äärmiselt madala kvaliteediga piltides.Image processing and analysis have evolved over the years into providing practical solutions to everyday challenges. The birth of new solutions and proposals also create new challenges usually surrounding the new innovations.Existing face recognition algorithms have performed well and they have been deployed into solutions such as social media image tagging, mobile phone facial bio-metric authentication, immigration border control face matching among other solutions. The existing algorithms have been able to perform well in these scenarios because of the quality of the image from these use cases are usually of high quality with high resolution (HR) [1]. In other possible application of face recognition such as city camera surveillance, airport security surveillance and other related scenarios where image stream quality cannot be directly controlled or manipulated, it becomes imperative to seek a more robust solution that can deal with face recognition regardless of the frame size, lighting condition, race, age, pose and other varying factors that can significantly change the way the images are perceived by existing algorithms.The goal of this thesis is to identify and test alternative methods of performing face recognition task in extremely low-quality images

POSEidon: Face-from-Depth for Driver Pose Estimation

Fast and accurate upper-body and head pose estimation is a key task for automatic monitoring of driver attention, a challenging context characterized by severe illumination changes, occlusions and extreme poses. In this work, we present a new deep learning framework for head localization and pose estimation on depth images. The core of the proposal is a regression neural network, called POSEidon, which is composed of three independent convolutional nets followed by a fusion layer, specially conceived for understanding the pose by depth. In addition, to recover the intrinsic value of face appearance for understanding head position and orientation, we propose a new Face-from-Depth approach for learning image faces from depth. Results in face reconstruction are qualitatively impressive. We test the proposed framework on two public datasets, namely Biwi Kinect Head Pose and ICT-3DHP, and on Pandora, a new challenging dataset mainly inspired by the automotive setup. Results show that our method overcomes all recent state-of-art works, running in real time at more than 30 frames per second

POSEidon: Face-from-Depth for Driver Pose Estimation

Fast and accurate upper-body and head pose estimation is a key task for automatic monitoring of driver attention, a challenging context characterized by severe illumination changes, occlusions and extreme poses. In this work, we present a new deep learning framework for head localization and pose estimation on depth images. The core of the proposal is a regression neural network, called POSEidon, which is composed of three independent convolutional nets followed by a fusion layer, specially conceived for understanding the pose by depth. In addition, to recover the intrinsic value of face appearance for understanding head position and orientation, we propose a new Face-from-Depth approach for learning image faces from depth. Results in face reconstruction are qualitatively impressive. We test the proposed framework on two public datasets, namely Biwi Kinect Head Pose and ICT-3DHP, and on Pandora, a new challenging dataset mainly inspired by the automotive setup. Results show that our method overcomes all recent state-of-art works, running in real time at more than 30 frames per second

Studies on Imaging System and Machine Learning: 3D Halftoning and Human Facial Landmark Localization

In this dissertation, studies on digital halftoning and human facial landmark localization will be discussed. 3D printing is becoming increasingly popular around the world today. By utilizing 3D printing technology, customized products can be manufactured much more quickly and efficiently with much less cost. However, 3D printing still suffers from low-quality surface reproduction compared with 2D printing. One approach to improve it is to develop an advanced halftoning algorithm for 3D printing. In this presentation, we will describe a novel method to 3D halftoning that can cooperate with 3D printing technology in order to generate a high-quality surface reproduction. In the second part of this report, a new method named direct element swap to create a threshold matrix for halftoning is proposed. This method directly swaps the elements in a threshold matrix to find the best element arrangement by minimizing a designated perceived error metric. Through experimental results, the new method yields halftone quality that is competitive with the conventional level-by-level matrix design method. Besides, by using direct element swap method, for the first time, threshold matrix can be designed through being trained with real images. In the second part of the dissertation, a novel facial landmark detection system is presented. Facial landmark detection plays a critical role in many face analysis tasks. However, it still remains a very challenging problem. The challenges come from the large variations of face appearance caused by different illuminations, different facial expressions, different yaw, pitch and roll angles of heads and different image qualities. To tackle this problem, a novel coarse-to-fine cascaded convolutional neural network system for robust facial landmark detection of faces in the wild is presented. The experiment result shows our method outperforms other state-of-the-art methods on public test datasets. Besides, a frontal and profile landmark localization system is proposed and designed. By using a frontal/profile face classifier, either frontal landmark configuration or profile landmark configuration is employed in the facial landmark prediction based on the input face yaw angle