Face Frontalization Based on Robustly Fitting a Deformable Shape Model to 3D Landmarks

Face frontalization consists of synthesizing a frontally-viewed face from an arbitrarily-viewed one. The main contribution of this paper is a robust face alignment method that enables pixel-to-pixel warping. The method simultaneously estimates the rigid transformation (scale, rotation, and translation) and the non-rigid deformation between two 3D point sets: a set of 3D landmarks extracted from an arbitrary-viewed face, and a set of 3D landmarks parameterized by a frontally-viewed deformable face model. An important merit of the proposed method is its ability to deal both with noise (small perturbations) and with outliers (large errors). We propose to model inliers and outliers with the generalized Student's t-probability distribution function, a heavy-tailed distribution that is immune to non-Gaussian errors in the data. We describe in detail the associated expectation-maximization (EM) algorithm that alternates between the estimation of (i) the rigid parameters, (ii) the deformation parameters, and (iii) the Student-t distribution parameters. We also propose to use the zero-mean normalized cross-correlation, between a frontalized face and the corresponding ground-truth frontally-viewed face, to evaluate the performance of frontalization. To this end, we use a dataset that contains pairs of profile-viewed and frontally-viewed faces. This evaluation, based on direct image-to-image comparison, stands in contrast with indirect evaluation, based on analyzing the effect of frontalization on face recognition

Face Image Quality Assessment: A Literature Survey

The performance of face analysis and recognition systems depends on the quality of the acquired face data, which is influenced by numerous factors. Automatically assessing the quality of face data in terms of biometric utility can thus be useful to detect low-quality data and make decisions accordingly. This survey provides an overview of the face image quality assessment literature, which predominantly focuses on visible wavelength face image input. A trend towards deep learning based methods is observed, including notable conceptual differences among the recent approaches, such as the integration of quality assessment into face recognition models. Besides image selection, face image quality assessment can also be used in a variety of other application scenarios, which are discussed herein. Open issues and challenges are pointed out, i.a. highlighting the importance of comparability for algorithm evaluations, and the challenge for future work to create deep learning approaches that are interpretable in addition to providing accurate utility predictions

Head pose estimation through multi-class face segmentation

The aim of this work is to explore the usefulness of face semantic segmentation for head pose estimation. We implement a multi-class face segmentation algorithm and we train a model for each considered pose. Given a new test image, the probabilities associated to face parts by the different models are used as the only information for estimating the head orientation. A simple algorithm is proposed to exploit such probabilites in order to predict the pose. The proposed scheme achieves competitive results when compared to most recent methods, according to mean absolute error and accuracy metrics. Moreover, we release and make publicly available a face segmentation dataset consisting of 294 images belonging to 13 different poses, manually labeled into six semantic regions, which we used to train the segmentation models

Pooling Faces: Template based Face Recognition with Pooled Face Images

We propose a novel approach to template based face recognition. Our dual goal is to both increase recognition accuracy and reduce the computational and storage costs of template matching. To do this, we leverage on an approach which was proven effective in many other domains, but, to our knowledge, never fully explored for face images: average pooling of face photos. We show how (and why!) the space of a template's images can be partitioned and then pooled based on image quality and head pose and the effect this has on accuracy and template size. We perform extensive tests on the IJB-A and Janus CS2 template based face identification and verification benchmarks. These show that not only does our approach outperform published state of the art despite requiring far fewer cross template comparisons, but also, surprisingly, that image pooling performs on par with deep feature pooling.Comment: Appeared in the IEEE Computer Society Workshop on Biometrics, IEEE Conf. on Computer Vision and Pattern Recognition (CVPR), June, 201

Spatial organization of visuomotor reflexes in Drosophila

In most animals, the visual system plays a central role in locomotor guidance. Here, we examined the functional organization of visuomotor reflexes in the fruit fly, Drosophila, using an electronic flight simulator. Flies exhibit powerful avoidance responses to visual expansion centered laterally. The amplitude of these expansion responses is three times larger than those generated by image rotation. Avoidance of a laterally positioned focus of expansion emerges from an inversion of the optomotor response when motion is restricted to the rear visual hemisphere. Furthermore, motion restricted to rear quarter-fields elicits turning responses that are independent of the direction of image motion about the animal's yaw axis. The spatial heterogeneity of visuomotor responses explains a seemingly peculiar behavior in which flies robustly fixate the contracting pole of a translating flow field

A Multi-Camera System for a Real-Time Pose Estimation

This paper presents a multi-camera system that performs face detection and pose estimation in real-time and may be used for intelligent computing within a visual sensor network for surveillance or humancomputer interaction. The system consists of a Scene View Camera (SVC), which operates at a fixed zoom level, and an Object View Camera (OVC), which continuously adjusts its zoom level to match objects of interest. The SVC is set to survey the whole filed of view. Once a region has been identified by the SVC as a potential object of interest, e.g. a face, the OVC zooms in to locate specific features. In this system, face candidate regions are selected based on skin color and face detection is accomplished using a Support Vector Machine classifier. The locations of the eyes and mouth are detected inside the face region using neural network feature detectors. Pose estimation is performed based on a geometrical model, where the head is modeled as a spherical object that rotates upon the vertical axis. The triangle formed by the mouth and eyes defines a vertical plane that intersects the head sphere. By projecting the eyes-mouth triangle onto a two dimensional viewing plane, equations were obtained that describe the change in its angles as the yaw pose angle increases. These equations are then combined and used for efficient pose estimation. The system achieves real-time performance for live video input. Testing results assessing system performance are presented for both still images and video

Employing Symmetry Features for Automatic Misalignment Correction in Neuroimages

A novel method to automatically compute the symmetry plane and to correct the 3D orientation of neuro-images is presented. In acquisition of neuroimaging scans, the lack of perfect alignment of a patient's head makes it challenging to evaluate brain images. By deploying a shape-based criterion, the symmetry plane is defined as a plane that best matches external surface points on one side of the head, with their counterparts on the other side. In our method, the head volume is represented as a re-parameterized surface point cloud, where each location is parameterized by its elevation (latitude), azimuth (longitude), and radius. The search for the best matching surfaces is implemented in a multi-resolution paradigm, and the computation time is significantly decreased. The algorithm was quantitatively evaluated using in both simulated data and in real T1, T2, Flair magnetic resonance patient images. This algorithm is found to be fast (< 10s per MR volume), robust and accurate (< .6 degree of Mean Angular Error), invariant to the acquisition noise, slice thickness, bias field, and pathological asymmetries