Simultaneous Facial Landmark Detection, Pose and Deformation Estimation under Facial Occlusion

Facial landmark detection, head pose estimation, and facial deformation analysis are typical facial behavior analysis tasks in computer vision. The existing methods usually perform each task independently and sequentially, ignoring their interactions. To tackle this problem, we propose a unified framework for simultaneous facial landmark detection, head pose estimation, and facial deformation analysis, and the proposed model is robust to facial occlusion. Following a cascade procedure augmented with model-based head pose estimation, we iteratively update the facial landmark locations, facial occlusion, head pose and facial de- formation until convergence. The experimental results on benchmark databases demonstrate the effectiveness of the proposed method for simultaneous facial landmark detection, head pose and facial deformation estimation, even if the images are under facial occlusion.Comment: International Conference on Computer Vision and Pattern Recognition, 201

3D Face Reconstruction by Learning from Synthetic Data

Fast and robust three-dimensional reconstruction of facial geometric structure from a single image is a challenging task with numerous applications. Here, we introduce a learning-based approach for reconstructing a three-dimensional face from a single image. Recent face recovery methods rely on accurate localization of key characteristic points. In contrast, the proposed approach is based on a Convolutional-Neural-Network (CNN) which extracts the face geometry directly from its image. Although such deep architectures outperform other models in complex computer vision problems, training them properly requires a large dataset of annotated examples. In the case of three-dimensional faces, currently, there are no large volume data sets, while acquiring such big-data is a tedious task. As an alternative, we propose to generate random, yet nearly photo-realistic, facial images for which the geometric form is known. The suggested model successfully recovers facial shapes from real images, even for faces with extreme expressions and under various lighting conditions.Comment: The first two authors contributed equally to this wor

Image tag completion by local learning

The problem of tag completion is to learn the missing tags of an image. In this paper, we propose to learn a tag scoring vector for each image by local linear learning. A local linear function is used in the neighborhood of each image to predict the tag scoring vectors of its neighboring images. We construct a unified objective function for the learning of both tag scoring vectors and local linear function parame- ters. In the objective, we impose the learned tag scoring vectors to be consistent with the known associations to the tags of each image, and also minimize the prediction error of each local linear function, while reducing the complexity of each local function. The objective function is optimized by an alternate optimization strategy and gradient descent methods in an iterative algorithm. We compare the proposed algorithm against different state-of-the-art tag completion methods, and the results show its advantages

High-contrast imaging at small separation: impact of the optical configuration of two deformable mirrors on dark holes

The direct detection and characterization of exoplanets will be a major scientific driver over the next decade, involving the development of very large telescopes and requires high-contrast imaging close to the optical axis. Some complex techniques have been developed to improve the performance at small separations (coronagraphy, wavefront shaping, etc). In this paper, we study some of the fundamental limitations of high contrast at the instrument design level, for cases that use a combination of a coronagraph and two deformable mirrors for wavefront shaping. In particular, we focus on small-separation point-source imaging (around 1 $\lambda$/D). First, we analytically or semi-analytically analysing the impact of several instrument design parameters: actuator number, deformable mirror locations and optic aberrations (level and frequency distribution). Second, we develop in-depth Monte Carlo simulation to compare the performance of dark hole correction using a generic test-bed model to test the Fresnel propagation of multiple randomly generated optics static phase errors. We demonstrate that imaging at small separations requires large setup and small dark hole size. The performance is sensitive to the optic aberration amount and spatial frequencies distribution but shows a weak dependence on actuator number or setup architecture when the dark hole is sufficiently small (from 1 to $\lesssim$ 5 $\lambda$/D).Comment: 13 pages, 18 figure

Direct and Simultaneous Observation of Ultrafast Electron and Hole Dynamics in Germanium

Understanding excited carrier dynamics in semiconductors is crucial for the development of photovoltaics and efficient photonic devices. However, overlapping spectral features in optical/NIR pump-probe spectroscopy often render assignments of separate electron and hole carrier dynamics ambiguous. Here, ultrafast electron and hole dynamics in germanium nanocrystalline thin films are directly and simultaneously observed by attosecond transient absorption spectroscopy (ATAS) in the extreme ultraviolet at the germanium M_{4,5}-edge (~30 eV). We decompose the ATAS spectra into contributions of electronic state blocking and photo-induced band shifts at a carrier density of 8*10^{20}cm^{-3}. Separate electron and hole relaxation times are observed as a function of hot carrier energies. A first order electron and hole decay of ~1 ps suggests a Shockley-Read-Hall recombination mechanism. The simultaneous observation of electrons and holes with ATAS paves the way for investigating few to sub-femtosecond dynamics of both holes and electrons in complex semiconductor materials and across junctions.Comment: Includes Supplementary Informatio