53 research outputs found
Robust Face Alignment Under Occlusion via Regional Predictive Power Estimation
Face alignment has been well studied in recent years, however, when a face alignment model is applied on facial images with heavy partial occlusion, the performance deteriorates significantly. In this paper, instead of training an occlusion-aware model with visibility annotation, we address this issue via a model adaptation scheme that uses the result of a local regression forest (RF) voting method. In the proposed scheme, the consistency of the votes of the local RF in each of several oversegmented regions is used to determine the reliability of predicting the location of the facial landmarks. The latter is what we call regional predictive power (RPP). Subsequently, we adapt a holistic voting method (cascaded pose regression based on random ferns) by putting weights on the votes of each fern according to the RPP of the regions used in the fern tests. The proposed method shows superior performance over existing face alignment models in the most challenging data sets (COFW and 300-W). Moreover, it can also estimate with high accuracy (72.4% overlap ratio) which image areas belong to the face or nonface objects, on the heavily occluded images of the COFW data set, without explicit occlusion modeling
Face Alignment in the Wild.
PhDFace alignment on a face image is a crucial step in many computer vision applications such
as face recognition, verification and facial expression recognition. In this thesis we present
a collection of methods for face alignment in real-world scenarios where the acquisition
of the face images cannot be controlled. We first investigate local based random regression
forest methods that work in a voting fashion. We focus on building better quality
random trees, first, by using privileged information and second, in contrast to using explicit
shape models, by incorporating spatial shape constraints within the forests. We also
propose a fine-tuning scheme that sieves and/or aggregates regression forest votes before
accumulating them into the Hough space. We then investigate holistic methods and propose
two schemes, namely the cascaded regression forests and the random subspace supervised
descent method (RSSDM). The former uses a regression forest as the primitive regressor
instead of random ferns and an intelligent initialization scheme. The RSSDM improves the
accuracy and generalization capacity of the popular SDM by using several linear regressions
in random subspaces. We also propose a Cascaded Pose Regression framework for
face alignment in different modalities, that is RGB and sketch images, based on a sketch
synthesis scheme. Finally, we introduce the concept of mirrorability which describes how
an object alignment method behaves on mirror images in comparison to how it behaves on
the original ones. We define a measure called mirror error to quantitatively analyse the mirrorability
and show two applications, namely difficult samples selection and cascaded face
alignment feedback that aids a re-initialisation scheme. The methods proposed in this thesis
perform better or comparable to state of the art methods. We also demonstrate the generality
by applying them on similar problems such as car alignment.China Scholarship Counci
Face Alignment Assisted by Head Pose Estimation
In this paper we propose a supervised initialization scheme for cascaded face
alignment based on explicit head pose estimation. We first investigate the
failure cases of most state of the art face alignment approaches and observe
that these failures often share one common global property, i.e. the head pose
variation is usually large. Inspired by this, we propose a deep convolutional
network model for reliable and accurate head pose estimation. Instead of using
a mean face shape, or randomly selected shapes for cascaded face alignment
initialisation, we propose two schemes for generating initialisation: the first
one relies on projecting a mean 3D face shape (represented by 3D facial
landmarks) onto 2D image under the estimated head pose; the second one searches
nearest neighbour shapes from the training set according to head pose distance.
By doing so, the initialisation gets closer to the actual shape, which enhances
the possibility of convergence and in turn improves the face alignment
performance. We demonstrate the proposed method on the benchmark 300W dataset
and show very competitive performance in both head pose estimation and face
alignment.Comment: Accepted by BMVC201
Recombinator Networks: Learning Coarse-to-Fine Feature Aggregation
Deep neural networks with alternating convolutional, max-pooling and
decimation layers are widely used in state of the art architectures for
computer vision. Max-pooling purposefully discards precise spatial information
in order to create features that are more robust, and typically organized as
lower resolution spatial feature maps. On some tasks, such as whole-image
classification, max-pooling derived features are well suited; however, for
tasks requiring precise localization, such as pixel level prediction and
segmentation, max-pooling destroys exactly the information required to perform
well. Precise localization may be preserved by shallow convnets without pooling
but at the expense of robustness. Can we have our max-pooled multi-layered cake
and eat it too? Several papers have proposed summation and concatenation based
methods for combining upsampled coarse, abstract features with finer features
to produce robust pixel level predictions. Here we introduce another model ---
dubbed Recombinator Networks --- where coarse features inform finer features
early in their formation such that finer features can make use of several
layers of computation in deciding how to use coarse features. The model is
trained once, end-to-end and performs better than summation-based
architectures, reducing the error from the previous state of the art on two
facial keypoint datasets, AFW and AFLW, by 30\% and beating the current
state-of-the-art on 300W without using extra data. We improve performance even
further by adding a denoising prediction model based on a novel convnet
formulation.Comment: accepted in CVPR 201
Pose-Invariant 3D Face Alignment
Face alignment aims to estimate the locations of a set of landmarks for a
given image. This problem has received much attention as evidenced by the
recent advancement in both the methodology and performance. However, most of
the existing works neither explicitly handle face images with arbitrary poses,
nor perform large-scale experiments on non-frontal and profile face images. In
order to address these limitations, this paper proposes a novel face alignment
algorithm that estimates both 2D and 3D landmarks and their 2D visibilities for
a face image with an arbitrary pose. By integrating a 3D deformable model, a
cascaded coupled-regressor approach is designed to estimate both the camera
projection matrix and the 3D landmarks. Furthermore, the 3D model also allows
us to automatically estimate the 2D landmark visibilities via surface normals.
We gather a substantially larger collection of all-pose face images to evaluate
our algorithm and demonstrate superior performances than the state-of-the-art
methods
- …