300 Faces in-the-Wild Challenge: the first facial landmark localization challenge

Automatic facial point detection plays arguably the most important role in face analysis. Several methods have been proposed which reported their results on databases of both constrained and unconstrained conditions. Most of these databases provide annotations with different mark-ups and in some cases the are problems related to the accuracy of the fiducial points. The aforementioned issues as well as the lack of a evaluation protocol makes it difficult to compare performance between different systems. In this paper, we present the 300 Faces in-the-Wild Challenge: The first facial landmark localization Challenge which is held in conjunction with the International Conference on Computer Vision 2013, Sydney, Australia. The main goal of this challenge is to compare the performance of different methods on a new-collected dataset using the same evaluation protocol and the same mark-up and hence to develop the first standardized benchmark for facial landmark localization

Towards Realistic Facial Expression Recognition

Automatic facial expression recognition has attracted significant attention over the past decades. Although substantial progress has been achieved for certain scenarios (such as frontal faces in strictly controlled laboratory settings), accurate recognition of facial expression in realistic environments remains unsolved for the most part. The main objective of this thesis is to investigate facial expression recognition in unconstrained environments. As one major problem faced by the literature is the lack of realistic training and testing data, this thesis presents a web search based framework to collect realistic facial expression dataset from the Web. By adopting an active learning based method to remove noisy images from text based image search results, the proposed approach minimizes the human efforts during the dataset construction and maximizes the scalability for future research. Various novel facial expression features are then proposed to address the challenges imposed by the newly collected dataset. Finally, a spectral embedding based feature fusion framework is presented to combine the proposed facial expression features to form a more descriptive representation. This thesis also systematically investigates how the number of frames of a facial expression sequence can affect the performance of facial expression recognition algorithms, since facial expression sequences may be captured under different frame rates in realistic scenarios. A facial expression keyframe selection method is proposed based on keypoint based frame representation. Comprehensive experiments have been performed to demonstrate the effectiveness of the presented methods

Shape-appearance-correlated active appearance model

© 2016 Elsevier Ltd Among the challenges faced by current active shape or appearance models, facial-feature localization in the wild, with occlusion in a novel face image, i.e. in a generic environment, is regarded as one of the most difficult computer-vision tasks. In this paper, we propose an Active Appearance Model (AAM) to tackle the problem of generic environment. Firstly, a fast face-model initialization scheme is proposed, based on the idea that the local appearance of feature points can be accurately approximated with locality constraints. Nearest neighbors, which have similar poses and textures to a test face, are retrieved from a training set for constructing the initial face model. To further improve the fitting of the initial model to the test face, an orthogonal CCA (oCCA) is employed to increase the correlation between shape features and appearance features represented by Principal Component Analysis (PCA). With these two contributions, we propose a novel AAM, namely the shape-appearance-correlated AAM (SAC-AAM), and the optimization is solved by using the recently proposed fast simultaneous inverse compositional (Fast-SIC) algorithm. Experiment results demonstrate a 5–10% improvement on controlled and semi-controlled datasets, and with around 10% improvement on wild face datasets in terms of fitting accuracy compared to other state-of-the-art AAM models

300 faces in-the-wild challenge: database and results

Computer Vision has recently witnessed great research advance towards automatic facial points detection. Numerous methodologies have been proposed during the last few years that achieve accurate and efficient performance. However, fair comparison between these methodologies is infeasible mainly due to two issues. (a) Most existing databases, captured under both constrained and unconstrained (in-the-wild) conditions have been annotated using different mark-ups and, in most cases, the accuracy of the annotations is low. (b) Most published works report experimental results using different training/testing sets, different error metrics and, of course, landmark points with semantically different locations. In this paper, we aim to overcome the aforementioned problems by (a) proposing a semi-automatic annotation technique that was employed to re-annotate most existing facial databases under a unified protocol, and (b) presenting the 300 Faces In-The-Wild Challenge (300-W), the first facial landmark localization challenge that was organized twice, in 2013 and 2015. To the best of our knowledge, this is the first effort towards a unified annotation scheme of massive databases and a fair experimental comparison of existing facial landmark localization systems. The images and annotations of the new testing database that was used in the 300-W challenge are available from http://ibug.doc.ic.ac.uk/resources/facial-point-annotations

A Comprehensive Performance Evaluation of Deformable Face Tracking "In-the-Wild"

Recently, technologies such as face detection, facial landmark localisation and face recognition and verification have matured enough to provide effective and efficient solutions for imagery captured under arbitrary conditions (referred to as "in-the-wild"). This is partially attributed to the fact that comprehensive "in-the-wild" benchmarks have been developed for face detection, landmark localisation and recognition/verification. A very important technology that has not been thoroughly evaluated yet is deformable face tracking "in-the-wild". Until now, the performance has mainly been assessed qualitatively by visually assessing the result of a deformable face tracking technology on short videos. In this paper, we perform the first, to the best of our knowledge, thorough evaluation of state-of-the-art deformable face tracking pipelines using the recently introduced 300VW benchmark. We evaluate many different architectures focusing mainly on the task of on-line deformable face tracking. In particular, we compare the following general strategies: (a) generic face detection plus generic facial landmark localisation, (b) generic model free tracking plus generic facial landmark localisation, as well as (c) hybrid approaches using state-of-the-art face detection, model free tracking and facial landmark localisation technologies. Our evaluation reveals future avenues for further research on the topic.Comment: E. Antonakos and P. Snape contributed equally and have joint second authorshi

Multiview Landmark Detection for Identity-preserving Alignment

Projecte realitzat en el marc d’un programa de mobilitat amb la KTH Royal Institute of Technology[ANGLÈS] Face recognition is a fundamental task in computer vision and has been an important field of study for many years. This report presents a unified process for automatically extract a set of face landmarks and remove all differences related to pose, expression and environment by bringing faces to a neutral pose-centered state. Landmark detection is based on a multiple viewpoint Pictorial Structure model. In this project we address both the problem of how to find a set of landmarks from a model and the problem of training such a model from a set of labelled examples. We show how such a model successfully captures a great range of deformations needing far less training examples than common commercial face detectors. The alignment process basically aims to remove differences between multiple faces so they all can be analysed under the same criteria. It is carried out with Thin-plate Splines to adjust the detected set of landmarks to the desired configuration. We present results of our algorithms both in a constrained environment and in the challenging LFPW face database. Successful outcomes are shown that prove our method to be a solid process for unitedly recognise and warp faces in the wild and to be on a par with other state-of-the-art procedures.[CASTELLÀ] El reconocimiento y clasificación de caras representa uno de los campos de estudio más importantes dentro la visión artificial por computadora y el procesado de imagen. En este proyecto se presenta un proceso unificado que extrae automáticamente, a partir de imágenes, un conjunto de puntos faciales característicos, eliminando al mismo tiempo las diferencias relacionadas con la orientación, expresión e iluminación. En una primera fase, se utiliza un modelo de Estructuras Pictográficas Multivista para la detección de puntos característicos. En este informe, se atienden con detalle tanto el problema de extraer puntos con este modelo, como el proceso de aprendizaje a partir de imágenes anotadas. En una segunda fase, se eliminan las diferencias entre caras mediante el uso de Thin-Plate Splines, para que todas las imágenes puedan ser analizadas bajo el mismo criterio en posibles aplicaciones futuras. Los algoritmos implementados, representan hasta la fecha, el primer sistema desarrollado, que simultáneamente extrae y alinea caras, y demuestran un comportamiento muy fiable y preciso tanto en entornos controlados como libres (base de datos LFPW).[CATALÀ] reconeixement i classificació de cares és un dels camps d'estudi més importants dintre la visió artificial per computadora i el processament d'imatge. En aquest projecte es presenta un procés unificat que extreu automàticament, a partir d'imatges, un conjunt de punts facials característics eliminant al mateix temps les diferències relacionades amb l'orientació, expressió i il·luminació. En una primera fase, s'utilitza un model d'Estructures Pictogràfiques Multivista per a la detecció de punts característics. En aquest informe, s'adreça amb detall tant el problema d'extreure punts amb aquest model com el procés d'aprenentatge a partir d'imatges anotades. En una segona fase, s'eliminen les diferències entre cares mitjançant Thin-Plate Splines, per tal que totes les imatges puguin ser analitzades sota el mateix criteri en possibles aplicacions futures. Els algoritmes implementats, representen fins al moment, el primer sistema desenvolupat que unificadament extreu i alinea cares, i demostren un comportament molt fiable i acurat tant en entorns controlats com lliures (base de dades LFPW)

Virtual View Networks for Object Reconstruction

All that structure from motion algorithms "see" are sets of 2D points. We show that these impoverished views of the world can be faked for the purpose of reconstructing objects in challenging settings, such as from a single image, or from a few ones far apart, by recognizing the object and getting help from a collection of images of other objects from the same class. We synthesize virtual views by computing geodesics on novel networks connecting objects with similar viewpoints, and introduce techniques to increase the specificity and robustness of factorization-based object reconstruction in this setting. We report accurate object shape reconstruction from a single image on challenging PASCAL VOC data, which suggests that the current domain of applications of rigid structure-from-motion techniques may be significantly extended