A Comprehensive Performance Evaluation of Deformable Face Tracking "In-the-Wild"

Recently, technologies such as face detection, facial landmark localisation and face recognition and verification have matured enough to provide effective and efficient solutions for imagery captured under arbitrary conditions (referred to as "in-the-wild"). This is partially attributed to the fact that comprehensive "in-the-wild" benchmarks have been developed for face detection, landmark localisation and recognition/verification. A very important technology that has not been thoroughly evaluated yet is deformable face tracking "in-the-wild". Until now, the performance has mainly been assessed qualitatively by visually assessing the result of a deformable face tracking technology on short videos. In this paper, we perform the first, to the best of our knowledge, thorough evaluation of state-of-the-art deformable face tracking pipelines using the recently introduced 300VW benchmark. We evaluate many different architectures focusing mainly on the task of on-line deformable face tracking. In particular, we compare the following general strategies: (a) generic face detection plus generic facial landmark localisation, (b) generic model free tracking plus generic facial landmark localisation, as well as (c) hybrid approaches using state-of-the-art face detection, model free tracking and facial landmark localisation technologies. Our evaluation reveals future avenues for further research on the topic.Comment: E. Antonakos and P. Snape contributed equally and have joint second authorshi

Adaptive active appearance model with incremental learning

The active appearance model (AAM) is a well-known model that can represent a non-rigid object like the face effectively. However, the AAM often fails to converge correctly when the illumination conditions of face images change largely because it uses a set of fixed appearance basis vectors that are usually obtained in a training phase. To overcome this problem, we propose an adaptive AAM that updates the appearance basis vectors with the current face image by the incremental principal component analysis (PCA). However, the update of the appearance basis vectors with ill-fitted face images can worsen the AAM fitting to the forthcoming face images. To avoid this situation, we devise a conditional update method that updates the appearance basis vectors when the AAM fitting is good and the AAM reconstruction error is large. We evaluate the goodness of AAM fitting in terms of the number of outliers. When the AAM fitting is good we update the online appearance model (OAM) parameters, where the OAM is taken to keep the variation of input face image continuously, and also evaluate the goodness of the appearance basis vectors in terms of the magnitude of AAM reconstruction error. When the appearance basis vectors of the current AAM produces a large AAM reconstruction error. we update the appearance basis vectors using the incremental PCA. The proposed conditional update of the appearance basis vectors stabilizes the AAM fitting and improves the face tracking performance especially when the illumination condition changes very dynamically. Experimental results show that the adaptive AAM is superior to the conventional AAM in terms of the occurrence rate of fitting error and the fitting accuracy. (c) 2008 Elsevier B.V. All rights reserved.X1126sciescopu