Beyond PCA: Deep Learning Approaches for Face Modeling and Aging

Modeling faces with large variations has been a challenging task in computer vision. These variations such as expressions, poses and occlusions are usually complex and non-linear. Moreover, new facial images also come with their own characteristic artifacts greatly diverse. Therefore, a good face modeling approach needs to be carefully designed for flexibly adapting to these challenging issues. Recently, Deep Learning approach has gained significant attention as one of the emerging research topics in both higher-level representation of data and the distribution of observations. Thanks to the nonlinear structure of deep learning models and the strength of latent variables organized in hidden layers, it can efficiently capture variations and structures in complex data. Inspired by this motivation, we present two novel approaches, i.e. Deep Appearance Models (DAM) and Robust Deep Appearance Models (RDAM), to accurately capture both shape and texture of face images under large variations. In DAM, three crucial components represented in hierarchical layers are modeled using Deep Boltzmann Machines (DBM) to robustly capture the variations of facial shapes and appearances. DAM has shown its potential in inferencing a representation for new face images under various challenging conditions. An improved version of DAM, named Robust DAM (RDAM), is also introduced to better handle the occluded face areas and, therefore, produces more plausible reconstruction results. These proposed approaches are evaluated in various applications to demonstrate their robustness and capabilities, e.g. facial super-resolution reconstruction, facial off-angle reconstruction, facial occlusion removal and age estimation using challenging face databases: Labeled Face Parts in the Wild (LFPW), Helen and FG-NET. Comparing to classical and other deep learning based approaches, the proposed DAM and RDAM achieve competitive results in those applications, thus this showed their advantages in handling occlusions, facial representation, and reconstruction. In addition to DAM and RDAM that are mainly used for modeling single facial image, the second part of the thesis focuses on novel deep models, i.e. Temporal Restricted Boltzmann Machines (TRBM) and tractable Temporal Non-volume Preserving (TNVP) approaches, to further model face sequences. By exploiting the additional temporal relationships presented in sequence data, the proposed models have their advantages in predicting the future of a sequence from its past. In the application of face age progression, age regression, and age-invariant face recognition, these models have shown their potential not only in efficiently capturing the non-linear age related variance but also producing a smooth synthesis in age progression across faces. Moreover, the structure of TNVP can be transformed into a deep convolutional network while keeping the advantages of probabilistic models with tractable log-likelihood density estimation. The proposed approach is evaluated in terms of synthesizing age-progressed faces and cross-age face verification. It consistently shows the state-of-the-art results in various face aging databases, i.e. FG-NET, MORPH, our collected large-scale aging database named AginG Faces in the Wild (AGFW), and Cross-Age Celebrity Dataset (CACD). A large-scale face verification on Megaface challenge 1 is also performed to further show the advantages of our proposed approach

Exemplar-based Age Progression Prediction in Children Faces

[[abstract]]本論文的研究目的在於發展一套能夠隨著兒童年齡成長合成兒童面貌的預測系統，期望能協助尋找失蹤兒童。論文中提出來的方法由多個模組整合而成，包含特徵抽取、相似度計算及人臉合成等模組，以預測在不同年齡的人臉面貌。當輸入一張失蹤兒童的影像時，我們會先抽取出他的臉部特徵，接著以兩種不同的距離計算方式，稱為根據曲率加 (Curvature-Weighted) 以及扭曲能量 (Bending Energy) 的距離計算以及基於學習的馬氏距離 (Learning-Based Mahalanobis Distance) 計算，為人臉每個特徵在資料庫同年齡層的資料中尋找相似的他人特徵，接著使用這些擁有相似特徵孩童的生長曲線來預測失蹤兒童在不同年齡的臉部特徵，最後使用將扭曲能量 (Bending Energy) 最小化的薄板曲線法 (Thin Plate Spline Method，TPS) 把預測出來的臉部特徵合併成一張完整的臉。實驗部分我們測試了本篇論文所提出的方法並應用在多個不同的人臉上，而實驗結果顯示這個方法預測出來的人臉確實與成長後真實的人臉有極高的相似度。[[abstract]]This work aims to develop a system for predicting age progression in children faces. Age progression prediction in children faces is critical to assist missing children searching. An integral module including feature extraction, distance measurement, and face synthesis is devised in this thesis to predict faces at different ages. In the proposed method, two different distance measures, namely the learning-based Mahalanobis distance and the curvature-weighted plus bending-energy distance, are employed for selecting similar facial components from an aging database. The growth curves of each facial component are used to predict the shape, size, and location of each component at a different age. Thin plate spline method is employed to synthesize faces by minimizing the bending energy. Experiments have been conducted to test the proposed method with various subjects. The experimental results show that the proposed method is very promissing.[[note]]碩