Image-based family verification in the wild

Facial image analysis has been an important subject of study in the communities of pat- tern recognition and computer vision. Facial images contain much information about the person they belong to: identity, age, gender, ethnicity, expression and many more. For that reason, the analysis of facial images has many applications in real world problems such as face recognition, age estimation, gender classification or facial expression recognition. Visual kinship recognition is a new research topic in the scope of facial image analysis. It is essential for many real-world applications. However, nowadays there exist only a few practical vision systems capable to handle such tasks. Hence, vision technology for kinship-based problems has not matured enough to be applied to real- world problems. This leads to a concern of unsatisfactory performance when attempted on real-world datasets. Kinship verification is to determine pairwise kin relations for a pair of given images. It can be viewed as a typical binary classification problem, i.e., a face pair is either related by kinship or it is not. Prior research works have addressed kinship types for which pre-existing datasets have provided images, annotations and a verification task protocol. Namely, father-son, father-daughter, mother-son and mother-daughter. The main objective of this Master work is the study and development of feature selection and fusion for the problem of family verification from facial images. To achieve this objective, there is a main tasks that can be addressed: perform a compara- tive study on face descriptors that include classic descriptors as well as deep descriptors. The main contributions of this Thesis work are: 1. Studying the state of the art of the problem of family verification in images. 2. Implementing and comparing several criteria that correspond to different face rep- resentations (Local Binary Patterns (LBP), Histogram Oriented Gradients (HOG), deep descriptors)

Image-based family verification in the wild

Facial image analysis has been an important subject of study in the communities of pat- tern recognition and computer vision. Facial images contain much information about the person they belong to: identity, age, gender, ethnicity, expression and many more. For that reason, the analysis of facial images has many applications in real world problems such as face recognition, age estimation, gender classification or facial expression recognition. Visual kinship recognition is a new research topic in the scope of facial image analysis. It is essential for many real-world applications. However, nowadays there exist only a few practical vision systems capable to handle such tasks. Hence, vision technology for kinship-based problems has not matured enough to be applied to real- world problems. This leads to a concern of unsatisfactory performance when attempted on real-world datasets. Kinship verification is to determine pairwise kin relations for a pair of given images. It can be viewed as a typical binary classification problem, i.e., a face pair is either related by kinship or it is not. Prior research works have addressed kinship types for which pre-existing datasets have provided images, annotations and a verification task protocol. Namely, father-son, father-daughter, mother-son and mother-daughter. The main objective of this Master work is the study and development of feature selection and fusion for the problem of family verification from facial images. To achieve this objective, there is a main tasks that can be addressed: perform a compara- tive study on face descriptors that include classic descriptors as well as deep descriptors. The main contributions of this Thesis work are: 1. Studying the state of the art of the problem of family verification in images. 2. Implementing and comparing several criteria that correspond to different face rep- resentations (Local Binary Patterns (LBP), Histogram Oriented Gradients (HOG), deep descriptors)

Improving Person-Independent Facial Expression Recognition Using Deep Learning

Over the past few years, deep learning, e.g., Convolutional Neural Networks (CNNs) and Generative Adversarial Networks (GANs), have shown promise on facial expression recog- nition. However, the performance degrades dramatically especially in close-to-real-world settings due to high intra-class variations and high inter-class similarities introduced by subtle facial appearance changes, head pose variations, illumination changes, occlusions, and identity-related attributes, e.g., age, race, and gender. In this work, we developed two novel CNN frameworks and one novel GAN approach to learn discriminative features for facial expression recognition. First, a novel island loss is proposed to enhance the discriminative power of learned deep features. Specifically, the island loss is designed to reduce the intra-class variations while enlarging the inter-class differences simultaneously. Experimental results on three posed facial expression datasets and, more importantly, two spontaneous facial expression datasets have shown that the proposed island loss outperforms the baseline CNNs with the traditional softmax loss or the center loss and achieves better or at least comparable performance compared with the state-of-the-art methods. Second, we proposed a novel Probabilistic Attribute Tree-CNN (PAT-CNN) to explic- itly deal with the large intra-class variations caused by identity-related attributes. Specif- ically, a novel PAT module with an associated PAT loss was proposed to learn features in a hierarchical tree structure organized according to identity-related attributes, where the final features are less affected by the attributes. We further proposed a semi-supervised strategy to learn the PAT-CNN from limited attribute-annotated samples to make the best use of available data. Experimental results on three posed facial expression datasets as well as four spontaneous facial expression datasets have demonstrated that the proposed PAT- CNN achieves the best performance compared with state-of-the-art methods by explicitly modeling attributes. Impressively, the PAT-CNN using a single model achieves the best performance on the SFEW test dataset, compared with the state-of-the-art methods using an ensemble of hundreds of CNNs. Last, we present a novel Identity-Free conditional Generative Adversarial Network (IF- GAN) to explicitly reduce high inter-subject variations caused by identity-related attributes, e.g., age, race, and gender, for facial expression recognition. Specifically, for any given in- put facial expression image, a conditional generative model was developed to transform it to an “average” identity expressive face with the same expression as the input face image. Since the generated images have the same synthetic “average” identity, they differ from each other only by the displayed expressions and thus can be used for identity-free facial expression classification. In this work, an end-to-end system was developed to perform facial expression generation and facial expression recognition in the IF-GAN framework. Experimental results on four well-known facial expression datasets including a sponta- neous facial expression dataset have demonstrated that the proposed IF-GAN outperforms the baseline CNN model and achieves the best performance compared with the state-of- the-art methods for facial expression recognition

Automatic age estimation based on facial aging patterns

While recognition of most facial variations, such as identity, expression, and gender, has been extensively studied, automatic age estimation has rarely been explored. In contrast to other facial variations, aging variation presents several unique characteristics which make age estimation a challenging task. This paper proposes an automatic age estimation method named AGES (AGing pattErn Subspace). The basic idea is to model the aging pattern, which is defined as the sequence of a particular individual\u27s face images sorted in time order, by constructing a representative subspace. The proper aging pattern for a previously unseen face image is determined by the projection in the subspace that can reconstruct the face image with minimum reconstruction error, while the position of the face image in that aging pattern will then indicate its age. In the experiments, AGES and its variants are compared with the limited existing age estimation methods (WAS and AAS) and some well-established classification methods (kNN, BP, C4.5, and SVM). Moreover, a comparison with human perception ability on age is conducted. It is interesting to note that the performance of AGES is not only significantly better than that of all the other algorithms, but also comparable to that of the human observers.<br /

Comparison of filtering methods for extracting transient facial wrinkle features

Facial local features comprise an essential information to identify a personal characteristic such as age, gender, identity and expression. One of the facial local features is a wrinkle. Wrinkle is a small furrow or crease in the skin. Recently, wrinkle detection has become a topic of interest in computer vision, where many researchers developed applications like age estimation, face detection, expression recognition, facial digital beauty and etc. However, most of the research focused on permanent wrinkles instead of transient wrinkles. Transient wrinkle can be seen during the movement of facial muscle such as a facial expression. This paper presents a comparison of filtering method for extracting transient wrinkles features. The filters that have been selected are Gabor wavelet and Kirsch operator. The extracted features are the number of wrinkles, the maximum perimeter of wrinkle, the average perimeter of wrinkle, total perimeter of wrinkle, the maximum area of the wrinkle, and the total area of the wrinkle. A total of 60 sets of data extracted from Cohn-Kanade database, images from internet and self-images. These images contain weak and strong transient wrinkles at forehead region. Features selection and analysis has been done to select which feature extraction method produces better wrinkle features that can be used for the classification of wrinkle detection system. The results show that both Gabor and Kirsch methods are successful to extract transient wrinkle features, where both methods scored 100% accuracy in the classification with SVM. However, Gabor method is slightly better than Kirsch method in term of detecting weak wrinkles. The Kirsch method requires an additional noise filtering method to eliminate noise particles after the convolution of Kirsch’s kernel. In conclusion, Gabor method is more applicable to a variety of applications than Kirsch method