Face beauty analysis via manifold based semi-supervised learning

Beauty has always played an important role in society, implicitly influencing the hu- man interactions of our daily lives and more significant aspects, such as the mate choice or job interviews. And now, with the progress made in deep learning and fea- ture extraction, automatic facial beauty analysis has become an emerging research topic too. However, the subjectivity of beauty still hinders the developement in this area, due to the cost of collecting reliable labeled data, since the beauty score of an individual has to be determined according to various raters. To address this problem, we study the performances of four different semi-supervised manifold based algorithms, which can take advantage of both labeled and unlabeled data in the training phase, and we use them in two different datasets: SCUT-FBP and M 2 B. The learning algorithms are Local and Global Consistency, Flexible Man- ifold Embedding and Kernel Flexible Manifold Embedding. There is an additional algorithm, which, unlike the rest of them, instead of performing classification, ob- tains a non-linear transformation of the data to make the classification easier. All of these algorithms were designed to work on discrete classes, but we perform regres- sion, where labels are real numbers. So the first step, in chapter 2, is to analyse how the algorithms can be adapted to regression and to hypothesize which problems we could be encountering in this process. Secondly, we empirically test them (chapter 3). The best results are obtained with KFME on both datasets, achieving a mean average error of 0.0104 (out of 1) and a Pearson correlation of 0.9782 on SCUT-FBP dataset. With respect to M 2 B dataset, a mean average error of 0.0697 and a Pear- son correlation of 0.7757 are achieved on eastern faces, while a mean average error of 0.0717 and a Pearson correlation of 0.7848 are achieved on western faces. This dissertation ends with a final chapter discussing the results and proposing new topics of study for future work

Multi-View Graph Fusion for Semi-Supervised Learning: Application to Image-Based Face Beauty Prediction

Facial Beauty Prediction (FBP) is an important visual recognition problem to evaluate the attractiveness of faces according to human perception. Most existing FBP methods are based on supervised solutions using geometric or deep features. Semi-supervised learning for FBP is an almost unexplored research area. In this work, we propose a graph-based semi-supervised method in which multiple graphs are constructed to find the appropriate graph representation of the face images (with and without scores). The proposed method combines both geometric and deep feature-based graphs to produce a high-level representation of face images instead of using a single face descriptor and also improves the discriminative ability of graph-based score propagation methods. In addition to the data graph, our proposed approach fuses an additional graph adaptively built on the predicted beauty values. Experimental results on the SCUTFBP-5500 facial beauty dataset demonstrate the superiority of the proposed algorithm compared to other state-of-the-art methods

On the use of high-order feature propagation in Graph Convolution Networks with Manifold Regularization

Graph Convolutional Networks (GCNs) have received a lot of attention in pattern recognition and machine learning. In this paper, we present a revisited scheme for the new method called "GCNs with Manifold Regularization" (GCNMR). While manifold regularization can add additional information, the GCN-based semi-supervised classification process cannot consider the full layer-wise structured information. Inspired by graph-based label propagation approaches, we will integrate high-order feature propagation into each GCN layer. High-order feature propagation over the graph can fully exploit the structured information provided by the latter at all the GCN's layers. It fully exploits the clustering assumption, which is valid for structured data but not well exploited in GCNs. Our proposed scheme would lead to more informative GCNs. Using the revisited model, we will conduct several semi-supervised classification experiments on public image datasets containing objects, faces and digits: Extended Yale, PF01, Caltech101 and MNIST. We will also consider three citation networks. The proposed scheme performs well compared to several semi-supervised methods. With respect to the recent GCNMR approach, the average improvements were 2.2%, 4.5%, 1.0% and 10.6% on Extended Yale, PF01, Caltech101 and MNIST, respectively.This work is supported in part by the University of the Basque Country UPV/EHU grant GIU19/027

Face beauty analysis via manifold based semi-supervised learning

Beauty has always played an important role in society, implicitly influencing the hu- man interactions of our daily lives and more significant aspects, such as the mate choice or job interviews. And now, with the progress made in deep learning and fea- ture extraction, automatic facial beauty analysis has become an emerging research topic too. However, the subjectivity of beauty still hinders the developement in this area, due to the cost of collecting reliable labeled data, since the beauty score of an individual has to be determined according to various raters. To address this problem, we study the performances of four different semi-supervised manifold based algorithms, which can take advantage of both labeled and unlabeled data in the training phase, and we use them in two different datasets: SCUT-FBP and M 2 B. The learning algorithms are Local and Global Consistency, Flexible Man- ifold Embedding and Kernel Flexible Manifold Embedding. There is an additional algorithm, which, unlike the rest of them, instead of performing classification, ob- tains a non-linear transformation of the data to make the classification easier. All of these algorithms were designed to work on discrete classes, but we perform regres- sion, where labels are real numbers. So the first step, in chapter 2, is to analyse how the algorithms can be adapted to regression and to hypothesize which problems we could be encountering in this process. Secondly, we empirically test them (chapter 3). The best results are obtained with KFME on both datasets, achieving a mean average error of 0.0104 (out of 1) and a Pearson correlation of 0.9782 on SCUT-FBP dataset. With respect to M 2 B dataset, a mean average error of 0.0697 and a Pear- son correlation of 0.7757 are achieved on eastern faces, while a mean average error of 0.0717 and a Pearson correlation of 0.7848 are achieved on western faces. This dissertation ends with a final chapter discussing the results and proposing new topics of study for future work

A Flexible Semi-supervised Feature Extraction Method for Image Classification

Abstract. This paper proposes a novel discriminant semi-supervised feature extraction for generic classification and recognition tasks. The paper has two main contributions. First, we propose a flexible linear semi-supervised feature extraction method that seeks a non-linear subspace that is close to a linear one. The proposed method is based on a criterion that simultaneously exploits the discrimination information provided by the labeled samples, maintains the graph-based smoothness associated with all samples, regularizes the complexity of the linear transform, and minimizes the discrepancy between the unknown linear regression and the unknown non-linear projection. Second, we provide extensive exper-iments on four benchmark databases in order to study the performance of the proposed method. These experiments demonstrate much improve-ment over the state-of-the-art algorithms that are either based on label propagation or semi-supervised graph-based embedding.

Contribution to Graph-based Manifold Learning with Application to Image Categorization.

122 pLos algoritmos de aprendizaje de variedades basados en grafos (Graph,based manifold) son técnicas que han demostrado ser potentes herramientas para la extracción de características y la reducción de la dimensionalidad en los campos de reconomiento de patrones, visión por computador y aprendizaje automático. Estos algoritmos utilizan información basada en las similitudes de pares de muestras y del grafo ponderado resultante para revelar la estructura geométrica intrínseca de la variedad