Semi-supervised auto-encoder for facial attributes recognition

The particularity of our faces encourages many researchers to exploit their features in different domains such as user identification, behaviour analysis, computer technology, security, and psychology. In this paper, we present a method for facial attributes analysis. The work addressed to analyse facial images and extract features in the purpose to recognize demographic attributes: age, gender, and ethnicity (AGE). In this work, we exploited the robustness of deep learning (DL) using an updating version of autoencoders called the deep sparse autoencoder (DSAE). In this work we used a new architecture of DSAE by adding the supervision to the classic model and we control the overfitting problem by regularizing the model. The pass from DSAE to the semi-supervised autoencoder (DSSAE) facilitates the supervision process and achieves an excellent performance to extract features. In this work we focused to estimate AGE jointly. The experiment results show that DSSAE is created to recognize facial features with high precision. The whole system achieves good performance and important rates in AGE using the MORPH II databas

Bias in Deep Learning and Applications to Face Analysis

Deep learning has fostered the progress in the field of face analysis, resulting in the integration of these models in multiple aspects of society. Even though the majority of research has focused on optimizing standard evaluation metrics, recent work has exposed the bias of such algorithms as well as the dangers of their unaccountable utilization.n this thesis, we explore the bias of deep learning models in the discriminative and the generative setting. We begin by investigating the bias of face analysis models with regards to different demographics. To this end, we collect KANFace, a large-scale video and image dataset of faces captured ``in-the-wild’'. The rich set of annotations allows us to expose the demographic bias of deep learning models, which we mitigate by utilizing adversarial learning to debias the deep representations. Furthermore, we explore neural augmentation as a strategy towards training fair classifiers. We propose a style-based multi-attribute transfer framework that is able to synthesize photo-realistic faces of the underrepresented demographics. This is achieved by introducing a multi-attribute extension to Adaptive Instance Normalisation that captures the multiplicative interactions between the representations of different attributes. Focusing on bias in gender recognition, we showcase the efficacy of the framework in training classifiers that are more fair compared to generative and fairness-aware methods.In the second part, we focus on bias in deep generative models. In particular, we start by studying the generalization of generative models on images of unseen attribute combinations. To this end, we extend the conditional Variational Autoencoder by introducing a multilinear conditioning framework. The proposed method is able to synthesize unseen attribute combinations by modeling the multiplicative interactions between the attributes. Lastly, in order to control protected attributes, we investigate controlled image generation without training on a labelled dataset. We leverage pre-trained Generative Adversarial Networks that are trained in an unsupervised fashion and exploit the clustering that occurs in the representation space of intermediate layers of the generator. We show that these clusters capture semantic attribute information and condition image synthesis on the cluster assignment using Implicit Maximum Likelihood Estimation.Open Acces

First impressions: A survey on vision-based apparent personality trait analysis

© 2019 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes,creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works.Personality analysis has been widely studied in psychology, neuropsychology, and signal processing fields, among others. From the past few years, it also became an attractive research area in visual computing. From the computational point of view, by far speech and text have been the most considered cues of information for analyzing personality. However, recently there has been an increasing interest from the computer vision community in analyzing personality from visual data. Recent computer vision approaches are able to accurately analyze human faces, body postures and behaviors, and use these information to infer apparent personality traits. Because of the overwhelming research interest in this topic, and of the potential impact that this sort of methods could have in society, we present in this paper an up-to-date review of existing vision-based approaches for apparent personality trait recognition. We describe seminal and cutting edge works on the subject, discussing and comparing their distinctive features and limitations. Future venues of research in the field are identified and discussed. Furthermore, aspects on the subjectivity in data labeling/evaluation, as well as current datasets and challenges organized to push the research on the field are reviewed.Peer ReviewedPostprint (author's final draft

Age & Gender Recognition in The Wild

L'estimació de grups demogràfics a partir d'imatges, i en particular pel que fa a l'estimació d'edat i sexe, és un sector amb un ampli ventall d'aplicacions. Tanmateix, l'estat de l'art actual està poc encarat a escenaris realistes que no contemplen cap mena de restriccions, la qual cosa fa que els seus mètodes siguin inservibles per certs tipus de dades de la vida real. Aquesta tesi analitza la qüestió de la predicció robusta per sexe i edat, i proposa un nou paradigma per construir un marc de treball alternatiu des d'on desenvolupar mètodes més capaços d'actuar en situacions realistes. En concret, es demostra empíricament com l'estat de l'art basat en trets facials no és capaç d'actuar al nostre conjunt de dades que representen aquestes situacions realistes, i presentem un mètode basat en Xarxes Neuronals Profundes (DNNs, per la seva abreviació en anglès) que actua com un model de predicció conjunta, incloent-hi prediccions fetes a partir de característiques extretes de tot el cos a més a més de les aconseguides a través del rostre. Això permet al model actuar quan les cares són poc visibles o estan obstruïdes, i aprofitar-se de la informació addicional quan aquestes són visibles. El sistema presentat combina diversos models aplicats en fred, com per exemple RetinaFace i ShuffleNet per tasques facials, i una Faster R-CNN pre-entrenada amb COCO amb una ResNet com a model vertebral per detecció humana. Per la meva part, també s'ha entrenat un mòdul per predir sexe i edat a partir de deteccions corporals, on es fa servir EfficientNet com a vertebral. Consegüentment, s'ha demostrat que els models basats en cos tenen la capacitat de ser més resilients.The estimation of demographics from image data, and in particular of gender and age, is a subject with an extensive amount of applications. However, current state-of-the-art is not entirely focused on realistic and unconstrained scenarios, which makes those approaches unusable for certain real-life settings. This thesis analyzes the issue of robust age and gender prediction, and proposes a new paradigm to build upon an alternative framework from which methods that are more capable in realistic situations can be developed. Namely, we present a method based on Deep Neural Networks (DNNs) that acts as an ensemble model, including predictions from both corporal and facial features. Thus, our model can act both when faces are not very visible or are occluded, and can take advantage of the extra information when they are visible. The system presented combines multiple off-the-shelf models such as RetinaFace and ShuffleNet for facial tasks, and Faster R-CNN with ResNet backbone pre-trained on COCO for human detection. From my side, a module was trained to predict gender and age based on body detections, where EfficientNet is used as backbone. Consequently, it was demonstrated that body-based models have the capacity to be more resilient

How Technology Impacts and Compares to Humans in Socially Consequential Arenas

One of the main promises of technology development is for it to be adopted by people, organizations, societies, and governments -- incorporated into their life, work stream, or processes. Often, this is socially beneficial as it automates mundane tasks, frees up more time for other more important things, or otherwise improves the lives of those who use the technology. However, these beneficial results do not apply in every scenario and may not impact everyone in a system the same way. Sometimes a technology is developed which produces both benefits and inflicts some harm. These harms may come at a higher cost to some people than others, raising the question: {\it how are benefits and harms weighed when deciding if and how a socially consequential technology gets developed?} The most natural way to answer this question, and in fact how people first approach it, is to compare the new technology to what used to exist. As such, in this work, I make comparative analyses between humans and machines in three scenarios and seek to understand how sentiment about a technology, performance of that technology, and the impacts of that technology combine to influence how one decides to answer my main research question.Comment: Doctoral thesis proposal. arXiv admin note: substantial text overlap with arXiv:2110.08396, arXiv:2108.12508, arXiv:2006.1262