Bias in Deep Learning and Applications to Face Analysis

Deep learning has fostered the progress in the field of face analysis, resulting in the integration of these models in multiple aspects of society. Even though the majority of research has focused on optimizing standard evaluation metrics, recent work has exposed the bias of such algorithms as well as the dangers of their unaccountable utilization.n this thesis, we explore the bias of deep learning models in the discriminative and the generative setting. We begin by investigating the bias of face analysis models with regards to different demographics. To this end, we collect KANFace, a large-scale video and image dataset of faces captured ``in-the-wild’'. The rich set of annotations allows us to expose the demographic bias of deep learning models, which we mitigate by utilizing adversarial learning to debias the deep representations. Furthermore, we explore neural augmentation as a strategy towards training fair classifiers. We propose a style-based multi-attribute transfer framework that is able to synthesize photo-realistic faces of the underrepresented demographics. This is achieved by introducing a multi-attribute extension to Adaptive Instance Normalisation that captures the multiplicative interactions between the representations of different attributes. Focusing on bias in gender recognition, we showcase the efficacy of the framework in training classifiers that are more fair compared to generative and fairness-aware methods.In the second part, we focus on bias in deep generative models. In particular, we start by studying the generalization of generative models on images of unseen attribute combinations. To this end, we extend the conditional Variational Autoencoder by introducing a multilinear conditioning framework. The proposed method is able to synthesize unseen attribute combinations by modeling the multiplicative interactions between the attributes. Lastly, in order to control protected attributes, we investigate controlled image generation without training on a labelled dataset. We leverage pre-trained Generative Adversarial Networks that are trained in an unsupervised fashion and exploit the clustering that occurs in the representation space of intermediate layers of the generator. We show that these clusters capture semantic attribute information and condition image synthesis on the cluster assignment using Implicit Maximum Likelihood Estimation.Open Acces

The Curvilinear Relationship between Age and Emotional Aperture : The Moderating Role of Agreeableness

The capability to correctly recognize collective emotion expressions (i.e., emotional aperture) is crucial for effective social and work-related interactions. Yet, little remains known about the antecedents of this ability. The present study therefore aims to shed new light onto key aspects that may promote or diminish an individual’s emotional aperture. We examine the role of age for this ability in an online sample of 181 participants (with an age range of 18 to 72 years, located in Germany), and we investigate agreeableness as a key contingency factor. Among individuals with lower agreeableness, on the one hand, our results indicate a curvilinear relationship between age and emotional aperture, such that emotional aperture remains at a relatively high level until these individuals’ middle adulthood (with a slight increase until their late 30s) and declines afterwards. Individuals with higher agreeableness, on the other hand, exhibit relatively high emotional aperture irrespective of their age. Together, these findings offer new insights for the emerging literature on emotional aperture, illustrating that specific demographic and personality characteristics may jointly shape such collective emotion recognition

The Curvilinear Relationship between Age and Emotional Aperture : The Moderating Role of Agreeableness

The capability to correctly recognize collective emotion expressions (i.e., emotional aperture) is crucial for effective social and work-related interactions. Yet, little remains known about the antecedents of this ability. The present study therefore aims to shed new light onto key aspects that may promote or diminish an individual’s emotional aperture. We examine the role of age for this ability in an online sample of 181 participants (with an age range of 18 to 72 years, located in Germany), and we investigate agreeableness as a key contingency factor. Among individuals with lower agreeableness, on the one hand, our results indicate a curvilinear relationship between age and emotional aperture, such that emotional aperture remains at a relatively high level until these individuals’ middle adulthood (with a slight increase until their late 30s) and declines afterwards. Individuals with higher agreeableness, on the other hand, exhibit relatively high emotional aperture irrespective of their age. Together, these findings offer new insights for the emerging literature on emotional aperture, illustrating that specific demographic and personality characteristics may jointly shape such collective emotion recognition

Role of Artificial Intelligence (AI) art in care of ageing society: focus on dementia

open access articleBackground: Art enhances both physical and mental health wellbeing. The health benefits include reduction in blood pressure, heart rate, pain perception and briefer inpatient stays, as well as improvement of communication skills and self-esteem. In addition to these, people living with dementia benefit from reduction of their noncognitive, behavioural changes, enhancement of their cognitive capacities and being socially active. Methods: The current study represents a narrative general literature review on available studies and knowledge about contribution of Artificial Intelligence (AI) in creative arts. Results: We review AI visual arts technologies, and their potential for use among people with dementia and care, drawing on similar experiences to date from traditional art in dementia care. Conclusion: The virtual reality, installations and the psychedelic properties of the AI created art provide a new venue for more detailed research about its therapeutic use in dementia

Pluralistic Aging Diffusion Autoencoder

Face aging is an ill-posed problem because multiple plausible aging patterns may correspond to a given input. Most existing methods often produce one deterministic estimation. This paper proposes a novel CLIP-driven Pluralistic Aging Diffusion Autoencoder (PADA) to enhance the diversity of aging patterns. First, we employ diffusion models to generate diverse low-level aging details via a sequential denoising reverse process. Second, we present Probabilistic Aging Embedding (PAE) to capture diverse high-level aging patterns, which represents age information as probabilistic distributions in the common CLIP latent space. A text-guided KL-divergence loss is designed to guide this learning. Our method can achieve pluralistic face aging conditioned on open-world aging texts and arbitrary unseen face images. Qualitative and quantitative experiments demonstrate that our method can generate more diverse and high-quality plausible aging results.Comment: Accepted by ICCV 202

Chapter 26: the temporal dynamics of emotional responding: implications for well-being and health from the MIDUS

In this chapter findings are reviewed from the MIDUS Neuroscience Project that underscore the value in examining the temporal dynamics of responses to brief emotional provocation for understanding linkages among emotions and factors contributing to health and well-being across the MIDUS study. This rich dataset has allowed the exploration of associations between individual differences in the affective chronometry of negative and positive emotional responses in vulnerable vs. resilient profiles. Findings from functional magnetic resonance imaging (fMRI) as well as electromyographic recordings (EMG) of the facial muscles to objectively measure emotional responses demonstrate that the temporal dynamics of emotional responses to affective stimuli are associated with aging, personality, psychopathology, stress exposure, biomarkers, and well-being. Overall, these findings suggest that variation in health and well-being are differentially predicted by specific temporal parameters of the emotional response, such as the magnitude of the immediate response to the presence of a stimulus (i.e., reactivity), residual activity and its duration after stimulus offset (i.e., recovery), as well as the change in response – or habituation - across repeated presentations of similarly-valenced stimuli. Besides the seemingly obvious import of recovering quickly from negative or unpleasant provocations, the chronometry of positive emotional responses appears to be particularly vital for determining how emotional processes may take a physiological toll or promote resiliency in the face of stress and disease. By examining such temporal dynamics in response to affective stimuli in MIDUS, a better understanding of the brain-behavior associations underlying emotion, and how emotions “get under the skin” to impact well-being and health across the lifespan is gained

Precision sketching with de-aging networks in forensics

Addressing the intricacies of facial aging in forensic facial recognition, traditional sketch portraits often fall short in precision. This study introduces a pioneering system that seamlessly integrates a de-aging module and a sketch generator module to overcome the limitations inherent in existing methodologies. The de-aging module utilizes a deepfake-based neural network to rejuvenate facial features, while the sketch generator module leverages a pix2pix-based Generative Adversarial Network (GAN) for the generation of lifelike sketches. Comprehensive evaluations on the CUHK and AR datasets underscore the system’s superior efficiency. Significantly, comprehensive testing reveals marked enhancements in realism during the training process, demonstrated by notable reductions in Frechet Inception Distance (FID) scores (41.7 for CUHK, 60.2 for AR), augmented Structural Similarity Index (SSIM) values (0.789 for CUHK, 0.692 for AR), and improved Peak Signal-to-Noise Ratio (PSNR) metrics (20.26 for CUHK, 19.42 for AR). These findings underscore substantial advancements in the accuracy and reliability of facial recognition applications. Importantly, the system, proficient in handling diverse facial characteristics across gender, race, and culture, produces both composite and hand-drawn sketches, surpassing the capabilities of current state-of-the-art methods. This research emphasizes the transformative potential arising from the integration of de-aging networks with sketch generation, particularly for age-invariant forensic applications, and highlights the ongoing necessity for innovative developments in de-aging technology with broader societal and technological implications

High-Quality Face Caricature via Style Translation

Caricature is an exaggerated form of artistic portraiture that accentuates unique yet subtle characteristics of human faces. Recently, advancements in deep end-to-end techniques have yielded encouraging outcomes in capturing both style and elevated exaggerations in creating face caricatures. Most of these approaches tend to produce cartoon-like results that could be more practical for real-world applications. In this study, we proposed a high-quality, unpaired face caricature method that is appropriate for use in the real world and uses computer vision techniques and GAN models. We attain the exaggeration of facial features and the stylization of appearance through a two-step process: Face caricature generation and face caricature projection. The face caricature generation step creates new caricature face datasets from real images and trains a generative model using the real and newly created caricature datasets. The Face caricature projection employs an encoder trained with real and caricature faces with the pretrained generator to project real and caricature faces. We perform an incremental facial exaggeration from the real image to the caricature faces using the encoder and generator's latent space. Our projection preserves the facial identity, attributes, and expressions from the input image. Also, it accounts for facial occlusions, such as reading glasses or sunglasses, to enhance the robustness of our model. Furthermore, we conducted a comprehensive comparison of our approach with various state-of-the-art face caricature methods, highlighting our process's distinctiveness and exceptional realism.Comment: 14 pages, 21 figure