The Profiling Potential of Computer Vision and the Challenge of Computational Empiricism

Computer vision and other biometrics data science applications have commenced a new project of profiling people. Rather than using 'transaction generated information', these systems measure the 'real world' and produce an assessment of the 'world state' - in this case an assessment of some individual trait. Instead of using proxies or scores to evaluate people, they increasingly deploy a logic of revealing the truth about reality and the people within it. While these profiling knowledge claims are sometimes tentative, they increasingly suggest that only through computation can these excesses of reality be captured and understood. This article explores the bases of those claims in the systems of measurement, representation, and classification deployed in computer vision. It asks if there is something new in this type of knowledge claim, sketches an account of a new form of computational empiricism being operationalised, and questions what kind of human subject is being constructed by these technological systems and practices. Finally, the article explores legal mechanisms for contesting the emergence of computational empiricism as the dominant knowledge platform for understanding the world and the people within it

The Influence of Phenotypic Variation on Criminal Judgement

The purpose of this study was to investigate the influence of phenotypic variation on criminal judgment. This study had two phases. In the first phase, participants rated multiple headshot photographs on the degree to which African American men possess pronounced Afrocentric features (fuller lips, broader nose, curlier hair, darker skin, etc.). The race of the participants predicted 34.2% of the variance in average skin color ratings above all other variables. White participants rated the Black faces as darker than any other participants rated the same faces. Researchers used the faces rated least, average, and most prototypical of Blacks as the targets for a criminal vignette in phase two. Controlling for participant political ideology and race, target Black prototypicality had a main effect on recommended years for the defendant to serve (Ruby & Brigham, 1996). The most prototypical defendant was more likely sentenced to prison time followed by a period of probation and to serve approximately six more years in the adult correction system than the least or average prototypical defendants. Phenotypic variation was a leading factor in the criminal judgment of African American men along with perceptions of the defendant, attitudes towards the legal system and Black people, and social Black 2 contact. These results have implications for understanding the saliency of phenotypic variation on target judgment and reevaluating the criminal legal process

Designing Computer Agents with Facial Personality to Improve Human-Machine Collaboration

The development of computer agents to enhance human-computer interfaces is an evolving field of study. This study examined whether people perceive personality in static digital faces that portray expressions of emotion, and if the digital faces would influence human performance on a simple human-machine collaborative task. The first experiment measured user-perception of personality, based on the emotional expression in two sets of five static digital faces. The results from this first phase revealed that participants provided different ratings, of the Big-Five personality model sub-traits, based on the emotional expression of a static digital face. This indicates a perception of personality based on expression. The second experiment measured how faces with identified personality traits influence decision making in a simple collaborative task. The results revealed that the different faces did not have a significant impact on performance criteria. Results from this study indicated some isolated differences related to gender and nationality

Cluster-analytic classification of facial expressions using infrared measurements of facial thermal features

In previous research, scientists were able to use transient facial thermal features extracted from Thermal Infra-Red Images (TIRIs) for making binary distinction between the affective states. For example, thermal asymmetries localised in facial TIRIs have been used to distinguish anxiety and deceit. Since affective human-computer interaction would require machines to distinguish between the subtle facial expressions of affective states, computers’ able to make such binary distinctions would not suffice a robust human-computer interaction. This work, for the first time, uses affective-state-specific transient facial thermal features extracted from TIRIs to recognise a much wider range of facial expressions under a much wider range of conditions. Using infrared thermal imaging within the 8-14 μm, a database of 324 discrete, time-sequential, visible-spectrum and thermal facial images was acquired, representing different facial expressions from 23 participants in different situations. A facial thermal feature extraction and pattern classification approach was developed, refined and tested on various Gaussian mixture models constructed using the image database. Attempts were made to classify: neutral and pretended happy and sad faces; multiple positive and negative facial expressions; six (pretended) basic facial expressions; partially covered or occluded faces; and faces with evoked happiness, sadness, disgust and anger. The cluster-analytic classification in this work began by segmentation and detection of thermal faces in the acquired TIRIs. The affective-state-specific temperature distributions on the facial skin surface were realised through the pixel grey-level analysis. Examining the affectivestate- specific temperature variations within the selected regions of interest in the TIRIs led to the discovery of some significant Facial Thermal Feature Points (FTFPs) along the major facial muscles. Following a multivariate analysis of the Thermal Intensity values (TIVs) measured at the FTFPs, the TIRIs were represented along the Principal Components (PCs) of a covariance matrix. The resulting PCs were ranked in the order of their effectiveness in the between-cluster separation. Only the most effective PCs were retained to construct an optimised eigenspace. A supervised learning algorithm was invoked for linear subdivision of the optimised eigenspace. The statistical significance levels of the classification results were estimated for validating the discriminant functions. The main contribution of this research has been to show that: the infrared imaging of facial thermal features within the 8-14 μm bandwidth may be used to observe affective-state-specific thermal variations on the face; the pixel-grey level analysis of TIRIs can help localise FTFPs along the major facial muscles of the face; cluster-analytic classification of transient thermal features may help distinguish between the facial expressions of affective states in an optimized eigenspace of input thermal feature vectors. The Gaussian mixture model with one cluster per affect worked better for some facial expressions than others. This made the influence of the Gaussian mixture model structure on the accuracy of the classification results obvious. However, the linear discrimination and confusion patterns observed in this work were consistent with the ones reported in several earlier studies. This investigation also unveiled some important dimensions of the future research on use of facial thermal features in affective human-computer interaction.EThOS - Electronic Theses Online ServiceGBUnited Kingdo

Assessing the Boundary Conditions of the Own-Age and Own-Race Perceptual Bias for Faces

Our interactions with other people rely on our ability to perceive and distinguish faces based on snap decisions about their features. Past research has revealed that facial recognition is consistently better when the observer shares the same race as the person being identified or is roughly in the same age category (Meissner & Brigham, 2001; Rhodes & Anastasi, 2012). Although a misidentification can be irritating in daily life, high discriminability is especially important in situations where a misidentification could have drastic consequences, such as in eye witness testimony or during security checkpoints conducted by law enforcement. Although much research has been conducted to try to explain the cause of bias in favor of own-race and own-age faces, little is understood about the precise circumstances that give rise to these biases and when they begin to affect our perception or memory. I investigated the own-age bias (OAB) in younger and older adults and the own-race bias (ORB) in Caucasian and African American adults in a perceptual recognition task; the participant was shown a unique target face and immediately asked to respond to an array of faces and indicate if the target was present or absent before moving on to the next trial. The aim was to determine the factors that are influential in producing an in-group bias without a large memory load. I was interested in how set size, retention interval, and intervening distractor faces impact the OAB and the ORB. I looked at discriminability (d\u27) because it is a better measure of sensitivity than the hit rate alone and I also measured response latency. Another goal of this study was to ascertain that any observable perceptual biases result from processing facial features and not from any other characteristic of the photographs or differences in photograph quality across facial categories. To ensure that participants relied solely on facial features rather than hair cues, the facial stimuli used in this study were carefully selected and all hair was removed by cropping the faces into an oval shape. I found that Caucasian individuals had higher discriminability, but not faster reaction times, for Caucasian faces compared to African American faces; however, African American individuals did not show an ORB. I failed to find any evidence of the OAB in either young or older adults. Larger test set size, longer retention interval, and the addition of intervening distractor faces had a general negative effect on recognition and reaction times but did not exacerbate the ORB or OAB

Affective Computing

This book provides an overview of state of the art research in Affective Computing. It presents new ideas, original results and practical experiences in this increasingly important research field. The book consists of 23 chapters categorized into four sections. Since one of the most important means of human communication is facial expression, the first section of this book (Chapters 1 to 7) presents a research on synthesis and recognition of facial expressions. Given that we not only use the face but also body movements to express ourselves, in the second section (Chapters 8 to 11) we present a research on perception and generation of emotional expressions by using full-body motions. The third section of the book (Chapters 12 to 16) presents computational models on emotion, as well as findings from neuroscience research. In the last section of the book (Chapters 17 to 22) we present applications related to affective computing

Facial stereotypes

Tese de mestrado em Psicologia (Psicologia Cognitiva), apresentada à Universidade de Lisboa através da Faculdade de Psicologia e de Ciências da Educação, 2001The tendency to judge personality from facial appearance is very common and, for centuries, people have believed that character is revealed in the face. Some of the information that the human face conveys can be detected and interpreted usually with a great level of accuracy, such as age, gender or even some emotions. However, the inference of personality characteristics from the face proves to be less accurate, although still extremely common. Signs of the practice of physiognomy {face reading) can be identified over the years, and although that practice is not so popular nowadays, those beliefs are still present in the form of what can be called facial stereotypes. A literature review demonstrated that the not very broad amount of research that is available on this subject shows that people are extremely consistent in their judgements of other people's honesty, intelligence, personality traits, intentions, occupation and even political opinions, based on facial appearance. However, despite the observed consistency of these judgements, it is usually found that they are barely valid. Nevertheless, the observation that people tend to hold and apply facial stereotypes in a consistent manner has conferred importance to a better understanding about the underlying mechanisms of these stereotypes. Accordingly, the present research work has been focused on the study of the nature of the processes underlying the activation and application of social stereotypes based on facial appearance. This thesis is organised in three main sections. Section I comprises the introduction to the field of research and the review of the literature relevant to the area. Section II describes all the experimental procedures and results observed, and includes the individual discussions of each experiment. Finally, Section III is dedicated to the general discussion and conclusions, including as well some suggestions for further directions in future studies. After the brief and general introduction to the field of facial stereotypes outlined in Chapter 1, the detailed literature review starts with an overlook at the more important issues related to face processing in general (Chapter 2). Here, the main differences between object and face recognition are outlined, the most prominent models of face processing are reviewed, and neuropsychological and neurophysiological data are mentioned, highlighting the specific aspects involved in processing faces. In Chapter 3, a general review of the main issues concerning the study of stereotypes is carried out. The notion of stereotype has been a central concept in the domain of social psychology and social cognition over the last decades, and the main findings on this field have important implications for the study, in particular, of facial stereotypes. The more significant literature regarding the aspects of representation, formation and activation of stereotypes, and the processing of stereotype congruent and incongruent information is examined. Facial stereotypes are then the central topic of Chapter 4, which covers the relevant literature on this field. Despite the fact that both face processing and social stereotypes are issues extensively explored, the studies on facial stereotypes are not so abundant. In fact, the present research work can be considered innovative, in the sense of bringing on together the study of stereotypes and face perception, under the perspectives of cognitive and experimental psychology. Most of the literature concerned with physiognomic stereotypes has been more devoted to explore the issue of whether the inferences about personality that are made based on facial appearance are valid or not, and which are the characteristics of the face that lead to certain judgements. However, some methodological issues were sometimes raised with respect to some of those studies. The possible mediating mechanisms linking physical features and inferential responses are reviewed, as well as the main findings regarding the accuracy of face reading. A model of appearance-trait relations is mentioned, which comprises four possible causal routes to actual appearance-trait relations, and some overgeneralization effects in perceiving faces are discussed. Then, findings related to two of the stereotypes that are most widely discussed in the available literature are presented, namely the attractiveness stereotype and the babyfacedness stereotype. Some issues regarding the perception of intelligence from facial appearance are also highlighted, and some neuropsychological data supporting the importance of the face in social judgements are reported. Chapter 5 presents an overview of the present work, summarising the most relevant theoretical background, the aims of the present research work and a brief description of the main experimental procedures. In general, the investigation carried out was directed at the issue of whether different types of stereotypes are automatically activated whenever we look at a face, or require more deliberate evaluation. The facial stereotypes that have been addressed were related to attractiveness, intelligence and trustworthiness. The experiments were specifically designed to investigate the potential interference of the activation of social stereotypes, either in learning labels attached to male and female adult faces, or in the reaction times and response accuracy in an Irrelevant Feature Paradigm. The collection of the initial face database and the methodology used to obtain the ratings are fully described in Chapter 6. Satisfactory interrater correlations were observed, and none of the raters deviated significantly from the mean rating value for each stimulus. This analysis validated both the sample of faces that were collected and the ratings that were obtained. So, the facial stimuli for the subsequent experiments were selected based on this set of ratings. Experiment 1 (described in Chapter 7) was based on a learning paradigm and the results observed support the experimental predictions of preferential recall of stereotype congruent information, under fairly high load processing conditions, for all the three traits (attractiveness, intelligence and tasteworthiness). Given the experimental conditions, it was suggested that the stereotypes had been automatically activated and that their activation influenced the representation in memory of information that is associated with the stereotypes. These results also provide experimental support for the social reality of facial stereotypes. In Experiment 2 (covered by Chapter 8), a different experimental paradigm was used - an irrelevant feature paradigm, which is a kind of interference paradigm, where the main task was a gender decision task. In this experiment, a significant congruency effect was found only in the attractiveness condition. The fact that attractiveness is probably one of the most readily judged characteristics from facial appearance was taken to explain this observation. It was also suggested that this evidence gave further support to the biological and evolutionary perspectives on the importance of attractiveness. Its relevance probably contributes to a higher accessibility of the attractiveness stereotype, which would be more readily picked up and would have more automatic effects on people's reactions. Furthermore, it is also reasonable to consider that the gender decision task used in this experiment did not actually require the processing of the characteristics that are associated with the facial stereotypes, relying instead on different cues. Consequently, only a highly accessible stereotype as attractiveness would have a significant interference on task performance. Finally, Experiment 3 (described in Chapter 9) addressed the question of whether the perceived attractiveness of a face can influence the perception of other characteristics. The experimental paradigm was again the irrelevant feature paradigm, but this time attractiveness and intelligence were simultaneously manipulated when selecting the facial stimuli. The observed results suggested that the judgements of other characteristics, namely intelligence, can indeed be influenced by people's facial attractiveness, and that the effects of the activation of the intelligence facial stereotype might be mediated by the level of attractiveness. The observed effects of attractiveness on the perception of intelligence were in such a way that a face which looks intelligent, and which also looks unattractive will probably seem less intelligence. On the other hand, an unintelligent looking face that is at the same time attractive will probably look slightly more intelligent. Chapter 10 includes the general discussion of the more relevant experimental observations, summarises the main conclusions and presents some suggestions for further studies

Electrophysiological investigations of the timing of face processing

As important social stimuli, faces playa critical role in our lives. Much of our interaction with other people depends on our ability to recognize faces accurately. It has been proposed that face processing consists of different stages and interacts with other systems (Bruce & Young, 1986). At a perceptual level, the initial two stages, namely structural encoding and face recognition, are particularly relevant and are the focus of this dissertation. Event-related potentials (ERPs) are averaged EEG signals time-locked to a particular event (such as the presentation of a face). With their excellent temporal resolution, ERPs can provide important timing information about neural processes. Previous research has identified several ERP components that are especially related to face processing, including the N 170, the P2 and the N250. Their nature with respect to the stages of face processing is still unclear, and is examined in Studies 1 and 2. In Study 1, participants made gender decisions on a large set of female faces interspersed with a few male faces. The ERP responses to facial characteristics of the female faces indicated that the N 170 amplitude from each side of the head was affected by information from eye region and by facial layout: the right N 170 was affected by eye color and by face width, while the left N 170 was affected by eye size and by the relation between the sizes of the top and bottom parts of a face. In contrast, the P100 and the N250 components were largely unaffected by facial characteristics. These results thus provided direct evidence for the link between the N 170 and structural encoding of faces. In Study 2, focusing on the face recognition stage, we manipulated face identity strength by morphing individual faces to an "average" face. Participants performed a face identification task. The effect of face identity strength was found on the late P2 and the N250 components: as identity strength decreased from an individual face to the "average" face, the late P2 increased and the N250 decreased. In contrast, the P100, the N170 and the early P2 components were not affected by face identity strength. These results suggest that face recognition occurs after 200 ms, but not earlier. Finally, because faces are often associated with social information, we investigated in Study 3 how group membership might affect ERP responses to faces. After participants learned in- and out-group memberships of the face stimuli based on arbitrarily assigned nationality and university affiliation, we found that the N170 latency differentiated in-group and out-group faces, taking longer to process the latter. In comparison, without group memberships, there was no difference in N170 latency among the faces. This dissertation provides evidence that at a neural level, structural encoding of faces, indexed by the N170, occurs within 200 ms. Face recognition, indexed by the late P2 and the N250, occurs shortly afterwards between 200 and 300 ms. Social cognitive factors can also influence face processing. The effect is already evident as early as 130-200 ms at the structural encoding stage