Investigating face perception in humans and DCNNs

This thesis aims to compare strengths and weaknesses of AI and humans performing face identification tasks, and to use recent advances in machine-learning to develop new techniques for understanding face identity processing. By better understanding underlying processing differences between Deep Convolutional Neural Networks (DCNNs) and humans, it can help improve the ways in which AI technology is used to support human decision-making and deepen understanding of face identity processing in humans and DCNNs. In Chapter 2, I test how the accuracy of humans and DCNNs is affected by image quality and find that humans and DCNNs are affected differently. This has important applied implications, for example, when identifying faces from poor-quality imagery in police investigations, and also points to different processing strategies used by humans and DCNNs. Given these diverging processing strategies, in Chapter 3, I investigate the potential for human and DCNN decisions to be combined in face identification decisions. I find a large overall benefit of 'fusing' algorithm and human face identity judgments, and that this depends on the idiosyncratic accuracy and response patterns of the particular DCNNs and humans in question. This points to new optimal ways that individual humans and DCNNs can be aggregated to improve the accuracy of face identity decisions in applied settings. Building on my background in computer vision, in Chapters 4 and 5, I then aim to better understand face information sampling by humans using a novel combination of eye-tracking and machine-learning approaches. In chapter 4, I develop exploratory methods for studying individual differences in face information sampling strategies. This reveals differences in the way that 'super-recognisers' sample face information compared to typical viewers. I then use DCNNs to assess the computational value of the face information sampled by these two groups of human observers, finding that sampling by 'super-recognisers' contains more computationally valuable face identity information. In Chapter 5, I develop a novel approach to measuring fixations to people in unconstrained natural settings by combining wearable eye-tracking technology with face and body detection algorithms. Together, these new approaches provide novel insight into individual differences in face information sampling, both when looking at faces in lab-based tasks performed on computer monitors and when looking at faces 'in the wild'

The effect of comparison on the perceived similarity of complex visual stimuli

Comparison has been shown to decrease the perceived similarity of artificial face stimuli that are difficult to discriminate (Mundy, Honey, & Dwyer, 2007). This thesis presents seven experiments that examined the effect of comparison on the perceived similarity of a wider range of unfamiliar face stimuli. Participants were asked to compare two faces before either rating their perceived similarity, or deciding whether they are the same person. In the first five experiments participants were shown face pairs that ranged in phenotypic similarity—the degree to which the two faces look alike. With the exception of highly similar face morphs comparison was found to increase the perceived similarity of both phenotypically similar and dissimilar face pairs, relative to a no-comparison control. This finding suggests that for most naturally occurring face stimuli, comparison results in an increase in perceived similarity. In the last two experiments the quality of one of the stimuli in each pair was degraded to simulate the effects of poor quality video footage. A comparison-related decrease in perceived similarity was found in both experiments. This finding suggests that pictorial differences between face stimuli—including differences in image quality, camera distance and lighting, variations in pose and facial expression, and the presence of disguises such as hats and sunglasses—play an important role in mediating the effect of comparison on perceived similarity