Convolutional neural net face recognition works in non-human-like ways

Convolutional neural networks (CNNs) give the state-of-the-art performance in many pattern recognition problems but can be fooled by carefully crafted patterns of noise. We report that CNN face recognition systems also make surprising ‘errors'. We tested six commercial face recognition CNNs and found that they outperform typical human participants on standard face-matching tasks. However, they also declare matches that humans would not, where one image from the pair has been transformed to appear a different sex or race. This is not due to poor performance; the best CNNs perform almost perfectly on the human face-matching tasks, but also declare the most matches for faces of a different apparent race or sex. Although differing on the salience of sex and race, humans and computer systems are not working in completely different ways. They tend to find the same pairs of images difficult, suggesting some agreement about the underlying similarity space

Differential aspects of attention predict the depth of visual working memory encoding: Evidence from pupillometry

What determines how much one encodes into visual working memory? Traditionally, encoding depth is considered to be indexed by spatiotemporal properties of gaze, such as gaze position and dwell time. Although these properties inform about where and how long one looks, they do not necessarily inform about the current arousal state or how strongly attention is deployed to facilitate encoding. Here, we found that two types of pupillary dynamics predict how much information is encoded during a copy task. The task involved encoding a spatial pattern of multiple items for later reproduction. Results showed that smaller baseline pupil sizes preceding and stronger pupil orienting responses during encoding predicted that more information was encoded into visual working memory. Additionally, we show that pupil size reflects not only how much but also how precisely material is encoded. We argue that a smaller pupil size preceding encoding is related to increased exploitation, whereas larger pupil constrictions signal stronger attentional (re)orienting to the to-be-encoded pattern. Our findings support the notion that the depth of visual working memory encoding is the integrative outcome of differential aspects of attention: how alert one is, how much attention one deploys, and how long it is deployed. Together, these factors determine how much information is encoded into visual working memory

Adaptation of the Missing Scan Task to a touchscreen format for assessing working memory capacity in children

Assessing children's working memory capacity (WMC) can be challenging for a variety of reasons, including the rapid increase in WMC across early childhood. Here, we developed and piloted an adapted WMC task, which involved minimal equipment, could be performed rapidly, and did not rely on verbal production ability (to facilitate the use of the task with younger children). In our adaptation, we portrayed the events of the object-based Missing Scan Task (creatures hiding in and emerging from a house) in a touchscreen format. In the full experiment, 67 participants aged 23 to 90 months achieved the longest set size (LSS) scores that were distributed across the full range of possible scores. A comparison of these scores with those obtained using object-based formats indicated general agreement between the versions of the task. Scores were found to increase with child age. We propose this (freely available) touchscreen adaptation as a suitable WMC task for use with children aged 2 to 7 years

Reconstructing face representations: a psychological and computational modelling approach to explore the effect of familiarity on human face recognition

When thinking about finding the face of a friend in a crowd, albeit challenging, most of us would be relatively certain we would be successful in finding them. However, if the same task were applied to an unfamiliar face, e.g. based on a photo, this would seem like an impossible task. Although the field of face recognition is very active across multiple disciplines, and there is a wealth of research on face recognition and the difference between familiar and unfamiliar faces, it still remains unclear what internal face representations we use. The aim of this thesis is to investigate what internal representations of faces we have and how these are different between familiar and unfamiliar faces. A better understanding of this internal representation will allow for the development of tasks that measure face processing abilities in more detail and increase our understanding of the visual information needed to identify a face, either familiar or unfamiliar. In the behavioural part of this thesis, I focused on exploring the relationship between image-based and perceptual similarity. In the computational part, I explore the concept of familiarity in deep neural networks. This thesis can be summed up in four major findings: there is a linear relationship between image-based spaces and perceived similarity that can be used to explore the effect of face image transformations; familiar faces are perceived as more similar overall compared to unfamiliar faces; the increase in sensitivity for greater changes between pairs is larger for familiar faces than for unfamiliar faces; and the effect of familiarity can be found at the representational level (in DNNs). Although many questions remain open, it should be evident from this thesis that a computational cognitive approach to face recognition, although technically challenging and on many occasions laborious, will progress our understanding of human face recognition

Exploring perceptual similarity and its relation to image-based spaces: an effect of familiarity

One challenge in exploring the internal representation of faces is the lack of controlled stimuli transformations. Researchers are often limited to verbalizable transformations in the creation of a dataset. An alternative approach to verbalization for interpretability is finding image-based measures that allow us to quantify image transformations. In this study, we explore whether PCA could be used to create controlled transformations to a face by testing the effect of these transformations on human perceptual similarity and on computational differences in Gabor, Pixel and DNN spaces. We found that perceptual similarity and the three image-based spaces are linearly related, almost perfectly in the case of the DNN, with a correlation of 0.94. This provides a controlled way to alter the appearance of a face. In experiment 2, the effect of familiarity on the perception of multidimensional transformations was explored. Our findings show that there is a positive relationship between the number of components transformed and both the perceptual similarity and the same three image-based spaces used in experiment 1. Furthermore, we found that familiar faces are rated more similar overall than unfamiliar faces. That is, a change to a familiar face is perceived as making less difference than the exact same change to an unfamiliar face. The ability to quantify, and thus control, these transformations is a powerful tool in exploring the factors that mediate a change in perceived identity

Exploring perceptual similarity and its relation to image-based spaces: an effect of familiarity

The lack of controlled stimuli transformations is an obstacle to the study of face identity recognition. Researchers are often limited to verbalizable transformations in the creation of a dataset. An alternative approach to verbalization for interpretability is finding image-based measures that allow us to quantify transformations. We explore whether PCA could be used to create controlled facial transformations by testing the effect of these transformations on human perceptual similarity and on computational differences in Gabor, Pixel and DNN spaces. We found that perceptual similarity and the three image-based spaces are linearly related, almost perfectly in the case of the DNN, with a correlation of 0.94. This provides a controlled way to alter the appearance of a face. In Experiment 2, the effect of familiarity on the perception of multidimensional transformations was explored. Our findings show that there is a significant relationship between the number of components transformed and both the perceptual similarity and the same three image-based spaces used in Experiment 1. Furthermore, we found that familiar faces are rated more similar overall than unfamiliar faces. The ability to quantify, and thus control, these transformations is a powerful tool in exploring the factors that mediate a change in perceived identity.Output Status: Forthcoming/Available Onlin

Evidence for the world as an external memory : A trade-off between internal and external visual memory storage

We use visual working memory (VWM) to maintain the visual features of objects in our world. Although the capacity of VWM is limited, it is unlikely that this limit will pose a problem in daily life, as visual information can be supplemented with input from our external visual world by using eye movements. In the current study, we influenced the trade-off between eye movements and VWM utilization by introducing a cost to a saccade. Higher costs were created by adding a delay in stimulus availability to a copying task. We show that increased saccade cost results in less saccades towards the model and an increased dwell time on the model. These results suggest a shift from making eye movements towards taxing internal VWM. Our findings reveal that the trade-off between executing eye-movements and building an internal representation of our world is based on an adaptive mechanism, governed by cost-efficiency