The nature of the animacy organization in human ventral temporal cortex

The principles underlying the animacy organization of the ventral temporal cortex (VTC) remain hotly debated, with recent evidence pointing to an animacy continuum rather than a dichotomy. What drives this continuum? According to the visual categorization hypothesis, the continuum reflects the degree to which animals contain animal-diagnostic features. By contrast, the agency hypothesis posits that the continuum reflects the degree to which animals are perceived as (social) agents. Here, we tested both hypotheses with a stimulus set in which visual categorizability and agency were dissociated based on representations in convolutional neural networks and behavioral experiments. Using fMRI, we found that visual categorizability and agency explained independent components of the animacy continuum in VTC. Modeled together, they fully explained the animacy continuum. Finally, clusters explained by visual categorizability were localized posterior to clusters explained by agency. These results show that multiple organizing principles, including agency, underlie the animacy continuum in VTC.Comment: 16 pages, 5 figures, code+data at - https://doi.org/10.17605/OSF.IO/VXWG9 Update - added supplementary results and edited abstrac

Naturalistic stimuli reveal a dominant role for agentic action in visual representation

Abstract Naturalistic, dynamic movies evoke strong, consistent, and information-rich patterns of activity over a broad expanse of cortex and engage multiple perceptual and cognitive systems in parallel. The use of naturalistic stimuli enables functional brain imaging research to explore cognitive domains that are poorly sampled in highly-controlled experiments. These domains include perception and understanding of agentic action, which plays a larger role in visual representation than was appreciated from experiments using static, controlled stimuli

A humanness dimension to visual object coding in the brain

Neuroimaging studies investigating human object recognition have primarily focused on a relatively small number of object categories, in particular, faces, bodies, scenes, and vehicles. More recent studies have taken a broader focus, investigating hypothesized dichotomies, for example, animate versus inanimate, and continuous feature dimensions, such as biologically similarity. These studies typically have used stimuli that are identified as animate or inanimate, neglecting objects that may not fit into this dichotomy. We generated a novel stimulus set including standard objects and objects that blur the animate-inanimate dichotomy, for example, robots and toy animals. We used MEG time-series decoding to study the brain's emerging representation of these objects. Our analysis examined contemporary models of object coding such as dichotomous animacy, as well as several new higher order models that take into account an object's capacity for agency (i.e. its ability to move voluntarily) and capacity to experience the world. We show that early (0–200 ​ms) responses are predicted by the stimulus shape, assessed using a retinotopic model and shape similarity computed from human judgments. Thereafter, higher order models of agency/experience provided a better explanation of the brain's representation of the stimuli. Strikingly, a model of human similarity provided the best account for the brain's representation after an initial perceptual processing phase. Our findings provide evidence for a new dimension of object coding in the human brain – one that has a “human-centric” focus

Perceiving animacy from shape

Superordinate visual classification—for example, identifying an image as “animal,” “plant,” or “mineral”—is computationally challenging because radically different items (e.g., “octopus,” “dog”) must be grouped into a common class (“animal”). It is plausible that learning superordinate categories teaches us not only the membership of particular (familiar) items, but also general features that are shared across class members, aiding us in classifying novel (unfamiliar) items. Here, we investigated visual shape features associated with animate and inanimate classes. One group of participants viewed images of 75 unfamiliar and atypical items and provided separate ratings of how much each image looked like an animal, plant, and mineral. Results show systematic tradeoffs between the ratings, indicating a class-like organization of items. A second group rated each image in terms of 22 midlevel shape features (e.g., “symmetrical,” “curved”). The results confirm that superordinate classes are associated with particular shape features (e.g., “animals” generally have high “symmetry” ratings). Moreover, linear discriminant analysis based on the 22-D feature vectors predicts the perceived classes approximately as well as the ground truth classification. This suggests that a generic set of midlevel visual shape features forms the basis for superordinate classification of novel objects along the animacy continuum

The Origins and Development of Visual Categorization

Forming categories is a core part of human cognition, allowing us to make quickly make inferences about our environment. This thesis investigated some of the major theoretical interpretations surrounding the neural basis of visual category development. In adults, there are category-selective regions (e.g. in ventral temporal cortex) and networks (which include regions outside traditional visual regions—e.g. the amygdala) that support visual categorization. While there has been extensive behavioural work investigating visual categorization in infants, the neural sequence of development remains poorly understood. Based on behavioral experiments, one view holds that infants are initially using subcortical structures to recognize faces. Indeed, it has been proposed that the subcortical pathway remains active for rapid face detection in adults. In order to test this in adults, I exploited the nasal-temporal asymmetry of the proposed retinocollicular pathway to see if preferentially presenting stimuli to the nasal hemiretina resulted in a fast face detection advantage when contrasted with presentations to the temporal hemiretina. Across four experiments, I failed to find any evidence of a subcortical advantage but still found that a rapid, coarse pathway exists. Therefore, I moved to investigate the development of the cortical visual categorization regions in the ventral temporal cortex (VTC). I characterised the maturity of the face, place and tool regions found in the VTC, looking at the long-range connectivity in 1-9 month-old infants using MRI tractography and a linear discriminant classifier. The face and place regions showed adult-like connectivity throughout infancy, but the tool-network underwent significant maturation until 9 months. Finally, given this maturity of face and place regions in early infancy, I decided to test whether the organization of the VTC was related to the sequence of categories infants acquire. I used language age of acquisition measurements, determining that infants produce significantly more animate than inanimate words up until 29-months, in line with the animacy distinction in the VTC. My work demonstrates the surprising role and maturity of the cortical regions and networks involved in visual categorization. My thesis develops new methods for studying the infant brain and underscores the utility of publicly available data when studying development

Unique contributions of perceptual and conceptual humanness to object representations in the human brain

The human brain is able to quickly and accurately identify objects in a dynamic visual world. Objects evoke different patterns of neural activity in the visual system, which reflect object category memberships. However, the underlying dimensions of object representations in the brain remain unclear. Recent research suggests that objects similarity to humans is one of the main dimensions used by the brain to organise objects, but the nature of the human-similarity features driving this organisation are still unknown. Here, we investigate the relative contributions of perceptual and conceptual features of humanness to the representational organisation of objects in the human visual system. We collected behavioural judgements of human-similarity of various objects, which were compared with time-resolved neuroimaging responses to the same objects. The behavioural judgement tasks targeted either perceptual or conceptual humanness features to determine their respective contribution to perceived human-similarity. Behavioural and neuroimaging data revealed significant and unique contributions of both perceptual and conceptual features of humanness, each explaining unique variance in neuroimaging data. Furthermore, our results showed distinct spatio-temporal dynamics in the processing of conceptual and perceptual humanness features, with later and more lateralised brain responses to conceptual features. This study highlights the critical importance of social requirements in information processing and organisation in the human brain

Animacy Dimensions Ratings and Approach for Decorrelating Stimuli Dimensions

The distinction between animate and inanimate objects plays an important role in object recognition. The following 5 dimensions were shown in previous studies to be important for animacy perception independently: “being alive”, “looking like an animal”, “having mobility”, “having agency” and “being unpredictable”. However, it is not known how these dimensions in combination determine how we perceive animacy. To investigate, we created a stimulus set (M = 300) with almost all dimension combinations for which we acquired behavioural ratings on the 5 dimensions. We show that subjects (N = 26) are consistent in animacy ratings (r = 0.6) and that “being alive” and “having agency” dimensions are highly correlated (r = 0.62). To design a stimulus sub-set that is decorrelated on animacy dimensions for future fMRI and EGG experiments we used a genetic algorithm. Our approach proved to be successful in stimuli selection (max r = 0.35, compared to max r = 0.59 when using a random search). In summary, our study systematically investigates animacy dimensions, provides new insights in animacy perception, and presents an approach for decorrelating stimuli dimensions that can be useful for other studies

The Importance of Configural Information and the Time-Course for the Perception of Animacy in Faces

It has been proposed that the processing of faces is a highly specialized function that is separate from the processing of other types of objects. Evidence for this appears with inversion, where it has been observed that face recognition is more affected than the recognition of other types of stimuli (Diamond and Carey, 1986). From these observations it can be inferred that holistic processing of faces makes face analysis unique. This thesis focuses on the question of whether configural information encoded in a face can be used quickly to determine the animacy of a face, or if a more detailed analysis of face features is necessary. By animacy we mean whether the image portrays a real human face or that of a “human-like” inanimate object such as a doll or a manikin. In the first part of this thesis face morphs between doll and human faces were inverted to see if animacy perception of faces would be affected with less configural information. In the second part of the thesis, the time-course of animacy perception in relation to face recognition is analyzed. The final part of the thesis examines the role of high and low spatial frequency ranges in encoding information used for animacy judgment. The results from the first experiment suggest that holistic processing of the face is necessary for animacy perception. From the temporal experiment it can be deduced that animacy perception occurs more quickly than face recognition, and may require less detailed face analysis. The results from the spatial frequency pilot experiment suggest that removing the high spatial frequencies or low spatial frequencies from the face images has no effect on animacy perception, however, more data is needed in order to form a strong conclusion