Category selectivity in human visual cortex:beyond visual object recognition

Item does not contain fulltextHuman ventral temporal cortex shows a categorical organization, with regions responding selectively to faces, bodies, tools, scenes, words, and other categories. Why is this? Traditional accounts explain category selectivity as arising within a hierarchical system dedicated to visual object recognition. For example, it has been proposed that category selectivity reflects the clustering of category-associated visual feature representations, or that it reflects category-specific computational algorithms needed to achieve view invariance. This visual object recognition framework has gained renewed interest with the success of deep neural network models trained to "recognize" objects: these hierarchical feed-forward networks show similarities to human visual cortex, including categorical separability. We argue that the object recognition framework is unlikely to fully account for category selectivity in visual cortex. Instead, we consider category selectivity in the context of other functions such as navigation, social cognition, tool use, and reading. Category-selective regions are activated during such tasks even in the absence of visual input and even in individuals with no prior visual experience. Further, they are engaged in close connections with broader domain-specific networks. Considering the diverse functions of these networks, category-selective regions likely encode their preferred stimuli in highly idiosyncratic formats; representations that are useful for navigation, social cognition, or reading are unlikely to be meaningfully similar to each other and to varying degrees may not be entirely visual. The demand for specific types of representations to support category-associated tasks may best account for category selectivity in visual cortex. This broader view invites new experimental and computational approaches.7 p

The nature of the animacy organization in human ventral temporal cortex

The principles underlying the animacy organization of the ventral temporal cortex (VTC) remain hotly debated, with recent evidence pointing to an animacy continuum rather than a dichotomy. What drives this continuum? According to the visual categorization hypothesis, the continuum reflects the degree to which animals contain animal-diagnostic features. By contrast, the agency hypothesis posits that the continuum reflects the degree to which animals are perceived as (social) agents. Here, we tested both hypotheses with a stimulus set in which visual categorizability and agency were dissociated based on representations in convolutional neural networks and behavioral experiments. Using fMRI, we found that visual categorizability and agency explained independent components of the animacy continuum in VTC. Modeled together, they fully explained the animacy continuum. Finally, clusters explained by visual categorizability were localized posterior to clusters explained by agency. These results show that multiple organizing principles, including agency, underlie the animacy continuum in VTC.Comment: 16 pages, 5 figures, code+data at - https://doi.org/10.17605/OSF.IO/VXWG9 Update - added supplementary results and edited abstrac

Review: Object vision in a structured world

In natural vision, objects appear at typical locations, both with respect to visual space (e.g., an airplane in the upper part of a scene) and other objects (e.g., a lamp above a table). Recent studies have shown that object vision is strongly adapted to such positional regularities. In this review we synthesize these developments, highlighting that adaptations to positional regularities facilitate object detection and recognition, and sharpen the representations of objects in visual cortex. These effects are pervasive across various types of high-level content. We posit that adaptations to real-world structure collectively support optimal usage of limited cortical processing resources. Taking positional regularities into account will thus be essential for understanding efficient object vision in the real world

Category-Specific Item Recognition and the Medial Temporal Lobe

Much neuropsychological and neuroimaging research has been focused on the contributions of different medial temporal lobe (MTL) structures to recognition memory. The majority of these studies have linked perirhinal cortex (PrC) to item recognition, whereas the hippocampus and parahippocampal cortex (PhC) have primarily been associated with the recollection of contextual detail pertaining to a specific prior stimulus encounter. Here, I report results from three fMRI studies that examined the neural correlates of item recognition with a specific focus on the relationship between such signals and category-specific effects in the MTL. In Chapter 2, I reveal that category-specific representations in both PrC and PhC can be brought to bear on item recognition decisions. In Chapter 3, I examined the specific stimulus properties that determine the relative contributions of PrC and PhC to item recognition, with a focus on landmark suitability. The results from this study revealed item recognition signals for non-landmark objects in PrC and landmarks in PhC. In Chapter 4, I focused specifically on face recognition to characterize the manner in which PrC codes item-recognition signals and to further explore the issue of category-specificity with independent functional localizer data. Results from this study indicate that item recognition signals in PrC can be distributed across voxels with directionally heterogeneous response profiles. Further, these data also revealed that the voxels comprising these patterns respond preferentially to faces under passive viewing conditions. Taken together, these findings suggest that item recognition signals are represented in a distributed, category-specific manner within both PrC and PhC

Differences in selectivity to natural images in early visual areas (V1–V3)

High-level regions of the ventral visual pathway respond more to intact objects compared to scrambled objects. The aim of this study was to determine if this selectivity for objects emerges at an earlier stage of processing. Visual areas (V1–V3) were defined for each participant using retinotopic mapping. Participants then viewed intact and scrambled images from different object categories (bottle, chair, face, house, shoe) while neural responses were measured using fMRI. Our rationale for using scrambled images is that they contain the same low-level properties as the intact objects, but lack the higher-order combinations of features that are characteristic of natural images. Neural responses were higher for scrambled than intact images in all regions. However, the difference between intact and scrambled images was smaller in V3 compared to V1 and V2. Next, we measured the spatial patterns of response to intact and scrambled images from different object categories. We found higher within-category compared to between category correlations for both intact and scrambled images demonstrating distinct patterns of response. Spatial patterns of response were more distinct for intact compared to scrambled images in V3, but not in V1 or V2. These findings demonstrate the emergence of selectivity to natural images in V3

Selectivity for mid‐level properties of faces and places in the fusiform face area and parahippocampal place area

Regions in the ventral visual pathway, such as the fusiform face area (FFA) and parahippocampal place area (PPA), are selective for images from specific object categories. Yet images from different object categories differ in their image properties. To investigate how these image properties are represented in the FFA and PPA, we compared neural responses to locally-scrambled images (in which mid-level, spatial properties are preserved) and globally-scrambled images (in which mid-level, spatial properties are not preserved). There was a greater response in the FFA and PPA to images from the preferred category relative to their non-preferred category for the scrambled conditions. However, there was a greater selectivity for locally-scrambled compared to globally-scrambled images. Next, we compared the magnitude of fMR adaptation to intact and scrambled images. fMR-adaptation was evident to locally-scrambled images from the preferred category. However, there was no adaptation to globally-scrambled images from the preferred category. These results show that the selectivity to faces and places in the FFA and PPA is dependent on mid-level properties of the image that are preserved by local scrambling

Reading the mind's eye: Decoding category information during mental imagery

Category information for visually presented objects can be read out from multi-voxel patterns of fMRI activity in ventral–temporal cortex. What is the nature and reliability of these patterns in the absence of any bottom–up visual input, for example, during visual imagery? Here, we first ask how well category information can be decoded for imagined objects and then compare the representations evoked during imagery and actual viewing. In an fMRI study, four object categories (food, tools, faces, buildings) were either visually presented to subjects, or imagined by them. Using pattern classification techniques, we could reliably decode category information (including for non-special categories, i.e., food and tools) from ventral–temporal cortex in both conditions, but only during actual viewing from retinotopic areas. Interestingly, in temporal cortex when the classifier was trained on the viewed condition and tested on the imagery condition, or vice versa, classification performance was comparable to within the imagery condition. The above results held even when we did not use information in the specialized category-selective areas. Thus, the patterns of representation during imagery and actual viewing are in fact surprisingly similar to each other. Consistent with this observation, the maps of “diagnostic voxels” (i.e., the classifier weights) for the perception and imagery classifiers were more similar in ventral–temporal cortex than in retinotopic cortex. These results suggest that in the absence of any bottom–up input, cortical back projections can selectively re-activate specific patterns of neural activity

Revealing Connections in Object and Scene Processing Using Consecutive TMS and fMR-Adaptation

When processing the visual world, our brain must perform many computations that may occur across several regions. It is important to understand communications between regions in order to understand perceptual processes underlying processing of our environment. We sought to determine the connectivity of object and scene processing regions of the cortex, which are not fully established. In order to determine these connections repetitive transcranial magnetic stimulation (rTMS) and functional magnetic resonance-adaptation (fMR-A) were paired together. rTMS was applied to object-selective lateral occipital (LO) and scene-selective transverse occipital sulcus (TOS). Immediately after stimulation, participants underwent fMR-A, and pre- and post-TMS responses were compared. TMS disrupted remote regions revealing connections from LO and TOS to remote object and scene-selective regions in the occipital cortex. In addition, we report important neural correlates regarding the transference of object related information between modalities, from LO to outside the ventral network to parietal and frontal areas

The Role of Visual and Semantic Properties in the Emergence of Category-Specific Patterns of Neural Response in the Human Brain

Brain-imaging studies have found distinct spatial and temporal patterns of response to different object categories across the brain. However, the extent to which these categorical patterns of response reflect higher-level semantic or lower-level visual properties of the stimulus remains unclear. To address this question, we measured patterns of EEG response to intact and scrambled images in the human brain. Our rationale for using scrambled images is that they have many of the visual properties found in intact images, but do not convey any semantic information. Images from different object categories (bottle, face, house) were briefly presented (400 ms) in an event-related design. A multivariate pattern analysis revealed categorical patterns of response to intact images emerged ∼80–100 ms after stimulus onset and were still evident when the stimulus was no longer present (∼800 ms). Next, we measured the patterns of response to scrambled images. Categorical patterns of response to scrambled images also emerged ∼80–100 ms after stimulus onset. However, in contrast to the intact images, distinct patterns of response to scrambled images were mostly evident while the stimulus was present (∼400 ms). Moreover, scrambled images were able to account only for all the variance in the intact images at early stages of processing. This direct manipulation of visual and semantic content provides new insights into the temporal dynamics of object perception and the extent to which different stages of processing are dependent on lower-level or higher-level properties of the image