Category selectivity in human visual cortex:beyond visual object recognition

Item does not contain fulltextHuman ventral temporal cortex shows a categorical organization, with regions responding selectively to faces, bodies, tools, scenes, words, and other categories. Why is this? Traditional accounts explain category selectivity as arising within a hierarchical system dedicated to visual object recognition. For example, it has been proposed that category selectivity reflects the clustering of category-associated visual feature representations, or that it reflects category-specific computational algorithms needed to achieve view invariance. This visual object recognition framework has gained renewed interest with the success of deep neural network models trained to "recognize" objects: these hierarchical feed-forward networks show similarities to human visual cortex, including categorical separability. We argue that the object recognition framework is unlikely to fully account for category selectivity in visual cortex. Instead, we consider category selectivity in the context of other functions such as navigation, social cognition, tool use, and reading. Category-selective regions are activated during such tasks even in the absence of visual input and even in individuals with no prior visual experience. Further, they are engaged in close connections with broader domain-specific networks. Considering the diverse functions of these networks, category-selective regions likely encode their preferred stimuli in highly idiosyncratic formats; representations that are useful for navigation, social cognition, or reading are unlikely to be meaningfully similar to each other and to varying degrees may not be entirely visual. The demand for specific types of representations to support category-associated tasks may best account for category selectivity in visual cortex. This broader view invites new experimental and computational approaches.7 p

Network Dynamics of Visual Object Recognition

Visual object recognition is the principal mechanism by which humans and many animals interpret their surroundings. Despite the complexity of neural computation required, object recognition is achieved with such rapidity and accuracy that it appears to us almost effortless. Extensive human and non-human primate research has identified putative category-selective regions within higher-level visual cortex, which are thought to mediate object recognition. Despite decades of study, however, the functional organization and network dynamics within these regions remain poorly understood, due to a lack of appropriate animal models as well as the spatiotemporal limitations of current non-invasive human neuroimaging techniques (e.g. fMRI, scalp EEG). To better understand these issues, we leveraged the high spatiotemporal resolution of intracranial EEG (icEEG) recordings to study rapid, transient interactions between the disseminated cortical substrates within category-specific networks. Employing novel techniques for the topologically accurate and statistically robust analysis of grouped icEEG, we found that category-selective regions were spatially arranged with respect to cortical folding patterns, and relative to each other, to generate a hierarchical information structuring of visual information within higher-level visual cortex. This may facilitate rapid visual categorization by enabling the extraction of different levels of object detail across multiple spatial scales. To characterize network interactions between distributed regions sharing the same category-selectivity, we evaluated feed-forward, hierarchal and parallel, distributed models of information flow during face perception via measurements of cortical activation, functional and structural connectivity, and transient disruption through electrical stimulation. We found that input from early visual cortex (EVC) to two face-selective regions – the occipital and fusiform face areas (OFA and FFA, respectively) – occurred in a parallelized, distributed fashion: Functional connectivity between EVC and FFA began prior to the onset of subsequent re-entrant connectivity between the OFA and FFA. Furthermore, electrophysiological measures of structural connectivity revealed independent cortico- cortical connections between the EVC and both the OFA and FFA. Finally, direct disruption of the FFA, but not OFA, impaired face-perception. Given that the FFA is downstream of the OFA, these findings are incompatible with the feed-forward, hierarchical models of visual processing, and argue instead for the existence of parallel, distributed network interactions

Using Multivariate Pattern Analysis to Identify Conceptual Knowledge Representation in the Brain

Representation of semantic knowledge is an important aspect of cognitive function. The processing of concrete (e.g., book) and abstract (e.g., freedom) semantic concepts show systematic differences on various behavioral measures in both healthy and clinical populations. However, previous studies examining the difference in the neural substrates correlating with abstract and concrete concept representations have reached inconsistent conclusions. This dissertation used multiple novel data analyses approaches on functional magnetic resonance imaging (fMRI) data, to investigate representational differences of abstract and concrete concepts and to provide converging evidence that the representations of abstract and concrete semantic knowledge in the brain rely on different mechanisms. Study 1 used meta-analysis method on a combined sample of 303 participants to quantitatively summarize the published neuroimaging studies on the brain regions with category-specific activations. Results suggested greater engagement of working memory and language system for processing abstract concepts, and greater engagement of the visual perceptual system for processing of concrete concepts, likely via mental imagery. Study 2 showed successful identifications of single trial fMRI data as being associated with the processing of either abstract or concrete concepts based on multivoxel activity patterns in widespread brain areas, suggesting that abstract vs. concrete differences were represented by multiple mechanisms. Study 3 investigated the classification based on condition-specific connectivity patterns. Results showed successful identifications of the connectivity patterns as abstract or concrete for an individual based on the connectivity patterns of other individuals, both by the connectivity for a priory selected seed regions as well as by the whole-brain voxel-by-voxel connectivity patterns. The results indicated the existence of condition-specific connectivity patterns that were consistent across individuals on a whole-brain scale. Moreover, the results also suggested the representation of abstract and concrete concepts differs from the semantic association perspective in addition to differences on coding forms. Study 4 illustrated the application of MVPA as a cross-modal prediction approach, which is a promising method for further investigation of semantic knowledge representation in the brain, by investigating the role of general semantic system on person-specific knowledge. Overall, the work described in this dissertation provides converging evidence of the representational difference between abstract and concrete concepts. The differences are suggested to occur at various levels, including the dependence on modality-specific perceptual systems, the organization of associations among different semantic-related systems, and the difficulty and strategy of retrieving contextual information

Representation of event and object concepts in ventral anterior temporal lobe and angular gyrus

Semantic knowledge includes understanding of objects and their features and also understanding of the characteristics of events. The hub-and-spoke theory holds that these conceptual representations rely on multiple information sources that are integrated in a central hub in the ventral anterior temporal lobes. The dual-hub theory expands this framework with the claim that the ventral anterior temporal lobe hub is specialized for object representation, while a second hub in angular gyrus is specialized for event representation. To test these ideas, we used representational similarity analysis, univariate and psychophysiological interaction analyses of fMRI data collected while participants processed object and event concepts (e.g. “an apple,” “a wedding”) presented as images and written words. Representational similarity analysis showed that angular gyrus encoded event concept similarity more than object similarity, although the left angular gyrus also encoded object similarity. Bilateral ventral anterior temporal lobes encoded both object and event concept structure, and left ventral anterior temporal lobe exhibited stronger coding for events. Psychophysiological interaction analysis revealed greater connectivity between left ventral anterior temporal lobe and right pMTG, and between right angular gyrus and bilateral ITG and middle occipital gyrus, for event concepts compared to object concepts. These findings support the specialization of angular gyrus for event semantics, though with some involvement in object coding, but do not support ventral anterior temporal lobe specialization for object concepts

The Hierarchical Structure of the Face Network Revealed by Its Functional Connectivity Pattern

A major principle of human brain organization is “integrating” some regions into networks while “segregating” other sets of regions into separate networks. However, little is known about the cognitive function of the integration and segregation of brain networks. Here, we examined the well-studied brain network for face processing, and asked whether the integration and segregation of the face network (FN) are related to face recognition performance. To do so, we used a voxel-based global brain connectivity method based on resting-state fMRI to characterize the within-network connectivity (WNC) and the between-network connectivity (BNC) of the FN. We found that 95.4% of voxels in the FN had a significantly stronger WNC than BNC, suggesting that the FN is a relatively encapsulated network. Importantly, individuals with a stronger WNC (i.e., integration) in the right fusiform face area were better at recognizing faces, whereas individuals with a weaker BNC (i.e., segregation) in the right occipital face area performed better in the face recognition tasks. In short, our study not only demonstrates the behavioral relevance of integration and segregation of the FN but also provides evidence supporting functional division of labor between the occipital face area and fusiform face area in the hierarchically organized FN

How the brain grasps tools: fMRI & motion-capture investigations

Humans’ ability to learn about and use tools is considered a defining feature of our species, with most related neuroimaging investigations involving proxy 2D picture viewing tasks. Using a novel tool grasping paradigm across three experiments, participants grasped 3D-printed tools (e.g., a knife) in ways that were considered to be typical (i.e., by the handle) or atypical (i.e., by the blade) for subsequent use. As a control, participants also performed grasps in corresponding directions on a series of 3D-printed non-tool objects, matched for properties including elongation and object size. Project 1 paired a powerful fMRI block-design with visual localiser Region of Interest (ROI) and searchlight Multivoxel Pattern Analysis (MVPA) approaches. Most remarkably, ROI MVPA revealed that hand-selective, but not anatomically overlapping tool-selective, areas of the left Lateral Occipital Temporal Cortex and Intraparietal Sulcus represented the typicality of tool grasping. Searchlight MVPA found similar evidence within left anterior temporal cortex as well as right parietal and temporal areas. Project 2 measured hand kinematics using motion-capture during a highly similar procedure, finding hallmark grip scaling effects despite the unnatural task demands. Further, slower movements were observed when grasping tools, relative to non-tools, with grip scaling also being poorer for atypical tool, compared to non-tool, grasping. Project 3 used a slow-event related fMRI design to investigate whether representations of typicality were detectable during motor planning, but MVPA was largely unsuccessful, presumably due to a lack of statistical power. Taken together, the representations of typicality identified within areas of the ventral and dorsal, but not ventro-dorsal, pathways have implications for specific predictions made by leading theories about the neural regions supporting human tool-use, including dual visual stream theory and the two-action systems model

Neural Correlates of Repetition Priming: Changes in fMRI Activation and Synchrony

The neural mechanisms of behavioural priming remain unclear. Recent studies have suggested that category-preferential regions in ventral occipitotemporal cortex (VOTC) play an important role; some have reported increased neural synchrony between prefrontal cortex and temporal cortex associated with stimulus repetition. Based on these results, I hypothesized that increased neural synchrony, as measured by functional connectivity analysis using functional MRI, between category-preferential regions in VOTC and broader category-related networks would underlie behavioural priming. To test this hypothesis, I localized several category-preferential regions in VOTC using an independent functional localizer. Then, Seed Partial Least Squares was used to assess task-related functional connectivity of these regions across repetition of stimuli from multiple categories during an independent semantic classification task. While the results did not show the hypothesized differences in functional connectivity across stimulus repetition, evidence of category-specificity of neural priming and novel insights about the nature of category-related organization of VOTC were revealed

Bio-Inspired Computer Vision: Towards a Synergistic Approach of Artificial and Biological Vision

To appear in CVIUStudies in biological vision have always been a great source of inspiration for design of computer vision algorithms. In the past, several successful methods were designed with varying degrees of correspondence with biological vision studies, ranging from purely functional inspiration to methods that utilise models that were primarily developed for explaining biological observations. Even though it seems well recognised that computational models of biological vision can help in design of computer vision algorithms, it is a non-trivial exercise for a computer vision researcher to mine relevant information from biological vision literature as very few studies in biology are organised at a task level. In this paper we aim to bridge this gap by providing a computer vision task centric presentation of models primarily originating in biological vision studies. Not only do we revisit some of the main features of biological vision and discuss the foundations of existing computational studies modelling biological vision, but also we consider three classical computer vision tasks from a biological perspective: image sensing, segmentation and optical flow. Using this task-centric approach, we discuss well-known biological functional principles and compare them with approaches taken by computer vision. Based on this comparative analysis of computer and biological vision, we present some recent models in biological vision and highlight a few models that we think are promising for future investigations in computer vision. To this extent, this paper provides new insights and a starting point for investigators interested in the design of biology-based computer vision algorithms and pave a way for much needed interaction between the two communities leading to the development of synergistic models of artificial and biological vision