Specialized Signals for Spatial Attention in the Ventral and Dorsal Visual Streams

Neuroscientists have traditionally conceived the visual system as having a ventral stream of vision for perception and a dorsal one associated with vision for action. However functional differences between them have become relatively blurred in recent years, not the least by the systematic parallel mapping of functions allowed by functional magnetic resonance imaging (fMRI). Here, using fMRI to simultaneously monitor several brain regions, we first studied a hallmark ventral stream computation: the processing of faces. We did so by probing responses to motion, an attribute whose processing is typically associated with the dorsal stream. In humans, it is known that face-selective regions in the superior temporal sulcus (STS) show enhanced responses to facial motion that are absent in the rest of the face-processing system. In macaques, face areas also exist, but their functional specializations for facial motion are unknown. We showed static and moving face and non-face objects to macaques and humans in an fMRI experiment in order to isolate potential functional specializations in the ventral stream face-processing system and to motivate putative homologies across species. Our results revealed all macaque face areas showed enhanced responses to moving faces. There was a difference between more dorsal face areas in the fundus of the STS, which are embedded in motion responsive cortex and ventral ones, where enhanced responses to motion interacted with object category and could not be explained by their proximity to motion responsive cortex. In humans watching the same stimuli, only the STS face area showed an enhancement for motion. These results suggest specializations for motion exist in the macaque face-processing network but they do not lend themselves to a direct equalization between human and macaque face areas. We then proceeded to compare ventral and dorsal stream functions in terms of their code for spatial attention, whose control was typically associated with the dorsal stream and prefrontal areas. We took advantage of recent fMRI studies that provide a systematic map of cortical areas modulated by spatial attention and suggest PITd, a ventral stream area in the temporal lobe, can support endogenous attention control. Covert attention and stimulus selection by saccades are represented in the same maps of visual space in attention control areas. Difficulties interpreting this multiplicity of functions led to the proposal that they encode priority maps, where multiple sources are summed to form a single priority signal, agnostic as to its eventual use by downstream areas. Using a paradigm that dissociates covert attention and response selection, we test this hypothesis with fMRI-guided electrophysiology in two cortical areas: parietal area LIP, where the priority map was first proposed to apply, and temporal area PITd. Our results indicate LIP sums disparate signals, but as a consequence independent channels of spatial information exist for attention and response planning. PITd represents relevant locations and, rather than summing signals, contains a single map for covert attention. Our findings have the potential to resolve a longstanding controversy about the nature of spatial signals in LIP and establish PITd as a robust map for covert attention in the ventral stream. Together, our results suggest that while the distribution of labor between ventral stream and dorsal stream areas is less linear than what a what a rough depiction of them can suggest, it is illuminated by their proposed function as supporting vision for perception and vision for action respectively

Accuracy of rats in discriminating visual objects Is explained by the complexity of their perceptual strategy

Despite their growing popularity as models of visual functions, it is widely assumed that rodents deploy perceptual strategies not nearly as advanced as those of primates, when processing visual objects. Such belief is fostered by the conflicting findings about the complexity of rodent pattern vision, which appears to range from mere detection of overall object luminance to view-invariant processing of discriminant shape features. Here, we sought to clarify how refined object vision is in rodents, by measuring how well a group of rats discriminated a reference object from eleven distractors, spanning a spectrum of image-level similarity with the reference. We also presented the animals with random variations of the reference, and we processed their responses to these stimuli to obtain subject-specific models of rat perceptual choices. These models captured very well the highly variable discrimination performance observed across subjects and object conditions. In particular, they revealed how the animals that succeeded with the more challenging distractors were those that integrated the wider variety of discriminant features into their perceptual strategy. Critically, these features remained highly subject-specific and largely invariant under changes in object appearance (e.g., size variation), although they were properly reformatted (e.g., rescaled) to deal with the specific transformations the objects underwent. Overall, these findings show that rat object vision, far from being poorly developed, can be characterized as a feature-based filtering process (iterated across multiple scales, positions, etc.), similar to the one that is at work in primates and state-of-the-art machine vision systems, such as convolutional neural networks

The neurons that mistook a hat for a face

Despite evidence that context promotes the visual recognition of objects, decades of research have led to the pervasive notion that the object processing pathway in primate cortex consists of multiple areas that each process the intrinsic features of a few particular categories (e.g. faces, bodies, hands, objects, and scenes). Here we report that such category-selective neurons do not in fact code individual categories in isolation but are also sensitive to object relationships that reflect statistical regularities of the experienced environment. We show by direct neuronal recording that face-selective neurons respond not just to an image of a face, but also to parts of an image where contextual cues-for example a body-indicate a face ought to be, even if what is there is not a face

Normalization Among Heterogeneous Population Confers Stimulus Discriminability on the Macaque Face Patch Neurons

Primates are capable of recognizing faces even in highly cluttered natural scenes. In order to understand how the primate brain achieves face recognition despite this clutter, it is crucial to study the representation of multiple faces in face selective cortex. However, contrary to the essence of natural scenes, most experiments on face recognition literatures use only few faces at a time on a homogeneous background to study neural response properties. It thus remains unclear how face selective neurons respond to multiple stimuli, some of which might be encompassed by their receptive fields (RFs), others not. How is the neural representation of a face affected by the concurrent presence of other stimuli? Two lines of evidence lead to opposite predictions: first, given the importance of MAX-like operations for achieving selectivity and invariance, as suggested by feedforward circuitry for object recognition, face representations may not be compromised in the presence of clutter. On the other hand, the psychophysical crowding effect - the reduced discriminability (but not detectability) of an object in clutter - suggests that an object representation may be impaired by additional stimuli. To address this question, we conducted electrophysiological recordings in the macaque temporal lobe, where bilateral face selective areas are tightly interconnected to form a hierarchical face processing stream. Assisted by functional MRI, these face patches could be targeted for single-cell recordings. For each neuron, the most preferred face stimulus was determined, then presented at the center of the neuron\u27s RF. In addition, multiple stimuli (preferred or non-preferred) were presented in different numbers (0,1,2,4 or 8), from different categories (face or non-face object), or at different proximity (adjacent to or separated from the center stimulus). We found the majority of neurons reduced mean ring rates more (1) with increasing numbers of distractors, (2) with face distractors rather than with non-face object distractors, (3) at closer distractor proximity, and, additionally, (4) the response to multiple preferred faces depends on RF size. Although these findings in single neurons could indicate reduced discriminability, we found that each stimulus condition was well separated and decodable in a high-dimensional space spanned by the neural population. We showed that this was because neuronal population was quite heterogeneous, yet changing response systematically as stimulus parameter changed. Few neurons showed MAX-like behavior. These findings were explained by divisive normalization model, highlighting the importance of the modular structure of the primate temporal lobe. Taken together, these data and modeling results indicate that neurons in the face patches acquire stimulus discriminability by virtue of the modularity of cortical organization, heterogeneity within the population, and systematicity of the neural response

Reverse engineering object recognition

Thesis (Ph. D.)--Massachusetts Institute of Technology, Dept. of Brain and Cognitive Sciences, 2007.This electronic version was submitted by the student author. The certified thesis is available in the Institute Archives and Special Collections.Page 95 blank.Includes bibliographical references (p. 83-94).Any given object in the world can cast an effectively infinite number of different images onto the retina, depending on its position relative to the viewer, the configuration of light sources, and the presence of other objects in the visual field. In spite of this, primates can robustly recognize a multitude of objects in a fraction of a second, with no apparent effort. The computational mechanisms underlying these amazing abilities are poorly understood. This thesis presents a collection of work from human psychophysics, monkey electrophysiology, and computational modelling in an effort to reverse-engineer the key computational components that enable this amazing ability in the primate visual system.by David Daniel Cox.Ph.D

Frontier of Self and Impact Prediction

The construction of a coherent representation of our body and the mapping of the space immediately surrounding it are of the highest ecological importance. This space has at least three specificities: it is a space where actions are planned in order to interact with our environment; it is a space that contributes to the experience of self and self-boundaries, through tactile processing and multisensory interactions; last, it is a space that contributes to the experience of body integrity against external events. In the last decades, numerous studies have been interested in peripersonal space (PPS), defined as the space directly surrounding us and which we can interact with (for reviews, see Cléry et al., 2015b; de Vignemont and Iannetti, 2015; di Pellegrino and Làdavas, 2015). These studies have contributed to the understanding of how this space is constructed, encoded and modulated. The majority of these studies focused on subparts of PPS (the hand, the face or the trunk) and very few of them investigated the interaction between PPS subparts. In the present review, we summarize the latest advances in this research and we discuss the new perspectives that are set forth for futures investigations on this topic. We describe the most recent methods used to estimate PPS boundaries by the means of dynamic stimuli. We then highlight how impact prediction and approaching stimuli modulate this space by social, emotional and action-related components involving principally a parieto-frontal network. In a next step, we review evidence that there is not a unique representation of PPS but at least three sub-sections (hand, face and trunk PPS). Last, we discuss how these subspaces interact, and we question whether and how bodily self-consciousness (BSC) is functionally and behaviorally linked to PPS

Change blindness: eradication of gestalt strategies

Arrays of eight, texture-defined rectangles were used as stimuli in a one-shot change blindness (CB) task where there was a 50% chance that one rectangle would change orientation between two successive presentations separated by an interval. CB was eliminated by cueing the target rectangle in the first stimulus, reduced by cueing in the interval and unaffected by cueing in the second presentation. This supports the idea that a representation was formed that persisted through the interval before being 'overwritten' by the second presentation (Landman et al, 2003 Vision Research 43149–164]. Another possibility is that participants used some kind of grouping or Gestalt strategy. To test this we changed the spatial position of the rectangles in the second presentation by shifting them along imaginary spokes (by ±1 degree) emanating from the central fixation point. There was no significant difference seen in performance between this and the standard task [F(1,4)=2.565, p=0.185]. This may suggest two things: (i) Gestalt grouping is not used as a strategy in these tasks, and (ii) it gives further weight to the argument that objects may be stored and retrieved from a pre-attentional store during this task

Learning a dictionary of shape-components in visual cortex : comparison with neurons, humans and machines

Thesis (Ph. D.)--Massachusetts Institute of Technology, Dept. of Brain and Cognitive Sciences, 2006.This electronic version was submitted by the student author. The certified thesis is available in the Institute Archives and Special Collections.Includes bibliographical references (p. [175]-211).In this thesis, I describe a quantitative model that accounts for the circuits and computations of the feedforward path of the ventral stream of visual cortex. This model is consistent with a general theory of visual processing that extends the hierarchical model of [Hubel and Wiesel, 1959] from primary to extrastriate visual areas. It attempts to explain the first few hundred milliseconds of visual processing and "immediate recognition". One of the key elements in the approach is the learning of a generic dictionary of shape components from V2 to IT, which provides an invariant representation to task-specific categorization circuits in higher brain areas. This vocabulary of shape-tuned units is learned in an unsupervised manner from natural images, and constitutes a large and redundant set of image features with different complexities and invariances. This theory significantly extends an earlier approach by [Riesenhuber and Poggio, 1999a] and builds upon several existing neurobiological models and conceptual proposals. First, I present evidence to show that the model can duplicate the tuning properties of neurons in various brain areas (e.g., V1, V4 and IT).(cont.) In particular, the model agrees with data from V4 about the response of neurons to combinations of simple two-bar stimuli [Reynolds et al., 1999] (within the receptive field of the S2 units) and some of the C2 units in the model show a tuning for boundary conformations which is consistent with recordings from V4 [Pasupathy and Connor, 2001]. Second, I show that not only can the model duplicate the tuning properties of neurons in various brain areas when probed with artificial stimuli, but it can also handle the recognition of objects in the real-world, to the extent of competing with the best computer vision systems. Third, I describe a comparison between the performance of the model and the performance of human observers in a rapid animal vs. non-animal recognition task for which recognition is fast and cortical back-projections are likely to be inactive. Results indicate that the model predicts human performance extremely well when the delay between the stimulus and the mask is about 50 ms. This suggests that cortical back-projections may not play a significant role when the time interval is in this range, and the model may therefore provide a satisfactory description of the feedforward path.(cont.) Taken together, the evidences suggest that we may have the skeleton of a successful theory of visual cortex. In addition, this may be the first time that a neurobiological model, faithful to the physiology and the anatomy of visual cortex, not only competes with some of the best computer vision systems thus providing a realistic alternative to engineered artificial vision systems, but also achieves performance close to that of humans in a categorization task involving complex natural images.by Thomas Serre.Ph.D

Learning a Dictionary of Shape-Components in Visual Cortex: Comparison with Neurons, Humans and Machines

PhD thesisIn this thesis, I describe a quantitative model that accounts for the circuits and computations of the feedforward path of the ventral stream of visual cortex. This model is consistent with a general theory of visual processing that extends the hierarchical model of (Hubel & Wiesel, 1959) from primary to extrastriate visual areas. It attempts to explain the first few hundred milliseconds of visual processing and Âimmediate recognitionÂ. One of the key elements in the approach is the learning of a generic dictionary of shape-components from V2 to IT, which provides an invariant representation to task-specific categorization circuits in higher brain areas. This vocabulary of shape-tuned units is learned in an unsupervised manner from natural images, and constitutes a large and redundant set of image features with different complexities and invariances. This theory significantly extends an earlier approach by (Riesenhuber & Poggio, 1999) and builds upon several existing neurobiological models and conceptual proposals.First, I present evidence to show that the model can duplicate the tuning properties of neurons in various brain areas (e.g., V1, V4 and IT). In particular, the model agrees with data from V4 about the response of neurons to combinations of simple two-bar stimuli (Reynolds et al, 1999) (within the receptive field of the S2 units) and some of the C2 units in the model show a tuning for boundary conformations which is consistent with recordings from V4 (Pasupathy & Connor, 2001). Second, I show that not only can the model duplicate the tuning properties of neurons in various brain areas when probed with artificial stimuli, but it can also handle the recognition of objects in the real-world, to the extent of competing with the best computer vision systems. Third, I describe a comparison between the performance of the model and the performance of human observers in a rapid animal vs. non-animal recognition task for which recognition is fast and cortical back-projections are likely to be inactive. Results indicate that the model predicts human performance extremely well when the delay between the stimulus and the mask is about 50 ms. This suggests that cortical back-projections may not play a significant role when the time interval is in this range, and the model may therefore provide a satisfactory description of the feedforward path.Taken together, the evidences suggest that we may have the skeleton of a successful theory of visual cortex. In addition, this may be the first time that a neurobiological model, faithful to the physiology and the anatomy of visual cortex, not only competes with some of the best computer vision systems thus providing a realistic alternative to engineered artificial vision systems, but also achieves performance close to that of humans in a categorization task involving complex natural images

Traitement des configurations spatiales dans le cortex visuel chez le primate non-humain

Le traitement des configurations spatiales est un mécanisme qui intervient en permanence au sein du cortex visuel. Dans ce monde emplit de régularités qui est le nôtre, il tient une place prépondérante dans l'analyse des objets de notre environnement en nous permettant d'établir des relations spatiales entre des ensembles d'éléments pour aboutir à une perception globale. Si certaines caractéristiques de ces mécanismes ont été étudiés chez les primate humain et non-humain, les observations issues de ces études ont été majoritairement portées par des approches différentes dont les méthodes non-invasives en neuroimagerie sont privilégiées chez l'humain et les méthodes plus invasives tel que l'électrophysiologie sont favorisées chez le singe. Bien qu'elles soient un support critique dans la compréhension des mécanismes neuronaux, les connaissances issues d'enregistrements unitaires chez le singe ne peuvent être transposées à l'humain qu'une fois l'identification d'homologies et de différences fonctionnelles établie à partir des mêmes approches expérimentales. Pour ce faire, nous proposons dans cette thèse de répondre aux besoins d'études comparatives entre les deux espèces dans le cadre du traitement visuel des configurations spatiales portant sur le traitement de la symétrie et le traitement configural des visages par une approche en IRMf. Une première étude menée en collaboration avec des chercheurs de l'Université de Stanford nous a permis d'étudier les réponses à des stimuli texturaux englobant des motifs de symétrie chez le macaque. Nous avons pu mettre en évidence (1) un réseau cortical de traitement de la symétrie par rotation similaire entre les primates humains et non-humains, (2) des réponses augmentant de manière paramétrique avec l'ordre de symétrie présenté (n rotations) (3) un réseau similaire de traitement de la symétrie par rotation et par réflexion chez le macaque (4) des réponses plus fortes pour des motifs symétriques à deux axes (horizontale et verticale) plutôt qu'un seul axe (horizontal). Nous avons ainsi observé que les réponses à la symétrie chez le macaque débutaient au-delà de V1, dans un réseau comprenant les aires V2, V3, V3A, V4 semblablement à l'humain mais également des réponses paramétriques à l'ordre de symétrie par rotation dans les aires V3, V4 et PITd tout comme reporté chez les sujets humains. En somme, l'ensemble de ces résultats ont mis en évidence le réseau cortical du traitement de la symétrie jusqu'alors jamais observé chez le macaque, supporté par des aires visuelles homologues à celles de l'humain. Ces résultats ouvrent de nouvelles pistes quant à la compréhension des mécanismes neuronaux unitaires par des approches plus invasives chez le singe, tout particulièrement dans l'aire V3 qui semble jouer un rôle important dans le traitement sophistiqué des paramètres de configurations spatiales. La secondé étude de ce projet de thèse visait à étudier les mécanismes de reconnaissance de l'identité faciale chez le singe à travers l'orientation configurale des visages porté par l'objectif de réaliser une comparaison inter-espèces du traitement holistique des visages. S'il est largement admis que l'humain est un expert de l'identification des visages dont les mécanismes dépendent de l'orientation dans laquelle ils sont présentés, les résultats sont bien plus contradictoires chez le singe. Pour résoudre ces contradictions, nous avons mis en place un protocole innovant visant à mesurer l'effet d'inversion chez les deux espèces qui ne nécessitait ni entrainement ni tâche comportementale. Cette étude menée en collaboration avec B. Rossion demeure en cours d'acquisition. Néanmoins, les données pourraient fournir des preuves de mécanismes fonctionnels distincts entre celles-ci, appelant à une potentielle réévaluation de l'utilisation du macaque dans l'étude et la compréhension des processus de reconnaissance de l'identité faciale chez l'humain.The processing of spatial configurations is a mechanism that constantly intervenes within the visual cortex. In this world full of regularities that are ours, it holds a prominent place in the analysis of objects in our environment, allowing us to establish spatial relationships between sets of elements to reach a global perception. While characteristics of these mechanisms have been studied in human and non-human primates, the resulting observations depend on different methodologies. In human studies, non-invasive neuroimaging methods are privileged, while more invasive technics (i.e electrophysiology) are favored in monkeys. Despite being critical in understanding neuronal mechanisms, outcomes from unit recordings in monkeys can only be transposed to humans once the identification of functional homologies and differences are established from the same experimental approaches. Tn this thesis, we propose to meet the needs of comparative studies between the two species within the visual treatment of spatial configurations framework relating to the processing of symmetry and the configural processing of faces by an fMRI approach. A first study conducted in collaboration with researchers at Stanford University allows us to investigate the responses to textural stimuli encompassing patterns of symmetry in the macaque brain. The study demonstrates (1) a similar cortical rotational symmetry processing network between human and non-human primates (2) responses increasing parametrically with the order of symmetry presented (n rotations) (3) a similar network for processing of rotational and reflection symmetry in the macaque (4) stronger responses for symmetrical patterns with two axes (horizontal and vertical) rather than a single axis (horizontal). We also observe that the responses to symmetry in the macaque begin beyond V1, in a network comprising the areas V2, V3, V3A, V4 similar to humans but also parametric responses to the order of rotation in symmetry in areas V3, V4, and PITd as reported in human subjects. Overall, all of these results highlight the cortical network of symmetry processing never observed in macaques so far, supported by visual areas homologous to those of humans. These results open up new possibilities for the understanding of unitary neuronal mechanisms by more invasive approaches in monkeys, especially in the V3 area which seems to play an important role in the sophisticated processing of spatial configuration parameters. The second study of this thesis project aims to investigate the mechanisms of facial identity recognition in monkeys through the configural orientation of faces, and carry out an interspecies comparison of holistic facial processing. It is widely accepted that humans are experts at identifying faces whose mechanisms depend on the orientation in which they are presented. However, results are much more contradictory in the monkey. To resolve these contradictions, we implement an innovative protocol to measure the face inversion effect in the two species that require no training or behavioral tasks. This study, conducted in collaboration with B. Rossion, is still in progress. Nonetheless, the futur data could provide evidence of distinct functional mechanisms between human and non-human species, calling for a potential reassessment of the use of the macaque in the study and understanding of facial identity recognition processes in humans