Cortical Dynamics of Contextually-Cued Attentive Visual Learning and Search: Spatial and Object Evidence Accumulation

How do humans use predictive contextual information to facilitate visual search? How are consistently paired scenic objects and positions learned and used to more efficiently guide search in familiar scenes? For example, a certain combination of objects can define a context for a kitchen and trigger a more efficient search for a typical object, such as a sink, in that context. A neural model, ARTSCENE Search, is developed to illustrate the neural mechanisms of such memory-based contextual learning and guidance, and to explain challenging behavioral data on positive/negative, spatial/object, and local/distant global cueing effects during visual search. The model proposes how global scene layout at a first glance rapidly forms a hypothesis about the target location. This hypothesis is then incrementally refined by enhancing target-like objects in space as a scene is scanned with saccadic eye movements. The model clarifies the functional roles of neuroanatomical, neurophysiological, and neuroimaging data in visual search for a desired goal object. In particular, the model simulates the interactive dynamics of spatial and object contextual cueing in the cortical What and Where streams starting from early visual areas through medial temporal lobe to prefrontal cortex. After learning, model dorsolateral prefrontal cortical cells (area 46) prime possible target locations in posterior parietal cortex based on goalmodulated percepts of spatial scene gist represented in parahippocampal cortex, whereas model ventral prefrontal cortical cells (area 47/12) prime possible target object representations in inferior temporal cortex based on the history of viewed objects represented in perirhinal cortex. The model hereby predicts how the cortical What and Where streams cooperate during scene perception, learning, and memory to accumulate evidence over time to drive efficient visual search of familiar scenes.CELEST, an NSF Science of Learning Center (SBE-0354378); SyNAPSE program of Defense Advanced Research Projects Agency (HR0011-09-3-0001, HR0011-09-C-0011

A theory of a saliency map in primary visual cortex (V1) tested by psychophysics of color-orientation interference in texture segmentation

It has been proposed that V1 creates a bottom-up saliency map, where saliency of any location increases with the firing rate of the most active V1 output cell responding to it, regardless the feature selectivity of the cell. Thus, a red vertical bar may have its saliency signalled by a cell tuned to red colour, or one tuned to vertical orientation, whichever cell is the most active. This theory predicts interference between colour and orientation features in texture segmentation tasks where bottom-up processes are significant. The theory not only explains existing data, but also provides a prediction. A subsequent psychophysical test confirmed the prediction by showing that segmentation of textures of oriented bars became more difficult as the colours of the bars were randomly drawn from more colour categories

Primary visual cortex as a saliency map: parameter-free prediction of behavior from V1 physiology

It has been hypothesized that neural activities in the primary visual cortex (V1) represent a saliency map of the visual field to exogenously guide attention. This hypothesis has so far provided only qualitative predictions and their confirmations. We report this hypothesis' first quantitative prediction, derived without free parameters, and its confirmation by human behavioral data. The hypothesis provides a direct link between V1 neural responses to a visual location and the saliency of that location to guide attention exogenously. In a visual input containing many bars, one of them saliently different from all the other bars which are identical to each other, saliency at the singleton's location can be measured by the shortness of the reaction time in a visual search task to find the singleton. The hypothesis predicts quantitatively the whole distribution of the reaction times to find a singleton unique in color, orientation, and motion direction from the reaction times to find other types of singletons. The predicted distribution matches the experimentally observed distribution in all six human observers. A requirement for this successful prediction is a data-motivated assumption that V1 lacks neurons tuned simultaneously to color, orientation, and motion direction of visual inputs. Since evidence suggests that extrastriate cortices do have such neurons, we discuss the possibility that the extrastriate cortices play no role in guiding exogenous attention so that they can be devoted to other functional roles like visual decoding or endogenous attention.Comment: 11 figures, 66 page

Behaviourally meaningful representations from normalisation and context-guided denoising

Many existing independent component analysis algorithms include a preprocessing stage where the inputs are sphered. This amounts to normalising the data such that all correlations between the variables are removed. In this work, I show that sphering allows very weak contextual modulation to steer the development of meaningful features. Context-biased competition has been proposed as a model of covert attention and I propose that sphering-like normalisation also allows weaker top-down bias to guide attention

Reflexive and preparatory selection and suppression of salient information in the right and left posterior parietal cortex

Attentional cues can trigger activity in the parietal cortex in anticipation of visual displays, and this activity may, in turn, induce changes in other areas of the visual cortex, hence, implementing attentional selection. In a recent TMS study [Mevorach, C., Humphreys, G. W., & Shalev, L. Opposite biases in salience-based selection for the left and right posterior parietal cortex. Nature Neuroscience, 9, 740-742, 2006b], it was shown that the posterior parietal cortex (PPC) can utilize the relative saliency (a nonspatial property) of a target and a distractor to bias visual selection. Furthermore, selection was lateralized so that the right PPC is engaged when salient information must be selected and the left PPC when the salient information must be ignored. However, it is not clear how the PPC implements these complementary forms of selection. Here we used on-line triple-pulse TMS over the right or left PPC prior to or after the onset of global/local displays. When delivered after the onset of the display, TMS to the right PPC disrupted the selection of the more salient aspect of the hierarchical letter. In contrast, left PPC TMS delivered prior to the onset of the stimulus disrupted responses to the lower saliency stimulus. These findings suggest that selection and suppression of saliency, rather than being "two sides of the same coin," are fundamentally different processes. Selection of saliency seems to operate reflexively, whereas suppression of saliency relies on a preparatory phase that "sets up" the system in order to effectively ignore saliency

The left intraparietal sulcus modulates the selection of low salient stimuli

Neuropsychological and functional imaging studies have suggested a general right hemisphere advantage for processing global visual information and a left hemisphere advantage for processing local information. In contrast, a recent transcranial magnetic stimulation study [Mevorach, C., Humphreys, G. W., & Shalev, L. Opposite biases in salience-based selection for the left and right posterior parietal cortex. Nature Neuroscience, 9, 740-742, 2006b] demonstrated that functional lateralization of selection in the parietal cortices on the basis of the relative salience of stimuli might provide an alternative explanation for previous results. In the present study, we applied a whole-brain analysis of the functional magnetic resonance signal when participants responded to either the local or the global levels of hierarchical figures. The task (respond to local or global) was crossed with the saliency of the target level (local salient, global salient) to provide, for the first time, a direct contrast between brain activation related to the stimulus level and that related to relative saliency. We found evidence for lateralization of salience-based selection but not for selection based on the level of processing. Activation along the left intraparietal sulcus (IPS) was found when a low saliency stimulus had to be selected irrespective of its level. A control task showed that this was not simply an effect of task difficulty. The data suggest a specific role for regions along the left IPS in salience-based selection, supporting the argument that previous reports of lateralized responses to local and global stimuli were contaminated by effects of saliency